# LiteRT Google's on-device runtime for deploying high-performance machine-learning and generative-AI models on edge platforms. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## On-Device AI Series (Part 5): LiteRT-LM DevFeed: [On-Device AI Series (Part 5): LiteRT-LM]() Original publisher: [Read original article]() Author: Oğuzhan Aslan Published: 2026-09-14T05:59:12Z Content type: tutorial Language: en Sources: [ProAndroidDev - Medium]() Topics: [LiteRT](), [On-device AI](), [Large Language Model](), [Inference](), [Streaming](), [GPU](), [LLM evaluation / benchmarking](), [PyTorch]() Tags: [ai](), [android](), [android-development](), [benchmarking](), [gpu](), [inference](), [litert](), [llama](), [llm](), [multimodal](), [on-device](), [on-device-ai](), [programming](), [pytorch](), [qwen](), [software-development](), [streaming]() ### AI overview This tutorial explains LiteRT-LM for running large language models on-device. It covers the Engine/Session API, streaming output, system prompts, tool calling, multimodal inputs, thinking mode, and CPU-versus-GPU benchmarking. The article also discusses tradeoffs involving privacy, network independence, latency, memory, sampling configuration, and model capability compared with cloud APIs. ### Source excerpt Put your phone in airplane mode. Open the app, type a question, and watch the answer arrive one token at a time -- no spinner waiting on a network round-trip, no API key, no per-token bill, and nothing you typed ever leaving the device. LiteRT-LM removes the genuinely hard parts of running an LLM on-device -- KV-cache management, token streaming, backend selection -- but it doesn't remove your job so much as relocate it. What's left on your plate is a short, specific list: sizing a combined input+output token budget, owning your own sampling defaults, hand-building system prompts and tool calling out of raw text, and one native-library collision that presents as a SIGSEGV rather than a build error. Know those going in and the API itself is a clean three-step pattern. We'll get there in that order: Why you'd choose this runtime and what it costs you versus the cloud. The Engine/Session model you need to read the code at all. Real implementation samples -- streaming, system prompts and tool calling, multimodal inputs, thinking mode, and CPU-vs-GPU benchmarking. The anti-patterns to avoid. A developer-friendliness rating on the same rubric as Parts 1-4. Why Use LiteRT-LM? You reach for LiteRT-LM instead of hand-rolling generation on top of raw LiteRT when: You need multi-turn conversation, not single-shot inference -- session state and KV-cache bookkeeping are handled for you, and resetting a conversation is a session swap, not a model reload. You need streaming output -- token-by-token delivery for a responsive chat UI, instead of a blocking call that returns everything at once. You're choosing between CPU and GPU per device -- the explicit backend parameter turns that into a runtime decision instead of a build-time guess. You want a pre-converted model without doing your own PyTorch-to-LiteRT conversion work -- the Model Zoo covers Gemma, Qwen, Llama, and more out of the box. You're willing to own sampling -- the engine won't pick sane decoding defaults for you; that's on the ## Mastering Edge AI on Raspberry Pi with LiteRT and Gemma DevFeed: [Mastering Edge AI on Raspberry Pi with LiteRT and Gemma]() Original publisher: [Read original article]() Author: Lu Wang; Terry Heo; Naushir Patuck; José María Casanova Published: 2026-09-12T11:04:33.891311Z Content type: tutorial Language: en Sources: [Google Developers Blog]() Topics: [LiteRT](), [On-device AI](), [Local AI](), [AI, ML & Data Engineering](), [Google AI](), [cpu](), [Security, Privacy and Abuse Prevention]() Tags: [cli](), [cpu](), [edge](), [gemma](), [gpu](), [inference](), [litert](), [offline](), [on-device](), [real-time](), [robotics]() ### AI overview The article explains how to deploy Gemma models with LiteRT on a Raspberry Pi for local, real-time edge AI applications such as robotics. It highlights LiteRT-LM, CPU and GPU optimization, and reported performance figures for Gemma 4 E2B on Raspberry Pi 5. ### Source excerpt Deploying secure, real-time Edge AI on Raspberry Pi is now simplified using LiteRT and lightweight Gemma open models. LiteRT optimizes CPU and GPU performance, delivering fast token speeds for models like Gemma4, enabling real-time local reasoning for robotics. Developers can quickly convert, quantize, and run these models using the lightweight LiteRT CLI tool. Support for Hailo AI accelerators is also coming very soon. ## Announcing ADK for Kotlin 1.0: Building Production-Ready AI Agents in Kotlin, Android, and Beyond DevFeed: [Announcing ADK for Kotlin 1.0: Building Production-Ready AI Agents in Kotlin, Android, and Beyond]() Original publisher: [Read original article]() Author: Guillaume Laforge Published: 2026-09-12T11:04:33.891311Z Content type: release Language: en Sources: [Google Developers Blog]() Topics: [Artificial Intelligence](), [Kotlin](), [AI Development](), [Android](), [Kotlin Multiplatform](), [multiplatform](), [Orchestration](), [Google](), [Persistence](), [Agent Skill](), [LiteRT](), [Cloud]() Tags: [agent-skill](), [agents](), [ai](), [ai-agent](), [ai-agents](), [android](), [building](), [database](), [development-kit](), [firebase](), [incident](), [kotlin](), [multiplatform](), [orchestration](), [persistence](), [production]() ### AI overview Google announces the 1.0 general availability release of the Agent Development Kit (ADK) for Kotlin, a production-ready toolkit for building multi-agent applications with Kotlin, Java, and Android. Built on Kotlin Multiplatform, it provides feature parity with the ADK 1.0 Core and adds Android-first extensions for on-device agents with LiteRT-LM and ML Kit, hybrid cloud workflows through Firebase AI Logic, and state persistence with Room and AppSearch. The release also includes type-safe, compile-time function calling through KSP and declarative agent skills. ### Source excerpt Google has officially released version 1.0 of the Agent Development Kit (ADK) for Kotlin, achieving full feature parity with the Python and Java ADK cores to enable idiomatic, multi-agent AI development. Built on Kotlin Multiplatform (KMP), the framework leverages Kotlin Symbol Processing (KSP) for zero-reflection, type-safe function calling, alongside advanced orchestration capabilities like human-in-the-loop workflows and context compaction. Additionally, the release introduces a robust suite of Android-first extensions, allowing mobile developers to integrate local models via LiteRT-LM, cloud reasoning through Firebase AI, session persistence using Room, and semantic memory powered by AppSearch. ## Running golf swing analysis on an Android device with a fine-tuned Gemma 4 model DevFeed: [Running golf swing analysis on an Android device with a fine-tuned Gemma 4 model]() Original publisher: [Read original article]() Published: 2026-08-29T23:00:00Z Content type: tutorial Language: en Sources: [John O'Reilly]() Topics: [Kotlin Multiplatform](), [gemma4](), [LiteRT](), [On-device AI](), [Fine-tuning]() Tags: [ai](), [android](), [cross-platform](), [fine-tuning](), [gemma](), [gemma-4](), [kotlin](), [kotlin-multiplatform](), [litert](), [model](), [multiplatform](), [on-device-ai]() ### AI overview This article explains how FormAI adds Android on-device golf-swing analysis using a small Gemma 4 model fine-tuned to imitate Gemini for a narrow coaching task. It covers generating training data with Gemini, fine-tuning with LoRA, converting the model for LiteRT-LM, and the current fallback to cloud analysis on other platforms. ### Source excerpt FormAI is a Kotlin Multiplatform app that analyses a video of your golf swing, basketball shot or running form and gives you coaching feedback. Up to now that has always meant uploading the video to Gemini and getting the response back over the network. We've added an option to do the golf swing analysis entirely on an Android device instead, using a small Gemma 4 model that we fine-tuned to imitate Gemini for that one task. LiteRT-LM, the runtime we use for this, is itself cross platform (Android, iOS, desktop and web), but we've only wired up the Android side so far, so this path lives in androidMain and the other targets report it as unavailable and fall back to the cloud. ## Engineering log: fine-tuning Gemma 4 E4B with LoRA to bring FormAI's coaching on-device DevFeed: [Engineering log: fine-tuning Gemma 4 E4B with LoRA to bring FormAI's coaching on-device]() Original publisher: [Read original article]() Published: 2026-08-28T23:00:00Z Content type: article Language: en Sources: [John O'Reilly]() Topics: [gemma4](), [Fine-tuning](), [lora](), [On-device AI](), [multiplatform](), [LiteRT](), [Android](), [Artificial Intelligence](), [API]() Tags: [ai](), [android](), [api](), [data](), [data-quality](), [engineering](), [fine-tuning](), [gemini](), [gemma](), [gemma-4](), [inference](), [kotlin-multiplatform](), [litert](), [lora](), [multiplatform](), [on-device]() ### AI overview This engineering log describes a prototype that distils FormAI's Gemini-based sports coaching feedback into Gemma 4 E4B fine-tuned with LoRA. The local pipeline uses seed videos, filters inadequate critiques, extracts frames, trains and merges the adapter, then converts the model to LiteRT-LM for possible on-device Android inference. ### Source excerpt ⚠ Note: this post is AI-generated. The text below was written by Claude, and documents findings from a series of Claude Code sessions working on this project -- the experiments, bugs and measurements described are ones that came out of those sessions. The engineering work is real and the numbers were measured rather than estimated, but the write-up is the model's own account of what it did, not a human's independent retelling of it. ## Codename One Adds On-Device AI and Loopback MCP Support DevFeed: [Codename One Adds On-Device AI and Loopback MCP Support]() Original publisher: [Read original article]() Author: Shai Almog Published: 2026-08-02T00:00:00Z Content type: release Language: en Sources: [CodeName One]() Topics: [On-device AI](), [Model Context Protocol](), [Artificial Intelligence](), [LiteRT](), [Inference](), [Mobile](), [supply-chain-security]() Tags: [applications](), [inference](), [litert](), [mcp](), [mobile](), [on-device](), [on-device-ai](), [tensorflow-lite]() ### AI overview Codename One has added on-device vision, language, and LiteRT inference to its core, alongside a guarded loopback MCP transport for inspecting and operating applications. The article explains platform support, asynchronous OCR, model sessions, secure model downloads, and privacy limitations of local inference. ### Source excerpt Codename One now exposes on-device vision, language, and LiteRT inference in the core, while a guarded loopback MCP transport lets an LLM inspect and drive real applications. ## Push V3: One Message From Your Server to Every Surface DevFeed: [Push V3: One Message From Your Server to Every Surface]() Original publisher: [Read original article]() Author: Shai Almog Published: 2026-07-31T00:00:00Z Content type: release Language: en Sources: [CodeName One]() Topics: [releases](), [Cloud](), [Mobile](), [API](), [LiteRT](), [MCP Server]() Tags: [api](), [cloud](), [litert](), [mcp-server](), [mobile](), [release](), [releases](), [updates]() ### AI overview Codename One Push V3 adds typed messages, managed credentials, subscriptions, server-side segments, campaigns, analytics, and Surface updates. Existing push applications should test the new cloud endpoint before the planned cutover. ### Source excerpt Codename One Push V3 adds typed messages, managed credentials, segmentation, analytics, and Surface updates. Existing push apps should test the new cloud endpoint before next week's cutover. ## What's new in TensorFlow 2.20 DevFeed: [What's new in TensorFlow 2.20]() Original publisher: [Read original article]() Author: TensorFlow Blog (noreply@blogger.com) Published: 2025-08-19T16:00:00Z Content type: news Language: en Sources: [The TensorFlow Blog]() Topics: [LiteRT](), [On-device AI](), [Inference](), [Latency](), [IO](), [GPU](), [data](), [Google](), [Cloud](), [Python](), [C++](), [Kotlin]() Tags: [announcement](), [c-plus-plus](), [data](), [google](), [gpu](), [inference](), [io](), [kotlin](), [latency](), [litert](), [npu](), [on-device](), [python](), [tensorflow](), [tensorflow-core](), [tensorflow-lite]() ### AI overview TensorFlow 2.20 introduces the independent LiteRT repository as the future home for on-device inference, replacing the deprecated tf.lite module. LiteRT adds Kotlin and C++ APIs, improves NPU and GPU acceleration, and supports lower-latency inference with fewer memory copies. The release also adds tf.data input-pipeline warm-up controls and makes Google Cloud Storage support optional. ### Source excerpt Posted by the TensorFlow team TensorFlow 2.20 has been released! For ongoing updates related to the multi-backend Keras, please note that all news and releases, starting with Keras 3.0, are now published directly on keras.io. You can find a complete list of all changes in the full release notes on GitHub. tf.lite is being replaced by LiteRT The tf.lite module will be deprecated with development for on-device inference moving to a new, independent repository: LiteRT. The new APIs are available in Kotlin and C++. This code base will decouple from the TensorFlow repository and tf.lite will be removed from future TensorFlow Python packages, so we encourage migration of projects to LiteRT to receive the latest updates. More details to follow. As announced at Google I/O '25, LiteRT improves upon TFLite, particularly for NPU and GPU hardware acceleration and performance for on-device ML and AI applications. LiteRT provides a unified interface for Neural Processing Units (NPUs), removing the need to navigate vendor-specific compilers or libraries. This approach avoids many device-specific complications, boosts performance for real-time and large-model inference, and minimizes memory copies through zero-copy hardware buffer usage. For more information on the new repository and to sign up for the NPU Early Access Program, please reach out to the team at g.co/ai/LiteRT-NPU-EAP. Faster input pipeline warm-up with tf.data To help reduce latency, especially the time it takes for your model to process the first element of a dataset, we've added autotune.min_parallelism in tf.data.Options. This new option allows asynchronous dataset operations like .map and .batch to immediately start with a specified minimum level of parallelism, speeding up the initial warm-up time for your input pipelines. Changes to I/O GCS filesystem package The tensorflow-io-gcs-filesystem package for Google Cloud Storage support is now optional. Previously, it was installed, by default, with TensorFlow. If y ## What's new in TensorFlow 2.19 DevFeed: [What's new in TensorFlow 2.19]() Original publisher: [Read original article]() Author: TensorFlow Blog (noreply@blogger.com) Published: 2025-03-13T16:00:00Z Content type: release Language: en Sources: [The TensorFlow Blog]() Topics: [Tensorflow](), [LiteRT](), [releases](), [API](), [C++](), [Release notes](), [Keras](), [migration]() Tags: [2025](), [announcement](), [api](), [c-plus-plus](), [keras](), [libentensorflow](), [litert](), [migration-guide](), [pypi](), [release](), [release-notes](), [tensorflow](), [tensorflow-core](), [tensorflowlite]() ### AI overview TensorFlow 2.19 introduces C++ API changes in LiteRT, adds bfloat16 support for the tflite casting operation, and deprecates tf.lite.Interpreter in favor of ai_edge_litert.interpreter. The release also stops publishing libtensorflow packages, although they remain extractable from the PyPI package. Updates for multi-backend Keras are directed to keras.io. ### Source excerpt Posted by the TensorFlow team TensorFlow 2.19 has been released! Highlights of this release include changes to the C++ API in LiteRT, bfloat16 support for tflite casting, discontinue of releasing libtensorflow packages. Learn more by reading the full release notes. Note: Release updates on the new multi-backend Keras will be published on keras.io, starting with Keras 3.0. For more information, please see https://keras.io/keras_3/. TensorFlow Core LiteRT The public constants tflite::Interpreter:kTensorsReservedCapacity and tflite::Interpreter:kTensorsCapacityHeadroom are now const references, rather than constexpr compile-time constants. (This is to enable better API compatibility for TFLite in Play services while preserving the implementation flexibility to change the values of these constants in the future.) TF-Lite tfl.Cast op is now supporting bfloat16 in the runtime kernel. tf.lite.Interpreter gives a deprecation warning redirecting to its new location at ai_edge_litert.interpreter, as the API tf.lite.Interpreter will be deleted in TF 2.20. See the migration guide for details. Libtensorflow We have stopped publishing libtensorflow packages but it can still be unpacked from the PyPI package. ## What's new in TensorFlow 2.18 DevFeed: [What's new in TensorFlow 2.18]() Original publisher: [Read original article]() Author: TensorFlow Blog (noreply@blogger.com) Published: 2024-10-28T19:00:00Z Content type: release Language: en Sources: [The TensorFlow Blog]() Topics: [Tensorflow](), [releases](), [LiteRT](), [NumPy](), [CUDA](), [cudnn](), [NCCL](), [Google]() Tags: [announcement](), [cuda](), [cudnn](), [gpu](), [litert](), [migration](), [nccl](), [nvidia](), [nvidia-rtx](), [python](), [release](), [releases](), [reproducible-builds](), [tensorflow](), [tensorflow-core]() ### AI overview TensorFlow 2.18 introduces NumPy 2.0 compatibility updates, transitions TFLite development to the LiteRT repository, and adds hermetic CUDA, cuDNN, and NCCL dependencies for more reproducible source builds. Binary packages add dedicated kernels for compute capability 8.9 GPUs, including NVIDIA RTX 40 series, L4, and L40, while dropping precompiled support for Maxwell GPUs. ### Source excerpt Posted by the TensorFlow team TensorFlow 2.18 has been released! Highlights of this release (and 2.17) include NumPy 2.0, LiteRT repository, CUDA Update, Hermetic CUDA and more. For the full release notes, please click here. Note: Release updates on the new multi-backend Keras will be published on keras.io, starting with Keras 3.0. For more information, please see https://keras.io/keras_3/. TensorFlow Core NumPy 2.0 The upcoming TensorFlow 2.18 release will include support for NumPy 2.0. While the majority of TensorFlow APIs will function seamlessly with NumPy 2.0, this may break some edge cases of usage, e.g., out-of-boundary conversion errors and numpy scalar representation errors. You can consult the following common solutions. Note that NumPy's type promotion rules have been changed (See NEP 50 for details). This may change the precision at which computations happen, leading either to type errors or to numerical changes to results. Please see the NumPy 2 migration guide. We've updated some TensorFlow tensor APIs to maintain compatibility with NumPy 2.0 while preserving the out-of-boundary conversion behavior in NumPy 1.x. LiteRT Repository We're making some changes to how LiteRT (formerly known as TFLite) is developed. Over the coming months, we'll be gradually transitioning TFLite's codebase to LiteRT. Once the migration is complete, we'll start accepting contributions directly through the LiteRT repository. There will no longer be any binary TFLite releases and developers should switch to LiteRT for the latest updates. Hermetic CUDA If you build TensorFlow from source, Bazel will now download specific versions of CUDA, CUDNN and NCCL distributions, and then use those tools as dependencies in various Bazel targets. This enables more reproducible builds for Google ML projects and supported CUDA versions because the build no longer relies on the locally installed versions. More details are provided here. CUDA Update TensorFlow binary distributions now ship with d ## Half-precision Inference Doubles On-Device Inference Performance DevFeed: [Half-precision Inference Doubles On-Device Inference Performance]() Original publisher: [Read original article]() Author: TensorFlow Blog (noreply@blogger.com) Published: 2023-11-29T18:00:00Z Content type: release Language: en Sources: [The TensorFlow Blog]() Topics: [Inference Performance](), [On-device AI](), [LiteRT](), [cpu](), [AI, ML & Data Engineering]() Tags: [ai](), [announcement](), [benchmarks](), [cpu](), [how-to](), [inference](), [inference-performance](), [learn](), [on-device](), [tensorflow-lite]() ### AI overview TensorFlow Lite and XNNPack add generally available FP16 inference for supported ARM CPUs, reporting close to 2x speedups for floating-point models in production and benchmarked mobile and laptop environments. ### Source excerpt Posted by Marat Dukhan and Frank Barchard, Software Engineers CPUs deliver the widest reach for ML inference and remain the default target for TensorFlow Lite. Consequently, improving CPU inference performance is a top priority, and we are excited to announce that we doubled floating-point inference performance in TensorFlow Lite's XNNPack backend by enabling half-precision inference on ARM CPUs. This means that more AI powered features may be deployed to older and lower tier devices. Traditionally, TensorFlow Lite supported two kinds of numerical computations in machine learning models: a) floating-point using IEEE 754 single-precision (32-bit) format and b) quantized using low-precision integers. While single-precision floating-point numbers provide maximum flexibility and ease of use, they come at the cost of 4X overhead in storage and memory and exhibit a performance overhead compared to 8-bit integer computations. In contrast, half-precision (FP16) floating-point numbers pose an interesting alternative balancing ease-of-use and performance: the processor needs to transfer twice fewer bytes and each vector operation produces twice more elements. By virtue of this property, FP16 inference paves the way for 2X speedup for floating-point models compared to the traditional FP32 way. For a long time FP16 inference on CPUs primarily remained a research topic, as the lack of hardware support for FP16 computations limited production use-cases. However, around 2017 new mobile chipsets started to include support for native FP16 computations, and by now most mobile phones, both on the high-end and the low-end. Building upon this broad availability, we are pleased to announce the general availability for half-precision inference in TensorFlow Lite and XNNPack. Performance Improvements Half-precision inference has already been battle-tested in production across Google Assistant, Google Meet, YouTube, and ML Kit, and demonstrated close to 2X speedups across a wide range of ne