# LiteRT Published articles for LiteRT. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## On-Device AI Series (Part 5): LiteRT-LM DevFeed: [On-Device AI Series (Part 5): LiteRT-LM]() Original publisher: [Read original article]() Author: Oğuzhan Aslan Published: 2026-09-14T05:59:12Z Content type: tutorial Language: en Sources: [ProAndroidDev - Medium]() Topics: [LiteRT](), [On-device AI](), [Large Language Model](), [Inference](), [Streaming](), [GPU](), [LLM evaluation / benchmarking](), [PyTorch]() Tags: [ai](), [android](), [android-development](), [benchmarking](), [gpu](), [inference](), [litert](), [llama](), [llm](), [multimodal](), [on-device](), [on-device-ai](), [programming](), [pytorch](), [qwen](), [software-development](), [streaming]() ### AI overview This tutorial explains LiteRT-LM for running large language models on-device. It covers the Engine/Session API, streaming output, system prompts, tool calling, multimodal inputs, thinking mode, and CPU-versus-GPU benchmarking. The article also discusses tradeoffs involving privacy, network independence, latency, memory, sampling configuration, and model capability compared with cloud APIs. ### Source excerpt Put your phone in airplane mode. Open the app, type a question, and watch the answer arrive one token at a time -- no spinner waiting on a network round-trip, no API key, no per-token bill, and nothing you typed ever leaving the device. LiteRT-LM removes the genuinely hard parts of running an LLM on-device -- KV-cache management, token streaming, backend selection -- but it doesn't remove your job so much as relocate it. What's left on your plate is a short, specific list: sizing a combined input+output token budget, owning your own sampling defaults, hand-building system prompts and tool calling out of raw text, and one native-library collision that presents as a SIGSEGV rather than a build error. Know those going in and the API itself is a clean three-step pattern. We'll get there in that order: Why you'd choose this runtime and what it costs you versus the cloud. The Engine/Session model you need to read the code at all. Real implementation samples -- streaming, system prompts and tool calling, multimodal inputs, thinking mode, and CPU-vs-GPU benchmarking. The anti-patterns to avoid. A developer-friendliness rating on the same rubric as Parts 1-4. Why Use LiteRT-LM? You reach for LiteRT-LM instead of hand-rolling generation on top of raw LiteRT when: You need multi-turn conversation, not single-shot inference -- session state and KV-cache bookkeeping are handled for you, and resetting a conversation is a session swap, not a model reload. You need streaming output -- token-by-token delivery for a responsive chat UI, instead of a blocking call that returns everything at once. You're choosing between CPU and GPU per device -- the explicit backend parameter turns that into a runtime decision instead of a build-time guess. You want a pre-converted model without doing your own PyTorch-to-LiteRT conversion work -- the Model Zoo covers Gemma, Qwen, Llama, and more out of the box. You're willing to own sampling -- the engine won't pick sane decoding defaults for you; that's on the ## Mastering Edge AI on Raspberry Pi with LiteRT and Gemma DevFeed: [Mastering Edge AI on Raspberry Pi with LiteRT and Gemma]() Original publisher: [Read original article]() Author: Lu Wang; Terry Heo; Naushir Patuck; José María Casanova Published: 2026-09-12T11:04:33.891311Z Content type: tutorial Language: en Sources: [Google Developers Blog]() Topics: [LiteRT](), [On-device AI](), [Local AI](), [AI, ML & Data Engineering](), [Google AI](), [cpu](), [Security, Privacy and Abuse Prevention]() Tags: [cli](), [cpu](), [edge](), [gemma](), [gpu](), [inference](), [litert](), [offline](), [on-device](), [real-time](), [robotics]() ### AI overview The article explains how to deploy Gemma models with LiteRT on a Raspberry Pi for local, real-time edge AI applications such as robotics. It highlights LiteRT-LM, CPU and GPU optimization, and reported performance figures for Gemma 4 E2B on Raspberry Pi 5. ### Source excerpt Deploying secure, real-time Edge AI on Raspberry Pi is now simplified using LiteRT and lightweight Gemma open models. LiteRT optimizes CPU and GPU performance, delivering fast token speeds for models like Gemma4, enabling real-time local reasoning for robotics. Developers can quickly convert, quantize, and run these models using the lightweight LiteRT CLI tool. Support for Hailo AI accelerators is also coming very soon. ## Running golf swing analysis on an Android device with a fine-tuned Gemma 4 model DevFeed: [Running golf swing analysis on an Android device with a fine-tuned Gemma 4 model]() Original publisher: [Read original article]() Published: 2026-08-29T23:00:00Z Content type: tutorial Language: en Sources: [John O'Reilly]() Topics: [Kotlin Multiplatform](), [gemma4](), [LiteRT](), [On-device AI](), [Fine-tuning]() Tags: [ai](), [android](), [cross-platform](), [fine-tuning](), [gemma](), [gemma-4](), [kotlin](), [kotlin-multiplatform](), [litert](), [model](), [multiplatform](), [on-device-ai]() ### AI overview This article explains how FormAI adds Android on-device golf-swing analysis using a small Gemma 4 model fine-tuned to imitate Gemini for a narrow coaching task. It covers generating training data with Gemini, fine-tuning with LoRA, converting the model for LiteRT-LM, and the current fallback to cloud analysis on other platforms. ### Source excerpt FormAI is a Kotlin Multiplatform app that analyses a video of your golf swing, basketball shot or running form and gives you coaching feedback. Up to now that has always meant uploading the video to Gemini and getting the response back over the network. We've added an option to do the golf swing analysis entirely on an Android device instead, using a small Gemma 4 model that we fine-tuned to imitate Gemini for that one task. LiteRT-LM, the runtime we use for this, is itself cross platform (Android, iOS, desktop and web), but we've only wired up the Android side so far, so this path lives in androidMain and the other targets report it as unavailable and fall back to the cloud. ## Engineering log: fine-tuning Gemma 4 E4B with LoRA to bring FormAI's coaching on-device DevFeed: [Engineering log: fine-tuning Gemma 4 E4B with LoRA to bring FormAI's coaching on-device]() Original publisher: [Read original article]() Published: 2026-08-28T23:00:00Z Content type: article Language: en Sources: [John O'Reilly]() Topics: [gemma4](), [Fine-tuning](), [lora](), [On-device AI](), [multiplatform](), [LiteRT](), [Android](), [Artificial Intelligence](), [API]() Tags: [ai](), [android](), [api](), [data](), [data-quality](), [engineering](), [fine-tuning](), [gemini](), [gemma](), [gemma-4](), [inference](), [kotlin-multiplatform](), [litert](), [lora](), [multiplatform](), [on-device]() ### AI overview This engineering log describes a prototype that distils FormAI's Gemini-based sports coaching feedback into Gemma 4 E4B fine-tuned with LoRA. The local pipeline uses seed videos, filters inadequate critiques, extracts frames, trains and merges the adapter, then converts the model to LiteRT-LM for possible on-device Android inference. ### Source excerpt ⚠ Note: this post is AI-generated. The text below was written by Claude, and documents findings from a series of Claude Code sessions working on this project -- the experiments, bugs and measurements described are ones that came out of those sessions. The engineering work is real and the numbers were measured rather than estimated, but the write-up is the model's own account of what it did, not a human's independent retelling of it. ## Is Raspberry Pi Dead? I Don't Think So DevFeed: [Is Raspberry Pi Dead? I Don't Think So]() Original publisher: [Read original article]() Author: Patrick Fromaget Published: 2026-08-26T05:00:00Z Content type: article Language: en Sources: [RaspberryTips]() Topics: [Hardware](), [Software](), [Availability](), [LiteRT]() Tags: [ai](), [availability](), [community](), [documentation](), [hardware](), [litert](), [platform](), [quick-tips](), [raspberry-pi](), [social-media](), [software](), [tutorials](), [usb]() ### AI overview The article argues that Raspberry Pi is not dead despite higher prices, past availability problems, and stronger competition from mini-PCs. It presents Raspberry Pi's continuing sales, broad hardware lineup, software, documentation, accessories, tutorials, and community as evidence that its platform and ecosystem remain active. ### Source excerpt Lots of people are saying "Raspberry Pi is dead" on social media or Reddit. If you spend some time there, you'll quickly get the feeling that Raspberry Pis are a relic of the past, no longer relevant for various reasons. I get why they might think that, but I'm not fully on board with that... ## Codename One Adds On-Device AI and Loopback MCP Support DevFeed: [Codename One Adds On-Device AI and Loopback MCP Support]() Original publisher: [Read original article]() Author: Shai Almog Published: 2026-08-02T00:00:00Z Content type: release Language: en Sources: [CodeName One]() Topics: [On-device AI](), [Model Context Protocol](), [Artificial Intelligence](), [LiteRT](), [Inference](), [Mobile](), [supply-chain-security]() Tags: [applications](), [inference](), [litert](), [mcp](), [mobile](), [on-device](), [on-device-ai](), [tensorflow-lite]() ### AI overview Codename One has added on-device vision, language, and LiteRT inference to its core, alongside a guarded loopback MCP transport for inspecting and operating applications. The article explains platform support, asynchronous OCR, model sessions, secure model downloads, and privacy limitations of local inference. ### Source excerpt Codename One now exposes on-device vision, language, and LiteRT inference in the core, while a guarded loopback MCP transport lets an LLM inspect and drive real applications. ## Push V3: One Message From Your Server to Every Surface DevFeed: [Push V3: One Message From Your Server to Every Surface]() Original publisher: [Read original article]() Author: Shai Almog Published: 2026-07-31T00:00:00Z Content type: release Language: en Sources: [CodeName One]() Topics: [releases](), [Cloud](), [Mobile](), [API](), [LiteRT](), [MCP Server]() Tags: [api](), [cloud](), [litert](), [mcp-server](), [mobile](), [release](), [releases](), [updates]() ### AI overview Codename One Push V3 adds typed messages, managed credentials, subscriptions, server-side segments, campaigns, analytics, and Surface updates. Existing push applications should test the new cloud endpoint before the planned cutover. ### Source excerpt Codename One Push V3 adds typed messages, managed credentials, segmentation, analytics, and Surface updates. Existing push apps should test the new cloud endpoint before next week's cutover. ## What's new in TensorFlow 2.20 DevFeed: [What's new in TensorFlow 2.20]() Original publisher: [Read original article]() Author: TensorFlow Blog (noreply@blogger.com) Published: 2025-08-19T16:00:00Z Content type: news Language: en Sources: [The TensorFlow Blog]() Topics: [LiteRT](), [On-device AI](), [Inference](), [Latency](), [IO](), [GPU](), [data](), [Google](), [Cloud](), [Python](), [C++](), [Kotlin]() Tags: [announcement](), [c-plus-plus](), [data](), [google](), [gpu](), [inference](), [io](), [kotlin](), [latency](), [litert](), [npu](), [on-device](), [python](), [tensorflow](), [tensorflow-core](), [tensorflow-lite]() ### AI overview TensorFlow 2.20 introduces the independent LiteRT repository as the future home for on-device inference, replacing the deprecated tf.lite module. LiteRT adds Kotlin and C++ APIs, improves NPU and GPU acceleration, and supports lower-latency inference with fewer memory copies. The release also adds tf.data input-pipeline warm-up controls and makes Google Cloud Storage support optional. ### Source excerpt Posted by the TensorFlow team TensorFlow 2.20 has been released! For ongoing updates related to the multi-backend Keras, please note that all news and releases, starting with Keras 3.0, are now published directly on keras.io. You can find a complete list of all changes in the full release notes on GitHub. tf.lite is being replaced by LiteRT The tf.lite module will be deprecated with development for on-device inference moving to a new, independent repository: LiteRT. The new APIs are available in Kotlin and C++. This code base will decouple from the TensorFlow repository and tf.lite will be removed from future TensorFlow Python packages, so we encourage migration of projects to LiteRT to receive the latest updates. More details to follow. As announced at Google I/O '25, LiteRT improves upon TFLite, particularly for NPU and GPU hardware acceleration and performance for on-device ML and AI applications. LiteRT provides a unified interface for Neural Processing Units (NPUs), removing the need to navigate vendor-specific compilers or libraries. This approach avoids many device-specific complications, boosts performance for real-time and large-model inference, and minimizes memory copies through zero-copy hardware buffer usage. For more information on the new repository and to sign up for the NPU Early Access Program, please reach out to the team at g.co/ai/LiteRT-NPU-EAP. Faster input pipeline warm-up with tf.data To help reduce latency, especially the time it takes for your model to process the first element of a dataset, we've added autotune.min_parallelism in tf.data.Options. This new option allows asynchronous dataset operations like .map and .batch to immediately start with a specified minimum level of parallelism, speeding up the initial warm-up time for your input pipelines. Changes to I/O GCS filesystem package The tensorflow-io-gcs-filesystem package for Google Cloud Storage support is now optional. Previously, it was installed, by default, with TensorFlow. If y ## What's new in TensorFlow 2.19 DevFeed: [What's new in TensorFlow 2.19]() Original publisher: [Read original article]() Author: TensorFlow Blog (noreply@blogger.com) Published: 2025-03-13T16:00:00Z Content type: release Language: en Sources: [The TensorFlow Blog]() Topics: [Tensorflow](), [LiteRT](), [releases](), [API](), [C++](), [Release notes](), [Keras](), [migration]() Tags: [2025](), [announcement](), [api](), [c-plus-plus](), [keras](), [libentensorflow](), [litert](), [migration-guide](), [pypi](), [release](), [release-notes](), [tensorflow](), [tensorflow-core](), [tensorflowlite]() ### AI overview TensorFlow 2.19 introduces C++ API changes in LiteRT, adds bfloat16 support for the tflite casting operation, and deprecates tf.lite.Interpreter in favor of ai_edge_litert.interpreter. The release also stops publishing libtensorflow packages, although they remain extractable from the PyPI package. Updates for multi-backend Keras are directed to keras.io. ### Source excerpt Posted by the TensorFlow team TensorFlow 2.19 has been released! Highlights of this release include changes to the C++ API in LiteRT, bfloat16 support for tflite casting, discontinue of releasing libtensorflow packages. Learn more by reading the full release notes. Note: Release updates on the new multi-backend Keras will be published on keras.io, starting with Keras 3.0. For more information, please see https://keras.io/keras_3/. TensorFlow Core LiteRT The public constants tflite::Interpreter:kTensorsReservedCapacity and tflite::Interpreter:kTensorsCapacityHeadroom are now const references, rather than constexpr compile-time constants. (This is to enable better API compatibility for TFLite in Play services while preserving the implementation flexibility to change the values of these constants in the future.) TF-Lite tfl.Cast op is now supporting bfloat16 in the runtime kernel. tf.lite.Interpreter gives a deprecation warning redirecting to its new location at ai_edge_litert.interpreter, as the API tf.lite.Interpreter will be deleted in TF 2.20. See the migration guide for details. Libtensorflow We have stopped publishing libtensorflow packages but it can still be unpacked from the PyPI package. ## Introducing Wake Vision: A High-Quality, Large-Scale Dataset for TinyML Computer Vision Applications DevFeed: [Introducing Wake Vision: A High-Quality, Large-Scale Dataset for TinyML Computer Vision Applications]() Original publisher: [Read original article]() Author: TensorFlow Blog (noreply@blogger.com) Published: 2024-12-05T17:00:00Z Content type: article Language: en Sources: [The TensorFlow Blog]() Topics: [Training AI Models](), [machine learning overfitting]() Tags: [ai](), [benchmarks](), [community](), [computer-vision](), [data-quality](), [datasets](), [edge](), [google-ai](), [litert](), [machine-learning](), [on-device-ai](), [person-detection](), [research](), [tensorflow-lite](), [training]() ### AI overview Wake Vision is a roughly 6-million-image dataset for TinyML person detection. It provides large and quality-focused training sets, emphasizing that accurate labels can be especially valuable for constrained models. ### Source excerpt Posted by Colby Banbury, Emil Njor, Andrea Mattia Garavagno, Vijay Janapa Reddi - Harvard University TinyML is an exciting frontier in machine learning, enabling models to run on extremely low-power devices such as microcontrollers and edge devices. However, the growth of this field has been stifled by a lack of tailored large and high-quality datasets. That's where Wake Vision comes in--a new dataset designed to accelerate research and development in TinyML. Why TinyML Needs Better Data The development of TinyML requires compact and efficient models, often only a few hundred kilobytes in size. The applications targeted by standard machine learning datasets, like ImageNet, are not well-suited for these highly constrained models. Existing datasets for TinyML, like Visual Wake Words (VWW), have laid the groundwork for progress in the field. However, their smaller size and inherent limitations pose challenges for training production-grade models. Wake Vision builds upon this foundation by providing a large, diverse, and high-quality dataset specifically tailored for person detection--the cornerstone vision task for TinyML. What Makes Wake Vision Different? Wake Vision is a new, large-scale dataset with roughly 6 million images, almost 100 times larger than VWW, the previous state-of-the-art dataset for person detection in TinyML. The dataset provides two distinct training sets: Wake Vision (Large): Prioritizes dataset size. Wake Vision (Quality): Prioritizes label quality. Wake Vision's comprehensive filtering and labeling process significantly enhances the dataset's quality. Why Data Quality Matters for TinyML Models In traditional overparameterized models, it is widely believed that data quantity matters more than data quality, as an overparameterized model can adapt to errors in the training data. But according to the image below, TinyML tells a different story: The figure above shows that high-quality labels (less error) are more beneficial for under-parameterized mo ## What's new in TensorFlow 2.18 DevFeed: [What's new in TensorFlow 2.18]() Original publisher: [Read original article]() Author: TensorFlow Blog (noreply@blogger.com) Published: 2024-10-28T19:00:00Z Content type: release Language: en Sources: [The TensorFlow Blog]() Topics: [Tensorflow](), [releases](), [LiteRT](), [NumPy](), [CUDA](), [cudnn](), [NCCL](), [Google]() Tags: [announcement](), [cuda](), [cudnn](), [gpu](), [litert](), [migration](), [nccl](), [nvidia](), [nvidia-rtx](), [python](), [release](), [releases](), [reproducible-builds](), [tensorflow](), [tensorflow-core]() ### AI overview TensorFlow 2.18 introduces NumPy 2.0 compatibility updates, transitions TFLite development to the LiteRT repository, and adds hermetic CUDA, cuDNN, and NCCL dependencies for more reproducible source builds. Binary packages add dedicated kernels for compute capability 8.9 GPUs, including NVIDIA RTX 40 series, L4, and L40, while dropping precompiled support for Maxwell GPUs. ### Source excerpt Posted by the TensorFlow team TensorFlow 2.18 has been released! Highlights of this release (and 2.17) include NumPy 2.0, LiteRT repository, CUDA Update, Hermetic CUDA and more. For the full release notes, please click here. Note: Release updates on the new multi-backend Keras will be published on keras.io, starting with Keras 3.0. For more information, please see https://keras.io/keras_3/. TensorFlow Core NumPy 2.0 The upcoming TensorFlow 2.18 release will include support for NumPy 2.0. While the majority of TensorFlow APIs will function seamlessly with NumPy 2.0, this may break some edge cases of usage, e.g., out-of-boundary conversion errors and numpy scalar representation errors. You can consult the following common solutions. Note that NumPy's type promotion rules have been changed (See NEP 50 for details). This may change the precision at which computations happen, leading either to type errors or to numerical changes to results. Please see the NumPy 2 migration guide. We've updated some TensorFlow tensor APIs to maintain compatibility with NumPy 2.0 while preserving the out-of-boundary conversion behavior in NumPy 1.x. LiteRT Repository We're making some changes to how LiteRT (formerly known as TFLite) is developed. Over the coming months, we'll be gradually transitioning TFLite's codebase to LiteRT. Once the migration is complete, we'll start accepting contributions directly through the LiteRT repository. There will no longer be any binary TFLite releases and developers should switch to LiteRT for the latest updates. Hermetic CUDA If you build TensorFlow from source, Bazel will now download specific versions of CUDA, CUDNN and NCCL distributions, and then use those tools as dependencies in various Bazel targets. This enables more reproducible builds for Google ML projects and supported CUDA versions because the build no longer relies on the locally installed versions. More details are provided here. CUDA Update TensorFlow binary distributions now ship with d