https://llmobile.news/tags/litert/A concise news ticker covering AI on mobile devices, local models, apps and hardware.en-GBhttps://llmobile.news/ticker/functiongemma-mobile-actions-extended/https://llmobile.news/ticker/functiongemma-mobile-actions-extended/Sat, 26 Sep 2026 21:37:00 +0200Google’s FunctionGemma 270M-it reaches 76.5% end-to-end accuracy on mobile device commands once it is fine-tuned on a synthetic dataset, up from 29.3% for the base model on the same held-out evaluation set. The paper behind the fine-tune introduces MobileActionsExtended, a set of about 9,500 records covering 15 device-control categories. UGrowAI released the weights, the datasets, the training code and a working Android app alongside it. The 15 categories cover toggling lights, sending email and SMS, placing calls, reading battery and Bluetooth status, taking pictures and screenshots, adjusting screen brightness, and listing or opening applications. About one third of the prompts require more than one call. A second model trained jointly on the new data and Google’s own mobile actions dataset holds that extended score and reaches 82.3% on Google’s benchmark, which the authors describe as an 8 percentage point trade-off in return for doubling category coverage. The figures come from greedy decoding with no schema-constrained decoding, so the model has to emit a valid call on its own rather than being forced into the right format. FunctionGemma 270M end-to-end accuracy on mobile function calling Benchmark FunctionGemma 270M (fine-tuned) Base FunctionGemma 270M MobileActionsExtended 76.48 29.33 MobileActionsGoogle 60.15 58.79 MobileActionsExtended (combined model) 76.48 29.33 MobileActionsGoogle (combined model) 82.31 58.79 Strict exact-match end-to-end accuracy in percent on the two held-out evaluation sets, all figures according to UGrowAI and measured on unquantized checkpoints. Google's own Mobile Actions specialist scores 90.32 on MobileActionsGoogle and 17.20 on MobileActionsExtended. The demo app, Handy Chat, runs the model on a Google Pixel 8 Pro through the LiteRT-LM CPU backend rather than the GPU, with an exported bundle of 285.6 MB. The training data is synthetic rather than collected from real users, and the dataset and the demo cover only English and Android. Both fine-tuned models and both datasets are on Hugging Face under the UGrowAI account, with GitHub repositories for the fine-tuning pipeline and a downloadable APK. Source: https://arxiv.org/abs/2609.25373 Read the article: https://llmobile.news/ticker/functiongemma-mobile-actions-extended/Tool callingResearchAndroidLiteRThttps://llmobile.news/ticker/antigravity-sdk-local-models/https://llmobile.news/ticker/antigravity-sdk-local-models/Sat, 26 Sep 2026 12:58:00 +0200Google has given the Antigravity SDK the ability to drive agents with models running on the developer’s own machine, so a full agentic loop can execute without an API call. The featured configuration runs Gemma 4 26B A4B through LiteRT, and Sachin Kotwani and Taylor Mullen recommend a machine with more than 24 GB of VRAM or unified memory. Google names cost, privacy and offline resiliency as the reasons to keep the loop local. Two configuration classes carry the feature. LiteRTAgentConfig points the agent at a local LiteRT model file, while LocalOpenAIAgentConfig connects to any OpenAI-compatible server, with Ollama, LM Studio and vLLM named as examples. The SDK repository installs both with pip install google-antigravity litert-lm, states that the setup works best with gemma-4-26B-A4B-it-gpu.litertlm, and suggests a 64K context size. That checkpoint is a mixture-of-experts model, where only part of the weights is active per token, with 26 billion parameters in total and roughly 4 billion active, and it ships at about 15.8 GB as a quantisation-aware trained int4 build with float activations. A hybrid arrangement Google calls the Architect-Builder pattern splits the work between cloud and device. Gemini 3.8 Flash plans in the cloud and hands the steps to a local swarm of Gemma 4 26B instances, which Google demonstrates on a task that audits and patches three vulnerable Python modules. The run consumed 95 cloud tokens against 3,322 local ones, putting 97.2% of the tokens on the machine, with only filenames leaving it. Your browser does not support this video. This video could not be loaded. Use the link below to open it directly. Google's recording of the Architect-Builder run, with Gemini 3.8 Flash planning and local Gemma 4 instances patching the code. Open video Throughput depends heavily on the hardware. The LiteRT community model card reports 4,487 prefill and 86 decode tokens per second on an Nvidia RTX 4090 under Linux, 1,430 prefill and 77 decode on an M4 Max, and 1,676 prefill and 76 decode on an RTX 5080 under Windows, with the macOS run taking about 16.2 GB of GPU memory. The model is published under Apache 2.0, and a second example in the announcement has the local agent write a terminal dashboard with the psutil and rich libraries that tracks CPU load, memory use and the five most memory-hungry processes. Source: https://developers.googleblog.com/introducing-support-for-local-ai-models-in-the-antigravity-sdk/ Read the article: https://llmobile.news/ticker/antigravity-sdk-local-models/AgentsDeveloper toolsLiteRTGPUhttps://llmobile.news/ticker/minicpm5-2b/https://llmobile.news/ticker/minicpm5-2b/Mon, 07 Sep 2026 10:00:00 +0200OpenBMB has released MiniCPM5-2B, published on Hugging Face and dated 7 September 2026 in the project changelog. The model card describes a dense Transformer “built for on-device, local deployment, and resource-constrained scenarios”, with 2,516,756,480 parameters of which 1,981,982,720 are non-embedding. The architecture uses 42 layers with grouped-query attention at 16 query heads and 2 key-value heads, and supports a context length of 131,072 tokens. It is the second model in the MiniCPM5 series after MiniCPM5-1B, scaling the same recipe to a larger footprint. OpenBMB reports an average score of 53.9 across its evaluation dimensions, including 69.1 on LiveCodeBench, 86.5 on AIME 2025 and 68.1 on NoLiMa. The release is licensed under Apache-2.0 and ships in BF16, GGUF, GPTQ 4-bit, MLX for Apple Silicon and LiteRT-LM for mobile, plus a DSpark draft model for speculative decoding. Source: https://huggingface.co/openbmb/MiniCPM5-2B Read the article: https://llmobile.news/ticker/minicpm5-2b/Open weightsQuantisationLiteRThttps://llmobile.news/ticker/litert-lm-on-device-genai/https://llmobile.news/ticker/litert-lm-on-device-genai/Tue, 19 May 2026 18:00:00 +0200Google has detailed LiteRT-LM, the runtime it uses to run Gemma 4 on-device in Chrome, ChromeOS, the Pixel Watch and the Google AI Edge Gallery app. It sits on top of LiteRT, formerly TensorFlow Lite, and adds GenAI libraries over the XNNPACK and ML Drift kernels. Running Gemma 4 E2B without multi-token prediction, Google measures 52 decode tokens per second on the Android GPU through OpenCL, 56 on iOS through Metal, and up to 76 in the browser on WebGPU. The Android figures come from a Samsung S26 Ultra, iOS from an iPhone 17 Pro, and web from Chrome on a 2024 MacBook Pro with an M4 Max. Chart: Google. The published comparison puts LiteRT-LM prefill at 3,808 tokens per second against 292 for llama.cpp on the Android GPU, 2,878 against 1,493 for MLX on iOS, and 4,182 against 1,127 for ONNX on the web. Decode figures in the same chart are 52 against 19, 56 against 32 and 76 against 29. Multi-token prediction raises decode further. Google reports up to a 2.2x speedup, taking Gemma 4 E2B from 52 to 85 tokens per second and E4B from 21 to 47, measured on a Samsung S26 Ultra GPU. The drafter and the primary model run on the same hardware block so the shared KV cache and activations stay in local memory. Chart: Google. On memory, LiteRT-LM keeps per-layer embeddings out of memory and loads image and audio encoders only when a task needs them. Google states that the 2.58 GB Gemma 4 E2B model runs with a physical footprint of 607 MB on Apple mobile CPUs using XNNPACK weight caching. For agentic use the runtime supports Thinking Mode, which developers can stream to the interface or strip to save KV cache space: By dedicating a scratchpad for step-by-step reasoning before the model commits to an action, LiteRT-LM can significantly improve the output quality. Constrained decoding enforces JSON schemas on tool payloads, and native function calling pauses execution, returns the tool-call request to the app and resumes on the tool’s output. Session save and restore serialise KV cache state so long conversations skip the prefill phase on return. Beyond the existing Kotlin and C++ interfaces on Android, Google is adding an open-source Swift API for Apple platforms and a JavaScript API for the web through WebAssembly and WebGPU. Source: https://developers.googleblog.com/blazing-fast-on-device-genai-with-litert-lm/ Read the article: https://llmobile.news/ticker/litert-lm-on-device-genai/LiteRTBenchmarksAndroidhttps://llmobile.news/ticker/gemma-4-edge-agent-skills/https://llmobile.news/ticker/gemma-4-edge-agent-skills/Thu, 02 Apr 2026 18:00:00 +0200Google DeepMind has launched Gemma 4 under the Apache 2.0 licence, and the Google AI Edge team has described how the E2B and E4B variants run on edge hardware. The models support more than 140 languages and a 128K context window. Gemma 4 enables multi-step planning, autonomous action, offline code generation, and even audio-visual processing, all without specialized fine-tuning. Image: Google. The Google AI Edge Gallery app on iOS and Android gains Agent Skills, which Google calls one of the first applications to run multi-step, autonomous agentic workflows entirely on-device. The examples given are querying Wikipedia, turning speech input into charts and flashcards, pairing photos with generated music, and building a small app that plays animal calls through conversation alone. For in-app deployment Google points to LiteRT-LM. It reports that Gemma 4 E2B runs in under 1.5 GB of memory on some devices using 2-bit and 4-bit weights with memory-mapped per-layer embeddings, and that GPU optimisations process 4,000 input tokens across two skills in under 3 seconds. Constrained decoding enforces structured output, and dynamic context lengths let a single model span CPUs and GPUs. On smaller hardware, LiteRT-LM reaches 133 prefill and 7.6 decode tokens per second on a Raspberry Pi 5 CPU, and 3,700 prefill and 31 decode tokens per second on the NPU of a Qualcomm Dragonwing IQ8, the processor behind the Arduino VENTUNO Q. Gemma 4 is available with CPU and GPU support on Android and iOS, system-wide on Android through the new AICore Developer Preview, on Windows, Linux and macOS via Metal, and in the browser through WebGPU. Google also released a Python package and a litert-lm CLI for Linux, macOS and Raspberry Pi, with tool calling included. Source: https://developers.googleblog.com/bring-state-of-the-art-agentic-skills-to-the-edge-with-gemma-4/ Read the article: https://llmobile.news/ticker/gemma-4-edge-agent-skills/AgentsOpen weightsLiteRThttps://llmobile.news/ticker/googleai-edge-gallery/https://llmobile.news/ticker/googleai-edge-gallery/Sun, 18 May 2025 00:21:00 +0200Google’s AI Edge team has published Google AI Edge Gallery, an app that runs generative models entirely on the device. The home screen offers three tasks, AI Chat for multi-turn conversation, Ask Image for questions about a photo from the camera or the gallery, and Prompt Lab for single-turn work, which covers free-form prompts, rewriting text in a chosen tone, summarising and generating code. Once a model file is on the phone, the app needs no connection and Google states that all processing happens on the device. Screenshot: Google. The third screen shows the per-model settings, including the GPU and CPU toggle. Models come from the LiteRT Community organisation on Hugging Face, which the app’s welcome text links to directly. The first release listed three downloads, Gemma 3 1B IT at 4-bit in 554.7 MB, Hammer 2.1 1.5B at 8-bit and Qwen2.5 1.5B Instruct at 8-bit, the latter two at roughly 1.6 GB each. Gated repositories ask for a Hugging Face access token, which the settings dialog stores and can clear again. Anything outside that list has to arrive as a LiteRT .task bundle picked from device storage, and the app tells the user that no other file type is supported. Each model then carries a settings dialog with an accelerator switch between GPU and CPU, next to max tokens, top-k, top-p and temperature. The shipped list sets a different default per model, GPU first for Gemma 3 1B and CPU only for Qwen2.5 1.5B Instruct. After every reply the app prints four of its own measurements, time to first token in seconds, prefill speed and decode speed in tokens per second, and total latency in seconds. It derives them from the session itself, dividing the prompt’s token count by the time the first token took and counting decoded tokens against the clock that starts with that first token, so the numbers describe whatever hardware the app happens to be running on. Google calls this an “experimental Alpha release” and ships it under the Apache 2.0 licence. There is no store listing at this point, so installing means taking the APK from the project’s releases page on a device running Android 8.0 or newer, with a project wiki covering the corporate-device case. The README lists Android as available now and iOS as coming soon. Underneath, the app builds on LiteRT and Google’s MediaPipe LLM inference API for Android. Source: https://github.com/google-ai-edge/gallery Read the article: https://llmobile.news/ticker/googleai-edge-gallery/AndroidLiteRTOpen source