On-device
Quant.npu runs 4-bit LLMs up to 15.1 percent faster on a Qualcomm NPU
Eight authors report a fully static integer quantisation method that runs 4-bit models up to 15.1 percent faster on a Qualcomm SM8650 NPU.
Tencent open-sources a 440 MB offline translation model for phones
Hy-MT1.5-1.8B-1.25bit compresses a 1.8-billion-parameter translation model from 3.3 GB to 440 MB and runs offline on a phone. Weights and an Android demo are public.
Airgap is a React Native kit for support chatbots that answer offline
Xavier Puspus published a React Native kit for support chatbots that answer without a network, running a 2.4 GB Gemma 4 E2B file through llama.rn.
Prism ML releases Ternary Bonsai, 1.58-bit models with an 8B version at 1.75 GB
Ternary Bonsai 8B needs 1.75 GB, scores 75.5 on Prism ML’s benchmark average and runs at 27 tok/s on an iPhone 17 Pro Max.
Gemma 4 lands on the edge with Agent Skills in Google AI Edge Gallery
Google says Gemma 4 E2B runs in under 1.5 GB on some devices and reaches 3,700 prefill tokens per second on a Qualcomm Dragonwing IQ8 NPU.
Prism ML releases 1-bit Bonsai 8B, an 8.2B-parameter model that needs 1.15 GB
1-bit Bonsai 8B stores all weights in one bit, needs 1.15 GB and runs at about 40 tok/s on an iPhone 17 Pro, according to Prism ML.
iPhone 16 Pro settles 41.5 percent below peak over 20 back-to-back prompts
Four authors ran Qwen 2.5 1.5B 20 times in a row on four edge platforms and report the iPhone 16 Pro settling 41.5 percent below its peak throughput.
Meta's MobileLLM-Flash goes shallow and wide for 1.8 times faster prefill
Meta designed 350M, 650M and 1.4B models by measuring latency on a Galaxy S25, reversing the deep-and-thin rule of the first MobileLLM.
FlexServe runs on-device LLM inference inside TrustZone for 4.41 percent more latency
Shanghai Jiao Tong University researchers kept model weights and prompts inside ARM TrustZone at 4.41 percent more time to first token.
Alibaba adds 0.8B and 2B sizes to Qwen3.5, with 262K context and a vision encoder
Alibaba released Qwen3.5-0.8B and Qwen3.5-2B, dense vision-language models with a 262,144-token context, Apache 2.0 weights and 4-bit MNN builds.
ClawMobile tries system commands before screen taps and finishes all six test tasks
An agent runtime that runs on the phone itself routes tasks to ADB commands before UI taps, and the authors report 100 percent completion on six tasks.
Apple ships Python bindings for the on-device Foundation Models framework
The apple-fm-sdk package calls the on-device Apple Intelligence model from Python on macOS 26, for scripting and batch evaluation outside Swift.