Models
S2-MoE speeds up MoE decoding by up to 5.3x in llama.cpp
S2-MoE is a self-speculative decoding method for mixture-of-experts models that reaches up to 5.3x speedup and about 2.0x on average.
Liquid AI releases LFM2.5-2.6B for on-device agents
Liquid AI reports 30 tokens per second on a phone and under 2.5 GB of memory for its 2.6-billion-parameter model, released with open weights.
Prism ML releases Bonsai 27B, a 3.9 GB 1-bit model that fits an iPhone 17 Pro
Prism ML’s 1-bit Bonsai 27B compresses Qwen3.6 27B to 3.9 GB and runs on iPhone, iPad, Mac and NVIDIA GPUs under Apache 2.0.
Apertus Mini distils an open-data 8B into 0.5B, 1.5B and 4B models
Swiss AI distilled its fully open Apertus 8B into 0.5B, 1.5B and 4B models on 1.7T tokens, with 3-bit to 6-bit MLX builds for Apple devices.
Meta AI builds MobileMoE, 5.3B parameters with 0.9B active per token
Meta AI trained three on-device mixture-of-experts models that store 1.3B to 5.3B parameters and run 272M to 922M of them per token.
Prism ML releases Bonsai Image 4B, a 1-bit image model that runs on iPhone
Bonsai Image 4B generates a 512x512 image in 9.4 seconds on an iPhone 17 Pro Max, and its 1-bit diffusion transformer needs 0.93 GB.
LiteRT-LM reports 52 decode tokens per second for Gemma 4 E2B on an Android GPU
Google publishes prefill and decode figures for its on-device runtime, adds Swift and JavaScript APIs, and reports a 2.2x speedup from multi-token prediction.
Tencent open-sources a 440 MB offline translation model for phones
Hy-MT1.5-1.8B-1.25bit compresses a 1.8-billion-parameter translation model from 3.3 GB to 440 MB and runs offline on a phone. Weights and an Android demo are public.
Prism ML releases Ternary Bonsai, 1.58-bit models with an 8B version at 1.75 GB
Ternary Bonsai 8B needs 1.75 GB, scores 75.5 on Prism ML’s benchmark average and runs at 27 tok/s on an iPhone 17 Pro Max.
Tempo compresses hour-long video with a 2B vision model so a 4B LLM can answer
Meta AI and KAUST report 52.7 on LVBench for a 6B system in which a 2B vision-language model squeezes long video down to about 3 tokens per frame.
Gemma 4 lands on the edge with Agent Skills in Google AI Edge Gallery
Google says Gemma 4 E2B runs in under 1.5 GB on some devices and reaches 3,700 prefill tokens per second on a Qualcomm Dragonwing IQ8 NPU.
Prism ML releases 1-bit Bonsai 8B, an 8.2B-parameter model that needs 1.15 GB
1-bit Bonsai 8B stores all weights in one bit, needs 1.15 GB and runs at about 40 tok/s on an iPhone 17 Pro, according to Prism ML.