On-device
Meta's MobileLLM-Pro runs a 128k context from a 590 MB 4-bit build
Meta Reality Labs released a 1.08B on-device model with a 128k context window, measured at 33.6 tok/s decode on a Galaxy S25 CPU.
Confidant fine-tunes Phi2-2.7B across a phone and two laptops in 40.1 hours
Zhejiang University researchers split a language model across three everyday devices and fine-tuned Phi2-2.7B on Alpaca in 40.1 hours without moving raw data.
ExecuTorch 1.0 reaches general availability for on-device PyTorch models
The PyTorch edge runtime promotes Core ML, Qualcomm Hexagon, Arm Ethos-U, Vulkan and XNNPACK backends to production status.
lm-Meter times on-device inference and finds prefill, not decode, is the bottleneck
A profiler that runs inside the phone measures each inference phase and reports prefill slowing 158 times as models grow, against 10 times for decode.
Meta trains 140M to 950M reasoning models on 4.2T tokens
MobileLLM-R1 spans 140M to 950M parameters, trained on 4.2T tokens, and Meta scores the 950M model at 74.0 on MATH500.
A19 Pro puts Neural Accelerators in every GPU core
Apple says the iPhone 17 Pro chip pairs Neural Accelerators in each of six GPU cores with a 16-core Neural Engine to run large local language models.
ShadowNPU scores attention on the Snapdragon NPU and reports 4.5 times faster inference
Peking University and BUPT move attention token scoring onto a Snapdragon NPU and report up to 4.5 times faster inference using a single CPU core.
P/D-Device prefills in the cloud and decodes on the phone, cutting TTFT 60%
Huawei researchers split prefill and decoding between cloud and phone, reporting time to first token down at least 60% and cloud throughput up to 15x.
Benchmark of 68 small language models finds architecture outweighs size on device
A study of 68 models from 100M to 5B puts Phi-3 near 70 percent accuracy and finds first-token time and memory tracking architecture, not parameter count.
Apple puts the cost of 2-bit compression at 3.4 MMLU points
Apple measures its on-device model at 67.8 MMLU in 16 bits and 64.4 after compression to 2 bits per weight, and details the distillation pipeline behind it.
Liquid AI releases LFM2, three CPU-first models from 350M to 1.2B
Liquid AI released open-weight models of 350M, 700M and 1.2B parameters and reports 2x faster decode and prefill than Qwen3 on CPU.
Hugging Face's SmolLM3 is a 3B model with 128k context and two reasoning modes
Hugging Face released a 3B model with a 128k context window, six languages and a switchable reasoning mode, along with the full training recipe.