On-device
Apple opens its on-device model to all apps with the Foundation Models framework
Any app can call the roughly 3-billion-parameter on-device model from Swift, offline and free of charge, with guided generation and tool calling.
Vijay Janapa Reddi argues edge generative AI needs models under 1B parameters
In ACM Queue, Vijay Janapa Reddi argues generative AI reaches edge devices only through small deployable models, with sub-1B ones the viable case.
Gemma 3n runs 5B and 8B models in 2 GB and 3 GB of memory
Google previewed a mobile-first model whose Per-Layer Embeddings cut RAM use, and said the same architecture powers the next Gemini Nano.
Google AI Edge Gallery runs Gemma 3 1B and Qwen2.5 offline on Android
An experimental Google app downloads LiteRT models from Hugging Face, runs chat, image questions and prompt tests offline, and prints decode speed per reply.
Alibaba gives Qwen3's 0.6B and 1.7B models a reasoning switch
Qwen3-0.6B and Qwen3-1.7B carry the family's switch between a reasoning mode and a fast mode, with 32K context and Apache 2.0 weights.
Google's ML Drift runs Llama 3.1 8B on a phone GPU at 12.7 tokens per second
The paper measures 37.1 decode tokens per second for Gemma2 2B and 12.7 for Llama 3.1 8B on the Adreno 750 GPU of a Samsung S24.
D2MoE picks a bit-width per token, 1.39 times the throughput at up to 53 percent less memory
A MobiCom 2025 paper routes every token to an expert and a bit-width, reporting up to 1.39 times the throughput of EdgeMoE at up to 53 percent less memory.
Locally AI runs Llama, Gemma and Qwen offline on iPhone and iPad through MLX
Adrien Grondin shipped a free iPhone and iPad app that downloads open-weight models and runs them on device, built on Apple silicon through MLX.
HuggingSnap describes what the iPhone camera sees with a 500M model on the phone
Hugging Face released an iPhone app that runs SmolVLM2 at 500M parameters through MLX, describing camera scenes, photos and video with no cloud call.
ROMA keeps a 4-bit 3B model in on-chip ROM and reports 31,800 tok/s in synthesis
A synthesised 7 nm accelerator holds a 4-bit 3B Llama in read-only memory and the LoRA adapter in SRAM, with no chip and no FPGA prototype built.
Gemma 3 adds a 1B size that fits in 0.5 GB as an int4 checkpoint
Google released Gemma 3 at 1B, 4B, 12B and 27B with quantisation-aware int4 checkpoints of 0.5 GB and 2.6 GB for the two smallest sizes.
Flower Intelligence runs models on device, with remote handoff off by default
Flower Labs released a preview library that runs Llama 3.2 and SmolLM2 locally via WebLLM or MLX Swift, and calls its remote service only if an app enables it.