Hardware
Pixel 11 series: Tensor G6 adds 50 percent more TPU compute
Google says Tensor G6 with the latest Gemini Nano model processes on-device AI tasks up to 3.5 times faster while using up to 3.5 times less energy.
NPUs lead LLM prefill, CPUs lead decoding on Snapdragon phones
A measurement study of five mobile LLM frameworks finds NPUs excel at prefill, CPUs at decoding, and framework gaps of up to 10x on NPUs.
Qualcomm CEO says agents will become the new app, cites more than 40 device designs
Cristiano Amon tells CNBC that Qualcomm has over 40 designs for AI wearables and that smart glasses shipments could reach hundreds of millions a year.
iPhone 16 Pro settles 41.5 percent below peak over 20 back-to-back prompts
Four authors ran Qwen 2.5 1.5B 20 times in a row on four edge platforms and report the iPhone 16 Pro settling 41.5 percent below its peak throughput.
NanoSD shrinks Stable Diffusion 1.5 for phone NPUs
NanoSD is a family of 130M to 315M-parameter distilled Stable Diffusion models for image restoration on mobile NPUs.
Intelligence per watt puts local model coverage at 88.7% of real queries
Stanford and Together AI measured 20+ local models on 1M real queries and propose accuracy per watt as the metric for local inference.
A19 Pro puts Neural Accelerators in every GPU core
Apple says the iPhone 17 Pro chip pairs Neural Accelerators in each of six GPU cores with a 16-core Neural Engine to run large local language models.
ShadowNPU scores attention on the Snapdragon NPU and reports 4.5 times faster inference
Peking University and BUPT move attention token scoring onto a Snapdragon NPU and report up to 4.5 times faster inference using a single CPU core.
ROMA keeps a 4-bit 3B model in on-chip ROM and reports 31,800 tok/s in synthesis
A synthesised 7 nm accelerator holds a 4-bit 3B Llama in read-only memory and the LoRA adapter in SRAM, with no chip and no FPGA prototype built.
Raspberry Pi 5 beats human reading speed only up to Phi 3.5 mini
Helsinki and EURECOM researchers measured 11 models on a Raspberry Pi 5 and a Jetson Orin Nano, where memory and battery gave out before speed did.
GenAI at the edge survey lists 12 accelerators, 8 of them only simulated
A Johns Hopkins and Duke survey of generative AI on edge devices puts peak accelerator efficiency at 74.34 TOPS/W, with 8 of 12 designs only simulated.
EXO Labs benchmarks put Llama 3.1 8B at 14 tok/s on an iPhone 15 Pro
EXO Labs published automated inference benchmarks from real devices, covering iPhone 15 Pro, Galaxy S24 Ultra, Mac mini M4 Pro clusters and desktop GPUs.