Research
73 updates on Research.
Amazon survey puts some small models at 10 to 100 times their parameter count
A survey of about 160 papers finds 1B to 8B models matching much larger ones on single benchmarks, and shows where models below 1B fall apart.
BlueLM-V-3B runs a multimodal model on a Dimensity 9300 NPU in 2.2 GB
CUHK and vivo AI Lab report 24.4 tok/s and 2.2 GB peak memory for a 3B vision-language model on a MediaTek Dimensity 9300.
PhoneLM searches for a fast architecture before training it and hits 58 tok/s
BUPT researchers picked their 0.5B and 1.5B transformer shapes by measuring speed on a Snapdragon 8 Gen 3 first, then pre-training the winner.
Apple team finds H100 last on tokens per dollar for models up to 2B
Seven Apple authors measured tokens per dollar for LLaMA-style models from 100M to 2B and found H100s last at every size, behind cheaper A100s.
CoMiGS splits on-device fine-tuning into shared generalists and private specialists
EPFL researchers split LoRA adapters into shared generalists and device-local specialists, and report 47.19 test perplexity against 54.38 for training alone.
ElastiLM resizes a shared phone LLM per request and switches in 0.31 seconds
Peking University and BUPT built an on-device LLM service that fits model and prompt to each request, switching sub-models in 0.31 seconds.
Ai2 releases OLMoE, 7B parameters with 1B active per token
Ai2's mixture-of-experts model holds 6.9B parameters but runs 1.3B per token, and its iOS app runs it offline on an iPhone 15 Pro or newer.
Spectra ships ternary 3.9B models that match 4-bit quantisation at half the bits
Nolano AI trained 54 models from 99M to 3.9B parameters on the same 300B tokens in ternary, quantised and half-precision form to compare them by bit size.
Alibaba builds Qwen2's 0.5B and 1.5B sizes for phones, earphones and glasses
Alibaba built Qwen2-0.5B and Qwen2-1.5B for smartphones, earphones and smart glasses, with 32K context and Apache 2.0 weights.
llm.npu prefills 1,106 tok/s for Qwen1.5-1.8B on a Snapdragon 8 Gen 3
Peking University and BUPT move prompt processing onto the Hexagon NPU and report up to 43.6x faster prefill and up to 59.5x lower energy use.
Apple's MobileCLIP-S0 encodes an image in 1.5 ms on an iPhone 12 Pro Max
Apple timed its image-text models on an iPhone and released four variants, the weights and the reinforced DataCompDR dataset.
BUPT measures 22 LLMs on four Android phones at about 200 ms per token
A BUPT team ran 22 models from 0.5B to 7B on four Android phones with llama.cpp and found 7B models take about 4 GB and 200 ms per token.