Memory
25 updates on Memory.
Qualcomm CEO says AI companies want 100B-parameter models running on phones by 2028
Cristiano Amon says unnamed AI companies have asked Qualcomm for phones that can run at least a 100-billion-parameter model almost continuously by 2028.
Apple launches M6 Mac mini and M5 Ultra Mac Studio for local AI
Apple’s new Mac Studio scales to 512GB of unified memory, which Apple says lets researchers run large language models entirely on device.
OPPO unveils ColorOS 17 with an on-device linear-attention model and 128K context
OPPO says the on-device model in ColorOS 17 uses linear attention, supports 128K context and needs 48% less memory and 55% less energy than its predecessor.
Edge0 releases MoE expert-offloading framework, demos 35B model on iPhone
Edge0 streams mixture-of-experts weights from storage. Its 35B tier reports 2.9 GiB peak memory on a Mac mini M4 Pro; a launch post shows a 35B model on an iPhone.
FBLayout fine-tunes transformers on phone GPUs 2.2 to 5.7 times faster
A MobiSys 2026 paper fine-tunes seven transformer models on phone GPUs 2.2 to 5.7 times faster than MNN, TFLite and TVM, with 4.2 times fewer cache misses.
Meta AI builds MobileMoE, 5.3B parameters with 0.9B active per token
Meta AI trained three on-device mixture-of-experts models that store 1.3B to 5.3B parameters and run 272M to 922M of them per token.
FlexServe runs on-device LLM inference inside TrustZone for 4.41 percent more latency
Shanghai Jiao Tong University researchers kept model weights and prompts inside ARM TrustZone at 4.41 percent more time to first token.
Confidant fine-tunes Phi2-2.7B across a phone and two laptops in 40.1 hours
Zhejiang University researchers split a language model across three everyday devices and fine-tuned Phi2-2.7B on Alpaca in 40.1 hours without moving raw data.
Benchmark of 68 small language models finds architecture outweighs size on device
A study of 68 models from 100M to 5B puts Phi-3 near 70 percent accuracy and finds first-token time and memory tracking architecture, not parameter count.
Vijay Janapa Reddi argues edge generative AI needs models under 1B parameters
In ACM Queue, Vijay Janapa Reddi argues generative AI reaches edge devices only through small deployable models, with sub-1B ones the viable case.
D2MoE picks a bit-width per token, 1.39 times the throughput at up to 53 percent less memory
A MobiCom 2025 paper routes every token to an expert and a bit-width, reporting up to 1.39 times the throughput of EdgeMoE at up to 53 percent less memory.
ROMA keeps a 4-bit 3B model in on-chip ROM and reports 31,800 tok/s in synthesis
A synthesised 7 nm accelerator holds a 4-bit 3B Llama in read-only memory and the LoRA adapter in SRAM, with no chip and no FPGA prototype built.