Memory
22 updates on Memory.
Edge0 releases MoE expert-offloading framework, demos 35B model on iPhone
Edge0 streams mixture-of-experts weights from storage. Its 35B tier reports 2.9 GiB peak memory on a Mac mini M4 Pro; a launch post shows a 35B model on an iPhone.
FBLayout fine-tunes transformers on phone GPUs 2.2 to 5.7 times faster
A MobiSys 2026 paper fine-tunes seven transformer models on phone GPUs 2.2 to 5.7 times faster than MNN, TFLite and TVM, with 4.2 times fewer cache misses.
Meta AI builds MobileMoE, 5.3B parameters with 0.9B active per token
Meta AI trained three on-device mixture-of-experts models that store 1.3B to 5.3B parameters and run 272M to 922M of them per token.
FlexServe runs on-device LLM inference inside TrustZone for 4.41 percent more latency
Shanghai Jiao Tong University researchers kept model weights and prompts inside ARM TrustZone at 4.41 percent more time to first token.
Confidant fine-tunes Phi2-2.7B across a phone and two laptops in 40.1 hours
Zhejiang University researchers split a language model across three everyday devices and fine-tuned Phi2-2.7B on Alpaca in 40.1 hours without moving raw data.
Benchmark of 68 small language models finds architecture outweighs size on device
A study of 68 models from 100M to 5B puts Phi-3 near 70 percent accuracy and finds first-token time and memory tracking architecture, not parameter count.
Vijay Janapa Reddi argues edge generative AI needs models under 1B parameters
In ACM Queue, Vijay Janapa Reddi argues generative AI reaches edge devices only through small deployable models, with sub-1B ones the viable case.
D2MoE picks a bit-width per token, 1.39 times the throughput at up to 53 percent less memory
A MobiCom 2025 paper routes every token to an expert and a bit-width, reporting up to 1.39 times the throughput of EdgeMoE at up to 53 percent less memory.
ROMA keeps a 4-bit 3B model in on-chip ROM and reports 31,800 tok/s in synthesis
A synthesised 7 nm accelerator holds a 4-bit 3B Llama in read-only memory and the LoRA adapter in SRAM, with no chip and no FPGA prototype built.
MobiLLM fine-tunes OPT-1.3B in 4.5 GB on device by moving backpropagation to a server
Researchers fine-tuned OPT-1.3B on a Jetson Xavier NX in 4.5 GB by keeping the frozen model on the device and the trainable side network on a server.
Meta's ParetoQ compares five bit widths and puts the accuracy cliff at 1 bit
Meta trained the same models at 1, 1.58, 2, 3 and 4 bits under one recipe and reports 1.58-bit to 3-bit beating 4-bit at matched memory.
Spectra ships ternary 3.9B models that match 4-bit quantisation at half the bits
Nolano AI trained 54 models from 99M to 3.9B parameters on the same 300B tokens in ternary, quantised and half-precision form to compare them by bit size.