Apple
21 updates on Apple.
Apple opens its on-device model to all apps with the Foundation Models framework
Any app can call the roughly 3-billion-parameter on-device model from Swift, offline and free of charge, with guided generation and tool calling.
Locally AI runs Llama, Gemma and Qwen offline on iPhone and iPad through MLX
Adrien Grondin shipped a free iPhone and iPad app that downloads open-weight models and runs them on device, built on Apple silicon through MLX.
Apple team finds H100 last on tokens per dollar for models up to 2B
Seven Apple authors measured tokens per dollar for LLaMA-style models from 100M to 2B and found H100s last at every size, behind cheaper A100s.
Apple's MobileCLIP-S0 encodes an image in 1.5 ms on an iPhone 12 Pro Max
Apple timed its image-text models on an iPhone and released four variants, the weights and the reinforced DataCompDR dataset.
Apple Intelligence pairs a 3-billion-parameter on-device model with a server model
Apple reports 0.6 ms per prompt token and 30 tokens per second on an iPhone 15 Pro for a model compressed to an average of 3.7 bits per weight.
ExecuTorch alpha runs Llama 2 7B on iPhone 15 Pro and Galaxy phones
PyTorch's edge runtime brought 4-bit Llama 2 7B to iPhone and Galaxy handsets, added early Llama 3 8B support and leaned on Apple, Arm and Qualcomm.
Apple releases OpenELM at 270M to 3B, with parameters spread unevenly across layers
Apple published four models from 270M to 3B parameters with the full training framework and code to run them through MLX on Apple silicon.
Apple researchers run models twice the size of available DRAM from flash
The LLM in a flash paper loads parameters from flash on demand and reports 4 to 5 times faster CPU and 20 to 25 times faster GPU inference.
Apple publishes MLX, where CPU and GPU share arrays without copies
Apple machine learning research released an array framework for Apple silicon with a unified memory model, lazy evaluation and Swift bindings for iOS.