Apps
torchchat runs Llama 3 8B on a Galaxy S23 and iPhone at more than 8 tok/s
PyTorch's demonstration codebase exports Llama models to ExecuTorch and ships iOS and Android sample apps for running them on a phone.
ExecuTorch alpha runs Llama 2 7B on iPhone 15 Pro and Galaxy phones
PyTorch's edge runtime brought 4-bit Llama 2 7B to iPhone and Galaxy handsets, added early Llama 3 8B support and leaned on Apple, Arm and Qualcomm.
Google ships an experimental MediaPipe LLM Inference API for web, Android and iOS
The experimental API runs Gemma 2B, Phi 2, Falcon 1B and Stable LM 3B fully on device, with int8 and int4 weights.
LLMFarm runs llama.cpp models offline on iOS and macOS
The iOS and macOS app loads a quantised model file off the device and chats with it offline, with 3B the practical size on an iPhone.
MLC LLM brings local language models to iPhone, browsers and consumer GPUs
A compiler stack built on Apache TVM deploys chat models natively to iOS, browsers and consumer GPUs, with an iPhone build handed out through TestFlight.
Sherpa runs LLaMA on an Android phone through a Flutter chat app
The Flutter app loads a ggml model file off the phone and chats with it offline, and development stopped three weeks after it started.