https://llmobile.news/companies/graphcore/A concise news ticker covering AI on mobile devices, local models, apps and hardware.en-GBhttps://llmobile.news/ticker/graphcore-llama-mobile/https://llmobile.news/ticker/graphcore-llama-mobile/Mon, 24 Aug 2026 00:00:00 +0200Graphcore Research has fitted the 11-billion-parameter vision-language model Llama-3.2-11B-Vision-Instruct into 3.6 GB, down from 21 GB, using a new 2.7-bit weight format called S3D8. In the team’s blog post, the model runs on a Pixel 8a with a custom C++ inference implementation and generates 3.8 tok/s. Your browser does not support this video. This video could not be loaded. Use the link below to open it directly. Open video S3D8 stores three weights in 8 bits, which comes to 2.68 bits per parameter. Each group of three weights holds a 5-bit index into a shared table of centroids plus 3 bits of sign information, and the values are decoded to channel-scaled INT8 for compute. On the Pixel 8a using 5 cores, the fused dequantize-and-multiply kernel reaches 33.8 GMAC/s for single-token generation (GMAC/s means giga multiply-accumulate operations per second, a standard measure of AI compute throughput on devices), compared with 26.5 GMAC/s for INT8 and 13.6 GMAC/s for bfloat16, according to Graphcore. Graphcore measured the 3.8 tok/s while generating 100 tokens from a single image tile. Graphcore measured accuracy on 1,024-example subsets of VQAv2, ChartQA, DocVQA and AI2D. S3D8 averages 0.661 against 0.744 for the bfloat16 original, while a plain integer format of similar size averages 0.347 and a student-t format 0.565. The team recovered quality with quantization-aware training by distillation, where a frozen bfloat16 teacher guides the quantized student over 2,048 steps. Training data consists of ImageNet images paired with text the teacher generated from a pool of 495 prompt seeds, and the team says the wrong data lowers the training loss while the model gets worse on downstream tasks. The demo needs a Pixel 8a or newer Android phone with 8 GB of RAM and 4 to 5.5 GB of free storage, and a CPU that supports the Arm i8mm and bf16 extensions. Collaborators from Arm contributed to the work. The demo APK can be downloaded from Graphcore, the Android code is on GitHub and the method is described in a paper on arXiv. Graphcore says the model is too large and compute-heavy for the phone, so although it runs, it is not practical. Update, September 19, 2026. The paper puts the compressed model at 3.73 GB including vocabulary and metadata, of which 3,569 MB is weight storage at 2.68 bits per parameter. The authors also ran S3D8 on a Graviton4 server chip with 96 Neoverse V2 cores, where end-to-end decoding reaches 36.8 tok/s against 26.4 tok/s for INT8. On the Pixel 8a, decoding reads weights at 12.5 GB/s. Per task, S3D8 scores 0.702 on VQAv2 (bfloat16 0.754), 0.648 on ChartQA (0.747), 0.740 on DocVQA (0.844) and 0.554 on AI2D (0.631). The paper lists its limits as a single evaluated model, CPU-only execution and one fixed format for all weights, since mixed-precision experiments did not work. Training used 2,048 steps at batch size 128 on 1.28 million ImageNet images with teacher-generated responses. Source: https://graphcore-research.github.io/2026-08-24-llama-mobile/ Read the article: https://llmobile.news/ticker/graphcore-llama-mobile/QuantisationResearchAndroidOpen source