All updates

Qualcomm has teased its next-generation Hexagon NPU ahead of Snapdragon Summit 2026 (Sep 22–24), detailing an architecture redesigned around persistent on-device AI agents rather than single-pass inference. The full platform — widely expected to be a Snapdragon 8 Elite Gen 6 — will be formally announced at Summit. Qualcomm has not yet named the chip or provided absolute figures for memory capacity or NPU throughput.

Two architectural changes stand out. The Element Accelerator is a new block inside the Hexagon NPU dedicated to transformer operations, working alongside the existing scalar, vector and matrix units. It targets the compute patterns that modern generative and agentic models use. A 50% larger shared memory subsystem inside the NPU keeps more model state, activations and intermediate tensors on-chip, reducing trips to external LPDDR — a move aimed at the memory wall that has long bottlenecked mobile AI, especially with long context windows.

The NPU also prepares for Mixture-of-Experts (MoE) models. The company cites a 30-billion-parameter MoE that activates only about 3B parameters per token, keeping tens of billions available without the bandwidth cost of dense models. This is paired with flash-to-memory expert management and caching to load the right experts from storage. INT2, INT4, INT8, FP8 and FP16 precision are supported, and the new platform delivers up to 50% higher prefill performance for INT4 models, faster decoding throughput and enhanced speculative decoding.

The design targets always-running AI with long-context reasoning and low-latency action loops, with the CPU handling routing and orchestration of multi-step workloads while Hexagon accelerates the heavy lifting. Adreno handles AI-accelerated graphics rendering and a new Neural Fusion layer extends AI acceleration into gaming. Whether this actually delivers faster, more capable local assistants depends on finished devices — several critical figures (absolute memory size, power consumption, real-world benchmarks) remain undisclosed. The full picture comes at Summit in two weeks.

Apple has introduced the iPhone 18 Pro and iPhone 18 Pro Max with the A20 Pro chip. According to Apple, the chip has a new dual 16-core Neural Engine, 32 cores in total, which the company describes as double the AI processing power of A19 Pro. Apple says the Neural Engine accelerates on-device AI models and computational photography.

The A20 Pro also offers 50 percent more memory bandwidth than A19 Pro, and its 6-core CPU includes integrated Neural Accelerators. Apple positions the chip for “more advanced on-device AI workloads” alongside games.

The phones ship with iOS 27, Apple Intelligence and the new Siri AI. Apple states that Apple Intelligence uses on-device processing together with Private Cloud Compute. Pre-orders start on 12 September, with availability from 18 September; prices start at $1,199 for the iPhone 18 Pro and $1,299 for the iPhone 18 Pro Max, both with 256 GB.

Update, September 20, 2026. The retailer ZEERA reports results from a Max Tech side-by-side comparison of the iPhone 18 Pro Max and iPhone 17 Pro Max. Local LLM generation rose from 33.5 to 55.4 tok/s, a gain of 65 percent, and measured memory bandwidth from 66.5 to 111.8 GB/s, a gain of 68 percent. The comparison also lists 25 percent higher single-core and 29 percent higher multi-core CPU scores in Geekbench 7, and a 48 percent higher graphics score.

Apple’s developer page for Apple Intelligence says the Foundation Models framework gives Swift apps direct access to the same on-device model that powers Apple Intelligence. New multimodal prompts pass images alongside text, and the model can call Vision framework tools such as OCR and barcode readers, all on-device according to Apple. Apps can also work with any model that conforms to the Language Model protocol, including cloud models like Claude and Gemini. Members of the App Store Small Business Program with fewer than 2 million first-time downloads get the next generation of Apple Foundation Models on Private Cloud Compute at no cloud API cost.

The new Siri AI is a three-tier hybrid system, not fully on-device. A new System Orchestrator in iOS 27 routes every request to the right tier:

• Tier 1 — On-Device: Simple tasks (timers, app launches, local context search) run entirely on Apple’s Foundation Models (~3B parameters, 4-bit quantized) on the Neural Engine. Nothing leaves the iPhone.
• Tier 2 — Private Cloud Compute: Moderately complex requests route to Apple’s own servers on Apple Silicon (M2 Ultra, now M5) in hardware-isolated Secure Enclaves. Personal data is stripped and tokenized. Apple retains control; no data retention.
• Tier 3 — Google Gemini: The heaviest reasoning, real-time knowledge, and multimodal tasks are sent through PCC’s privacy proxy to a custom 1.2-trillion-parameter Gemini model on Nvidia Blackwell B200 GPUs in Google Cloud. Before any query reaches Google, the user’s identity and personal data are removed. Google is contractually barred from training on these queries.

Apple reportedly pays Google ~$1 billion per year for this partnership. The System Orchestrator decides the routing threshold in real time based on task complexity and context size. Apple’s public materials only name On-Device and PCC as the two processing locations — Gemini is never mentioned on Apple’s consumer pages.

Foundation Models can now work with any language model, including Apple’s own, Claude, and Gemini, through a single Swift interface. Apple did not publish details of the routing logic or which types of queries are sent off-device.

Update, September 21, 2026. Adrien Grondin, who works on the Locally AI app, says on X that the iPhone 18 Pro runs a 27B model at double the speed of the iPhone 17 Pro. His 11-second video shows the demo, but the post names neither the model nor its quantization and gives no tok/s figures.

Wccftech reports on the same demo, attributing it to Grondin and saying the model is Bonsai 2 27B from Prism ML — a ternary-weight version of Qwen3.8 27B at 1.76 bits per weight, 5.9 GB total. PrismML describes Bonsai 2 27B as released under Apache 2.0, running on Apple devices through MLX and on NVIDIA GPUs through CUDA.

The iPhone 18 Pro is much better than what I expected for on-device AI

It can run a 27B model at double the speed of the 17 Pro

The new A20 Pro chip is a beast pic.twitter.com/gb11TvBfn0

— Adrien Grondin (@adrgrondin) September 19, 2026

Google has released ADK for Kotlin 1.0, the general-availability version of its Agent Development Kit for Kotlin, Java and Android. The kit is built on Kotlin Multiplatform and has full feature parity with ADK 1.0 Core, according to Google. It adds a set of Android extensions for agents that run on the device.

The LiteRT-LM module runs open models such as Gemma on the device with tool calling, on Android and on the JVM, and needs JDK 21 or newer according to the project README. A separate ML Kit module runs the built-in Gemini Nano model through the ML Kit GenAI Prompt API. It is Android-only, ships as a 1.0.0-beta and does not support tool calling yet. The documentation says agents can run without an API key and without a network connection at inference time, and developers can combine on-device and cloud models in one multi-agent system, with cloud Gemini models reached through Firebase AI Logic.

Further Android modules store chat sessions in a local Room database, keep long-term memory in an on-device AppSearch index and save agent files in app-private storage. Skills, which are bundles of domain-specific prompts and tools, can ship inside the APK’s assets folder. Tools are declared with @Tool and @Param annotations, and a Kotlin Symbol Processing step generates their schemas at compile time with no runtime reflection.

ADK for Kotlin is available under the Apache 2.0 license on GitHub, and the documentation covers setup. The repository includes runnable examples, among them a Java interoperability sample.

Desert Ant Labs, a newly launched company, has introduced 18 on-device models as its first release, each built for a single task such as transcription, audio enhancement, personal data masking, language detection and video summarization, according to its launch post. Twelve of the models are stable and six are in beta.

Voz, the transcription model, converts 10 minutes of audio in 2 seconds on an iPhone, according to the company. Desert Ant Labs reports 298x realtime speed on an iPhone 17 Pro and 319x on an M3 Ultra, and says Voz is 4.7x faster than Whisper and returns word-level timestamps. The audio enhancement model Clear is 9 MB and processes a 5-minute recording in 1 second, at 302x realtime on an iPhone 16 Pro and 345x on a MacBook Pro with M5.

Redact masks personal data in 27 languages, takes up 12 MB and detects 88.8% of personal data, the company says. Tongue identifies 84 languages in 2 MB and reaches an accuracy of 0.933 from three words. Clips turns a 10-minute video into clips in 5 seconds with a 284 MB model, which Desert Ant Labs says is 10x faster than Claude Sonnet and uses 470x less energy at the same quality.

All models are free up to 100k monthly active devices, with no tokens and no logins, according to the Desert Ant Labs website. Developers integrate them through SDKs for Swift, Kotlin and JavaScript, which are published on GitHub. The models are also on Hugging Face, where browser demos let users try them, and the company also offers a command-line tool for testing. Documentation is at desertant.com/docs.

OpenBMB has released MiniCPM5-2B, published on Hugging Face and dated 7 September 2026 in the project changelog. The model card describes a dense Transformer “built for on-device, local deployment, and resource-constrained scenarios”, with 2,516,756,480 parameters of which 1,981,982,720 are non-embedding.

The architecture uses 42 layers with grouped-query attention at 16 query heads and 2 key-value heads, and supports a context length of 131,072 tokens. It is the second model in the MiniCPM5 series after MiniCPM5-1B, scaling the same recipe to a larger footprint.

OpenBMB reports an average score of 53.9 across its evaluation dimensions, including 69.1 on LiveCodeBench, 86.5 on AIME 2025 and 68.1 on NoLiMa. The release is licensed under Apache-2.0 and ships in BF16, GGUF, GPTQ 4-bit, MLX for Apple Silicon and LiteRT-LM for mobile, plus a DSpark draft model for speculative decoding.

Modality Dance has released PalmClaw, an open-source framework that manages sessions, memory, skills, tools and the agent loop directly on an Android phone. It exposes device capabilities as structured tools with explicit arguments and defined execution boundaries instead of driving the screen through GUI actions. The paper is listed as an EMNLP 2026 system demonstration.

The language model does not run locally. In the evaluation, a Xiaomi Redmi phone with Android 13 calls DeepSeek-V4-Flash through a remote API. On 70 adapted phone tasks covering calendar, weather, contacts, audio, notes, files, Bluetooth and media, PalmClaw reaches 97.1% task success, an 11.5% relative gain over the strongest baseline. Tasks take 17.7 seconds on average, a 94.9% reduction, and need 2.8 actions each against 103.9 for the GUI-based ApkClaw.

The exposed tools cover device status and permissions, media, Bluetooth, calendar, contacts, workspace files, web access, memory, communication and scheduled automation. On a 19-task AssistantBench subset for web information seeking, PalmClaw scores 36.85%. The authors name a modest task set without third-party GUI workflows, a single model backbone and one run per task as limitations. The code is on GitHub and the paper carries a CC BY-NC-SA 4.0 license.

Google’s ML Kit GenAI documentation now lists the first devices running nano-v4, 9to5Google reports. The list covers the Pixel 11, Pixel 11 Pro, Pixel 11 Pro XL and Pixel 11 Pro Fold, plus Samsung’s Galaxy Z Flip8, Galaxy Z Fold8 and Galaxy Z Fold8 Ultra.

The same documentation sets Nano v3 or greater as the requirement for Gemini Intelligence, Google’s on-device feature set. According to the report, that requirement first appeared in May 2026, was removed, and has now been reinstated. Listed hardware requirements include 12 GB or more of RAM, a qualified flagship system-on-chip, five or more OS upgrades and six years of security support.

Gemini Intelligence features named in the report include Rambler and Proactive Assistance on Pixel 11, and task automation across more than 40 apps on Samsung’s foldables.

Artificial Analysis has published a benchmark of small language models running locally on an iPhone 17 Pro. The study covers 41 quantised builds, 33 of which ran successfully on the device. Models were executed with llama.cpp at 4-bit quantisation or smaller, and had to fit within 8 GB of memory including the KV cache at 8K context. Artificial Analysis states that the inference benchmarking is operated in partnership with Liquid AI.

Nanbeige4.2-3B and LFM2.5-2.6B tied for the highest intelligence score at 63. On speed and memory the two differ sharply: LFM2.5-2.6B completed the test prompt in 8.0 seconds using 2.3 GB, against 21.4 seconds and 4.0 GB for Nanbeige4.2-3B. Ornith-1.0-9B scored 62 and Qwen3.5 9B (Reasoning) 61.

End-to-end generation time for a 1,024-token prompt plus 256 output tokens ranged from 0.9 seconds for LFM2.5-230M to 26.7 seconds for Falcon-H1R-7B. Peak in-test memory at 4K context ran from 0.4 GB to 6.9 GB. The intelligence index combines tool calling, instruction following, knowledge, scientific reasoning and mathematics. Artificial Analysis notes that context limits, not parameter count alone, held back several 9B models, which frequently hit a 16K ceiling.

Graphcore Research has fitted the 11-billion-parameter vision-language model Llama-3.2-11B-Vision-Instruct into 3.6 GB, down from 21 GB, using a new 2.7-bit weight format called S3D8. In the team’s blog post, the model runs on a Pixel 8a with a custom C++ inference implementation and generates 3.8 tok/s.

S3D8 stores three weights in 8 bits, which comes to 2.68 bits per parameter. Each group of three weights holds a 5-bit index into a shared table of centroids plus 3 bits of sign information, and the values are decoded to channel-scaled INT8 for compute. On the Pixel 8a using 5 cores, the fused dequantize-and-multiply kernel reaches 33.8 GMAC/s for single-token generation (GMAC/s means giga multiply-accumulate operations per second, a standard measure of AI compute throughput on devices), compared with 26.5 GMAC/s for INT8 and 13.6 GMAC/s for bfloat16, according to Graphcore. Graphcore measured the 3.8 tok/s while generating 100 tokens from a single image tile.

Graphcore measured accuracy on 1,024-example subsets of VQAv2, ChartQA, DocVQA and AI2D. S3D8 averages 0.661 against 0.744 for the bfloat16 original, while a plain integer format of similar size averages 0.347 and a student-t format 0.565. The team recovered quality with quantization-aware training by distillation, where a frozen bfloat16 teacher guides the quantized student over 2,048 steps. Training data consists of ImageNet images paired with text the teacher generated from a pool of 495 prompt seeds, and the team says the wrong data lowers the training loss while the model gets worse on downstream tasks.

The demo needs a Pixel 8a or newer Android phone with 8 GB of RAM and 4 to 5.5 GB of free storage, and a CPU that supports the Arm i8mm and bf16 extensions. Collaborators from Arm contributed to the work. The demo APK can be downloaded from Graphcore, the Android code is on GitHub and the method is described in a paper on arXiv. Graphcore says the model is too large and compute-heavy for the phone, so although it runs, it is not practical.

Update, September 19, 2026. The paper puts the compressed model at 3.73 GB including vocabulary and metadata, of which 3,569 MB is weight storage at 2.68 bits per parameter. The authors also ran S3D8 on a Graviton4 server chip with 96 Neoverse V2 cores, where end-to-end decoding reaches 36.8 tok/s against 26.4 tok/s for INT8. On the Pixel 8a, decoding reads weights at 12.5 GB/s.

Per task, S3D8 scores 0.702 on VQAv2 (bfloat16 0.754), 0.648 on ChartQA (0.747), 0.740 on DocVQA (0.844) and 0.554 on AI2D (0.631). The paper lists its limits as a single evaluated model, CPU-only execution and one fixed format for all weights, since mixed-precision experiments did not work. Training used 2,048 steps at batch size 128 on 1.28 million ImageNet images with teacher-generated responses.

Ornith has released Ornith-1.5, a model family with a 9B dense model, a 35B mixture-of-experts model that activates about 3B parameters per token and a 397B mixture-of-experts model. The 9B model also comes as Ornith-1.5-9B-Mobile, a quantized variant (about 1,5 GB on disk) designed for iPhone and Android.

The 9B model scores 47.0 on Terminal-Bench 2.1 with the Claude Code harness and 70.6 on SWE-bench Verified in Ornith’s tests. The company says the model matches or exceeds much larger models such as Gemma 4-31B and Qwen 3.6-35B. In the company’s chart, Qwen3.6-35B-A3B leads on SWE-bench Verified with 73.4 and on Terminal-Bench 2.1 with 52.5, while the 9B model scores 86.4 on GPQA Diamond and 54.2 on MCP-Atlas. The previous Ornith-1.0-9B reaches 43.1 on Terminal-Bench 2.1 in the same chart.

According to Ornith, its training loop lets the model propose new tasks, generate task-specific scaffolds and produce solution rollouts, with the reward from the rollouts propagated across all three stages. The company reports that the 35B model scores 67.8 on Terminal-Bench 2.1 with the Terminus-2 harness, against 52.5 for Qwen 3.6-35B. The models are on Hugging Face, with GGUF builds of all three sizes and MLX builds of the 9B and 35B models.

Researchers at Peking University have published S2-MoE, a self-speculative decoding framework for mixture-of-experts (MoE) models on edge devices. MoE models activate only a few experts per token, which makes verifying several drafted tokens at once expensive. The method reports a speedup of up to 5.3x, about 2.0x on average, over standard autoregressive decoding and is implemented in llama.cpp.

The paper tests DeepSeek-V2-Lite-Chat (15.3B parameters, 1.35B active), OLMoE-1B-7B (6.7B, 1.1B active), Qwen3-30B-A3B (30.5B, 3.3B active) and GPT-OSS-120B (116.8B, 5.7B active). Hardware is a Jetson Orin NX 16GB, Jetson AGX Orin with 32 GB or 64 GB, and an RTX 4090 with constrained GPU memory. On the Orin NX, OLMoE rises from 2.30 to 12.08 tok/s and DeepSeek from 0.55 to 2.07 tok/s. Speedups on that board range from 2.10x to 3.17x for Qwen3 and from 1.57x to 2.53x for GPT-OSS.

The draft pass uses fewer experts per token, and the draft and target share one KV cache. Routing-aware expansion uses the draft’s expert routing to estimate verification cost and adds candidates only where the expected gain is higher. Reuse-aware gating nudges the router toward frequently activated experts, and expert-level offloading handles models that do not fit in memory.

The authors measure a perplexity ratio of 1.012 to 1.013 and top-1 agreement above 89%, with comparable results on LongBench. They compare against standard autoregressive decoding, layer-sparsity and expert-sparsity self-speculation, EAGLE-3 and Cascade, and say other MoE offloading work lacks public llama.cpp implementations. The code is on GitHub.

I-Ju Lin and Zhang-Wei Hong published Online-SDFT, a method that keeps fine-tuning a small language model on a phone from the user’s own stream of decisions. On a synthetic notification-routing benchmark with Liquid AI’s LFM2.5-230M, the authors report 70.28% of decisions matching a hidden user preference against 52.78% for the strongest baseline they tested, and cumulative regret of 44.76 against 105.26, over 720 decisions in three paired streams. They state that this comparison ran on Apple MPS rather than on a phone, and that the confidence intervals are nominal across only three streams.

Self-distillation here means the model teaches itself. The model picks an action first, the phone later observes what happened, and the same model is then run again with that outcome added to its input. Its better-informed answer becomes the training target for the version that saw only the original context, so no second teacher model, no human label and no reward score is needed. The authors keep the 230M base frozen and train a LoRA adapter of 172,032 parameters across 48 tensors in six attention layers, with the teacher pass running with the adapter switched off so student updates cannot move the teacher.

Feeding one interaction at a time into so small an adapter makes it noisy and prone to forgetting, so the authors keep a 64-row buffer of recent lessons and sample each batch from it. Their ablation puts that buffer ahead of the exploration schedule as the load-bearing part, with accuracy dropping from 70.28% to 39.58% and regret climbing from 44.76 to 134.90 once replay is removed, while dropping exploration instead leaves accuracy at 69.44% and regret at 62.00.

The repository carries an Android proof of concept in which LFM2.5-230M decides and ONNX Runtime Training performs the adapter update on the device, with replay and checkpoints in app-private storage that survive a restart. Exporting the model still happens on a Linux host, and the deployment guide asks for Android 12 or newer on ARM64 with 12 GB of RAM at minimum, 16 GB preferred, and 8 GB of free storage, because the training graph is FP32.

The authors call that graph a correctness implementation rather than a claim of practical background training on low-memory phones, and state that they did not measure Android latency, peak memory, battery use or thermal behaviour, and that the benchmark uses synthetic streams rather than real users. The code, the Android app and a Colab notebook are on GitHub under the MIT license. No paper or preprint accompanies the release.