https://llmobile.news/tags/hugging-face/A concise news ticker covering AI on mobile devices, local models, apps and hardware.en-GBhttps://llmobile.news/ticker/smollm3/https://llmobile.news/ticker/smollm3/Tue, 08 Jul 2025 12:30:00 +0200Hugging Face released SmolLM3 on July 8, 2025, a 3B model trained on 11.2 trillion tokens that takes up to 128k tokens of context, covers six languages and answers in either of two modes. It is the successor to SmolLM2, whose largest model was 1.7B with an 8k context window and, in Hugging Face’s own words, primarily understands and generates content in English. The longer context comes from an architecture change plus a separate training stage. Hugging Face removed rotary position embeddings from every fourth layer, a technique it calls NoPE, and replaced multi-head attention with grouped-query attention to shrink the key-value cache during inference. After the main pretraining run it trained the model on another 100 billion tokens in two steps, 4k to 32k and then 32k to 64k, and YaRN, a method for extrapolating past the trained length, carries it to 128k tokens at inference, though the published configuration ships set to 64k and has to be edited to go further. Multilingual text made up 12 percent of the web data at every pretraining stage, and Hugging Face lists English, French, Spanish, German, Italian and Portuguese as supported. Hugging Face’s own base-model figures put SmolLM3 at 76.2 on HellaSwag, 65.6 on ARC and 79.0 on BoolQ, ahead of both Llama 3.2 3B and Qwen2.5 3B and ahead of the larger Qwen3 4B on all three. Qwen3 4B stays in front on maths and code, at 51.2 against 46.1 on MATH and 54.9 against 30.5 on HumanEval+, and on the Ruler 64k long-context test Llama 3.2 3B leads at 72.9 against 67.9 for SmolLM3. On the Flores-200 translation benchmark SmolLM3 scores 56.3 against 55.8 for Qwen3 4B and 54.9 for Qwen2.5 3B, and its multilingual chart shows it ahead of both 3B rivals in all five non-English languages and behind Qwen3 4B in all five. Table: Hugging Face's own base-model figures. A flag in the system prompt, /think or /no_think, decides whether the instruct model reasons before it answers. Hugging Face reports the reasoning mode at 36.7 percent on the AIME 2025 maths competition against 9.3 percent without it, 41.7 against 35.7 on GPQA Diamond and 30.5 against 15.2 on LiveCodeBench v4. Qwen3 4B leads on most of the eight tests in that comparison in both modes, at 58.8 on AIME 2025 and 52.9 on LiveCodeBench, though SmolLM3 without reasoning tops it on the IFEval instruction-following test at 76.7 to 68.9. Table: Hugging Face's own instruct-model figures. Hugging Face published the recipe as well as the weights. Pretraining ran in three stages that moved web data from 85 percent to 75 and then 63 percent while code rose from 12 percent to 24 and maths from 3 percent to 13, each share drawn from named public datasets, and the company released the training configs with the exact data weights along with intermediate checkpoints. Both the base and instruct models are on Hugging Face under the Apache 2.0 license, and the model card names llama.cpp, ONNX, MLX, MLC and ExecuTorch for local inference. The 4-bit GGUF published by ggml-org on release day is 1.92 GB, against 3.27 GB for the 8-bit build for MLX and 6.15 GB for the bfloat16 weights. Diagram: Hugging Face. Source: https://huggingface.co/blog/smollm3 Read the article: https://llmobile.news/ticker/smollm3/Hugging FaceOpen weightsBenchmarksOpen sourcellama.cpphttps://llmobile.news/ticker/googleai-edge-gallery/https://llmobile.news/ticker/googleai-edge-gallery/Sun, 18 May 2025 00:21:00 +0200Google’s AI Edge team has published Google AI Edge Gallery, an app that runs generative models entirely on the device. The home screen offers three tasks, AI Chat for multi-turn conversation, Ask Image for questions about a photo from the camera or the gallery, and Prompt Lab for single-turn work, which covers free-form prompts, rewriting text in a chosen tone, summarising and generating code. Once a model file is on the phone, the app needs no connection and Google states that all processing happens on the device. Screenshot: Google. The third screen shows the per-model settings, including the GPU and CPU toggle. Models come from the LiteRT Community organisation on Hugging Face, which the app’s welcome text links to directly. The first release listed three downloads, Gemma 3 1B IT at 4-bit in 554.7 MB, Hammer 2.1 1.5B at 8-bit and Qwen2.5 1.5B Instruct at 8-bit, the latter two at roughly 1.6 GB each. Gated repositories ask for a Hugging Face access token, which the settings dialog stores and can clear again. Anything outside that list has to arrive as a LiteRT .task bundle picked from device storage, and the app tells the user that no other file type is supported. Each model then carries a settings dialog with an accelerator switch between GPU and CPU, next to max tokens, top-k, top-p and temperature. The shipped list sets a different default per model, GPU first for Gemma 3 1B and CPU only for Qwen2.5 1.5B Instruct. After every reply the app prints four of its own measurements, time to first token in seconds, prefill speed and decode speed in tokens per second, and total latency in seconds. It derives them from the session itself, dividing the prompt’s token count by the time the first token took and counting decoded tokens against the clock that starts with that first token, so the numbers describe whatever hardware the app happens to be running on. Google calls this an “experimental Alpha release” and ships it under the Apache 2.0 licence. There is no store listing at this point, so installing means taking the APK from the project’s releases page on a device running Android 8.0 or newer, with a project wiki covering the corporate-device case. The README lists Android as available now and iOS as coming soon. Underneath, the app builds on LiteRT and Google’s MediaPipe LLM inference API for Android. Source: https://github.com/google-ai-edge/gallery Read the article: https://llmobile.news/ticker/googleai-edge-gallery/GoogleAndroidLiteRTHugging FaceOpen sourcehttps://llmobile.news/ticker/huggingsnap/https://llmobile.news/ticker/huggingsnap/Wed, 19 Mar 2025 16:16:00 +0100Hugging Face published HuggingSnap, an iPhone app that describes what the camera is pointed at, with the model running on the phone rather than in the cloud. The app’s runtime configuration loads SmolVLM2 at 500M parameters in Apple’s MLX format. Hugging Face had trailed the app in its SmolVLM2 announcement on February 20, 2025, writing that the 500M model lets users analyse and understand video content directly on their device with no cloud required. The README describes the app as a way to point the camera and have text translated or summarised, identify plants and animals and learn more about surrounding objects. Inference runs through MLX Swift, Apple’s machine learning framework for Apple silicon, with a modified fork of Apple’s MLX example code supplying the vision-language support. The app pulls the weights from the Hugging Face Hub the first time it runs, then holds Metal memory to 82 percent of what iOS reports as available to the process. Input comes from the live camera, from stills and clips captured in the app, and from photos and videos picked out of the library. A tap on the shutter takes a picture and a press and hold records video, after which the app offers a describe action for images and a summarise action for video, with questions typed or dictated. The app’s own interface states that photos and videos are processed entirely on the device and that no data is sent to the cloud. The README states that installation needs an iPhone running iOS 18, and the Xcode project sets iOS 18.0 as the deployment target and iPhone as the only device family, with no Mac Catalyst or visionOS build. TechCrunch reported the app as a free App Store download on March 19, 2025, quoting the listing as saying it works offline and processes all data on the phone. The SmolVLM2 weights are Apache 2.0, while the app repository itself carries no licence file. Source: https://github.com/huggingface/HuggingSnap Read the article: https://llmobile.news/ticker/huggingsnap/Hugging FaceiOSiPhoneApple SiliconOpen weightshttps://llmobile.news/ticker/smollm/https://llmobile.news/ticker/smollm/Tue, 16 Jul 2024 16:00:00 +0200Hugging Face released SmolLM on July 16, 2024, a family of three base models at 135M, 360M and 1.7B parameters. The team says the models are designed to be small enough to run locally across a range of hardware, from smartphones to laptops, and cites the iPhone 15 with 6 GB of DRAM and the iPhone 15 Pro with 8 GB as reference points. Hugging Face trained the 135M and 360M models on 600B tokens each and the 1.7B model on 1T tokens. The substance is the training data, released as SmolLM-Corpus. Cosmopedia v2 contributes 28B tokens of synthetic textbooks, stories and articles generated by Mixtral-8x7B-Instruct, 39 million documents built from a predefined list of 34,000 topics taken from the BISAC book classification instead of from unsupervised clustering. FineWeb-Edu contributes 220B deduplicated tokens of educational web pages, pulled out of FineWeb by a quality classifier trained on annotations from Llama3-70B-Instruct. Python-Edu applies the same idea to code, keeping 4B tokens of the 40B Python tokens in The Stack that scored 4 or higher on an educational code classifier. On Hugging Face’s own evaluation, run with the same lighteval setup for every model it could download, SmolLM-135M averages 39.50 across MMLU, TriviaQA, ARC, PIQA, HellaSwag, OpenBookQA and Winogrande, against 32.75 for GPT2-137M and 32.49 for Pythia-160M. SmolLM-360M averages 44.44 against 39.60 for Qwen2-500M and 39.32 for Qwen1.5-500M, and SmolLM-1.7B averages 51.97 against 50.36 for Microsoft’s Phi-1.5 and 47.98 for Qwen2-1.5B. Meta’s MobileLLM is not publicly available, so Hugging Face took its per-benchmark numbers from the paper and reports no average for those rows. Table: Hugging Face's own figures, evaluated with the same setup for every model except MobileLLM. Hugging Face also benchmarked the memory footprint of each size. SmolLM-135M takes 269.03 MB in bf16, 162.87 MB at int8 and 109.78 MB at int4. SmolLM-360M takes 723.65 MB, 409.07 MB and 251.79 MB across the same three precisions, and SmolLM-1.7B takes 3422.76 MB, 1812.14 MB and 1006.84 MB. Table: Hugging Face. The weights are on Hugging Face under Apache-2.0, with ONNX checkpoints published beside the transformers ones and a GGUF build for llama.cpp announced as planned. Hugging Face also put up browser demos of the instruction-tuned 135M and 360M models that run client-side through WebGPU, with no server call. The corpus is released separately under ODC-By. Source: https://huggingface.co/blog/smollm Read the article: https://llmobile.news/ticker/smollm/Hugging FaceOpen weightsBenchmarksMemoryWebGPU