Alibaba gives Qwen3's 0.6B and 1.7B models a reasoning switch

Alibaba’s Qwen Team published the Qwen3 Technical Report on May 14, 2025, covering six dense models from 0.6B to 32B parameters and two mixture-of-experts models, where only part of the model is active per token. The two smallest dense sizes, Qwen3-0.6B and Qwen3-1.7B, carry the same mode switch as the rest of the family, so one set of weights either works through a problem step by step before answering or replies straight away. Alibaba reports Qwen3-1.7B scoring 93.4 on MATH-500, a 500-problem maths set, with reasoning on, against 73.0 with it off.

Earlier Qwen releases needed two separate models for that, a chat model and a dedicated reasoning model such as QwQ-32B. In Qwen3 the user picks the behaviour with a flag in the prompt or the system message, /think or /no_think, and the report states that reasoning is the default and that across a multi-turn conversation the model follows the last flag it saw. The same section describes a reasoning budget, where the model’s working is cut off once it passes a token limit the user set and it answers from what it has so far, which the authors say they never trained for and which emerged from merging the two modes.

The benchmark tables give both small sizes in both modes. For Qwen3-0.6B, Alibaba reports 55.6 on MMLU-Redux, a corrected version of the MMLU knowledge test, with reasoning on against 44.6 with it off, and 77.6 against 55.2 on MATH-500. For Qwen3-1.7B the same pair of tables gives 73.9 and 64.4 on MMLU-Redux, 48.3 against 13.4 on the AIME 2024 competition maths set, and 33.2 against 11.6 on LiveCodeBench v5. Against the reasoning baselines in the report, Qwen3-0.6B is ahead of DeepSeek-R1-Distill-Qwen-1.5B on MMLU-Redux, 55.6 to 45.4, and behind it on MATH-500, 77.6 to 83.9.

Alibaba built the small sizes by copying behaviour from the large ones rather than by running the full reinforcement learning pipeline on each. The report describes two stages of distillation from Qwen3-32B and Qwen3-235B-A22B, first on stored teacher outputs generated in both modes, then by matching the small model’s output probabilities against the teacher’s while the small model generates, and it credits the first stage with teaching the smaller models to switch modes at all. Pre-training ran on about 36 trillion tokens covering 119 languages and dialects, up from 29 in Qwen2.5.

Context length is where the two small sizes sit apart from the family. Table 1 of the report puts Qwen3-0.6B and Qwen3-1.7B at 32K tokens against 128K for Qwen3-8B and every larger model, and both tie the input embedding to the output layer so one weight matrix does both jobs. All eight models are published under the Apache 2.0 license on Hugging Face, ModelScope and Kaggle. The Qwen3 blog post of April 29, 2025 names Ollama, LM Studio, MLX, llama.cpp and KTransformers as the tools it recommends for running them locally.