New Microsoft AI Model Brings 10x Speed to Reasoning on Edge Devices, Apps

Microsoft has released Phi-4-mini-flash-reasoning, a compact AI model engineered for fast, on-device logical reasoning. This new addition to the Phi family is designed for low-latency environments, such as mobile apps and edge deployments, offering performance improvements of up to 10 times throughput and two to three times lower latency compared to its predecessor.

The 3.8 billion parameter open model maintains support for a 64k token context length and is fine-tuned on high-quality synthetic data for structured, math-focused reasoning tasks. Unlike earlier Phi models, Phi-4-mini-flash-reasoning introduces a new “decoder-hybrid-decoder” architecture called SambaY, combining state-space models (Mamba), sliding window attention, and a novel Gated Memory Unit (GMU) to reduce decoding complexity and boost long-context performance.

According to Microsoft, this setup allows the model to maintain linear prefill computation time while interleaving lightweight GMUs with expensive attention layers. The result is significantly improved inference efficiency, making it viable for use on a single GPU or in latency-sensitive deployments, such as real-time tutoring tools and adaptive learning apps. 

Benchmarks shared by Microsoft indicate that Phi-4-mini-flash-reasoning outperforms models twice its size on tasks such as AIME24/25 and Math500, while maintaining faster response times on the vLLM inference framework.

The release aligns with Microsoft’s broader push for responsible AI, with safety mechanisms including supervised fine-tuning (SFT), direct preference optimisation (DPO), and reinforcement learning from human feedback (RLHF). The company notes that all Phi models follow its core principles of transparency, privacy, and inclusiveness.

The model is already available through Azure AI Foundry, Hugging Face, and the NVIDIA API Catalogue. For more technical details, one can explore the research paper and the Phi Cookbook for developers.

Recently, Hugging Face also introduced SmolLM3, a 3B-parameter open language model featuring long-context reasoning (up to 128k tokens), multilingual support (six languages), and dual inference modes. Trained on 11.2 trillion tokens, it outperforms peers like Llama-3.2-3B and Qwen2.5-3B, while competing with larger 4B models such as Gemma3 and Qwen3. 

With advancements in small language models and companies adding reasoning capabilities, it should translate to better on-device AI performance compared to that of an AI model with more parameters.