Article complete

New Microsoft AI Model Brings 10x Speed to Reasoning on Edge Devices, Apps

Microsoft is enhancing the AI models capable of running on a single GPU to include reasoning capabilities.

Two robotic hands hold a Microsoft logo against a blue background, symbolizing advancements in AI technology.

Key takeaways

  • Launch Phi-4-mini-flash-reasoning offers 10x speed improvement for reasoning on edge devices.
  • Utilizes a new SambaY architecture for enhanced long-context performance and reduced decoding complexity.
  • Maintains 64k token context length while outperforming larger models on specific reasoning tasks.
  • Supports low-latency environments, ideal for real-time applications like tutoring and adaptive learning.
  • Aligns with Microsoft's commitment to responsible AI through integrated safety mechanisms.

Takeaways drafted with AI from the article text.

The 3.8 billion parameter open model maintains support for a 64k token context length and is fine-tuned on high-quality synthetic data for structured, math-focused reasoning tasks. Unlike earlier Phi models, Phi-4-mini-flash-reasoning introduces a new “decoder-hybrid-decoder” architecture called SambaY, combining state-space models (Mamba), sliding window attention, and a novel Gated Memory Unit (GMU) to reduce decoding complexity and boost long-context performance.

According to Microsoft, this setup allows the model to maintain linear prefill computation time while interleaving lightweight GMUs with expensive attention layers. The result is significantly improved inference efficiency, making it viable for use on a single GPU or in latency-sensitive deployments, such as real-time tutoring tools and adaptive learning apps. 

Benchmarks shared by Microsoft indicate that Phi-4-mini-flash-reasoning outperforms models twice its size on tasks such as AIME24/25 and Math500, while maintaining faster response times on the vLLM inference framework.

The release aligns with Microsoft’s broader push for responsible AI, with safety mechanisms including supervised fine-tuning (SFT), direct preference optimisation (DPO), and reinforcement learning from human feedback (RLHF). The company notes that all Phi models follow its core principles of transparency, privacy, and inclusiveness.

The model is already available through Azure AI Foundry, Hugging Face, and the NVIDIA API Catalogue. For more technical details, one can explore the research paper and the Phi Cookbook for developers.

Recently, Hugging Face also introduced SmolLM3, a 3B-parameter open language model featuring long-context reasoning (up to 128k tokens), multilingual support (six languages), and dual inference modes. Trained on 11.2 trillion tokens, it outperforms peers like Llama-3.2-3B and Qwen2.5-3B, while competing with larger 4B models such as Gemma3 and Qwen3. 

With advancements in small language models and companies adding reasoning capabilities, it should translate to better on-device AI performance compared to that of an AI model with more parameters.

Published

Spotted an error? Tell our editors. Read our editorial standards and corrections.

More in this story

Microsoft · 5 stories
  1. Meta Launches $1,500 VR Headset and Other Highlights From Meta Connect 2022
  2. Key Takeaways from Today’s Microsoft Future Ready Leadership Summit in Mumbai
  3. Microsoft Rolls Out New M365 Features including AI-enabled Copilot Actions
  4. You are here: New Microsoft AI Model Brings 10x Speed to Reasoning on Edge Devices, Apps
  5. Microsoft Unveils Maia 300 With 40% More Tokens Per Megawatt
About the author

Editor

I am a tech aficionado and a computer science graduate with a keen interest in AI, Coding, Open Source, Global SaaS, and Cloud. Have a tip? Reach out to

All stories
Belamy · Daily

The day’s AI news, before your first meeting.