Over the last two years, there has been plenty of talk on how Apple’s AI efforts in the consumer space have not been up to the mark, if not entirely failed. We’re all awaiting the evolution of Apple Intelligence now that the company has signed a deal with Google to use its Gemini AI models.
But beyond the hype, there’s a lesser-known story of Apple’s efforts in AI—one that has received a fair amount of praise from the developer community.
Apple Silicon, or the M-series of chips, has been an overwhelming success, utilising ARM-based architecture and unified memory to deliver faster CPU and graphics performance. Not only has this led to growth in MacBook sales, but it has also given rise to MLX, an open-source array framework specifically engineered by Apple’s machine learning research team to maximise the potential of the M-series chips.
Introduced in December 2023, it quickly established itself as a dominant force in the open-source AI ecosystem, amassing over 12,000 GitHub stars within months of its release and spawning a dedicated Hugging Face community hosting thousands of highly optimised models.
The Unified Memory Advantage
Designed for developers and researchers, MLX leverages Apple Silicon’s unified memory architecture, which allows the CPU and GPU to share a single pool of physical memory. This eliminates the latency of copying data between system RAM and dedicated video memory, allowing massive AI models to run with unprecedented efficiency directly on Mac hardware.
For instance, Adrien Grondin, Founder of on-device AI app Locally AI, recently shared a demonstration of Google's Gemma 4, a 26-billion-parameter model, running natively on a MacBook Pro equipped with the M5 Max chip, built on top of the MLX framework. He achieved an inference speed of over 100 tokens per second.
Google's Gemma 4 26B A4B running on MacBook Pro M5 Max
100% native Swift
Achieving over 100tk/s with Apple MLX framework pic.twitter.com/j5w1foONU3
— Adrien Grondin (@adrgrondin) April 15, 2026
Similar projects have also been developed to deploy open-source models from DeepSeek, Qwen, and Kimi to run locally on Macs.
Additionally, developers actively port popular open-source projects to MLX.
Even the popular autoresearch project from former OpenAI researcher Andrej Karpathy found itself an MLX port. The project had heavy PyTorch dependencies and required expensive NVIDIA GPUs utilising CUDA to execute continuous training. But a developer adapted the autonomous AI research loop to run entirely natively on Apple Silicon.
I'm open-sourcing an MLX port of the ESM-2 protein model family so more people can tinker with AI & biology on Apple silicon! (1/5) pic.twitter.com/5702rxnQlD
— Joseph Jojoe (@josephjojoe) April 6, 2026
Apple’s MLX decouples the entire training process from traditional hardware constraints to run locally on Mac devices.
Similarly, developers have also built open-source frameworks that leverage MLX to run even multi-modal vision models locally on Macs. This was made possible by Prince Canuma, a developer who built a dedicated package that brings vision-language models to Apple Silicon using MLX.
For example, one developer recently used Canuma’s package to run Meta's powerful new vision AI (SAM3) directly on a Mac. The AI instantly drew precise outlines around multiple moving cars in a video in real-time.
MLX creator @awnihannun sharing the story of MLX. Apple management called him right after the launch: “why didn’t you tell us this was going to be so big??” 😂 pic.twitter.com/tpRN7vWq1p
— Ronald Mannak (@ronaldmannak) April 10, 2026
Local-First Infrastructure
While projects like these highlight MLX's potential for private, dependency-free on-device AI, how does it truly differentiate itself under the hood?
In an interaction with AIM, Manthan Gupta, an AI research engineer, explains that MLX currently occupies a highly specific but increasingly crucial niche: local-first, Apple-native machine learning infrastructure.
He notes that while today’s AI ecosystem is heavily reliant on frameworks built around NVIDIA's cloud-based GPUs, MLX completely breaks from that assumption by treating Apple Silicon as a primary, standalone compute platform.
“If PyTorch is the Linux + CUDA world, MLX is the macOS + unified memory world,” he says. “What’s interesting is that MLX isn’t trying to win on ecosystem breadth yet. It’s winning on tight hardware software co-design, which historically is how Apple plays long games.”
Gupta states that MLX's native use of unified memory eliminates the data-transfer bottlenecks that plague traditional setups. “Compared to PyTorch’s growing complexity or JAX’s functional rigidity, MLX feels closer to ‘just write numpy-like code, and it runs efficiently.’ That matters a lot when iterating locally,” he says.
However, Gupta also states that while MLX does not beat PyTorch in absolute performance ceilings, specifically on MacBooks, it often wins in practical usability per watt.
Furthermore, he explains that MLX fundamentally lowers the learning curve for building AI products through a smart design choice called ‘lazy evaluation’.
In the broader landscape, engineering teams often have to choose between frameworks like PyTorch, which is easy to write but less optimised out of the box, and JAX, which runs extremely fast but forces developers to completely rewire how they think about programming. MLX finds the ideal middle ground. It allows developers to write straightforward, intuitive code while the system automatically figures out the most efficient way to execute it in the background.
“For me, this matters more than raw performance as it lowers the barrier to doing serious work locally,” Gupta remarks. “MLX makes local experimentation fast enough that you start treating your laptop like a real inference lab, not a fallback environment.”
The Llama.cpp Delta and Hidden Hardware
Besides the PyTorch and JAX dynamic, another major point of comparison for developers building local-first tools is Llama.cpp—a highly popular C++ framework designed to run models on practically any device. However, that universal reach comes at the cost of hardware-specific optimisation.
Jesai Tarun, an AI developer currently building Sentient OS, experienced this performance delta while building a product that acts as an ambient, on-device intelligence layer for a user's entire digital life. He found that processing a single screenshot with Llama.cpp took 20 seconds. With MLX, it took three.
“On Apple GPUs, if you run a model with MLX versus Llama.cpp, you literally get four times the time to first token,” he tells AIM.
Interestingly, he points out that Apple actually features five distinct hardware accelerators on its chips, contrary to popular knowledge. While most are familiar with the CPU, GPU, and the Neural Engine, Tarun highlights two more hidden components: neural accelerators embedded within each GPU core and an undocumented ‘Apple Matrix Coprocessor.’
This architectural depth is exactly where MLX gains its massive edge. “Llama.cpp only works on the GPU when you use it on Apple devices,” Tarun explains. “It doesn't use the neural accelerators. It doesn't use the neural engine. It doesn't use the matrix coprocessor.”
Because MLX is engineered to utilise these hidden co-processors simultaneously, it squeezes out performance that hardware-agnostic frameworks simply cannot reach.
He also points out that Apple uses MLX internally on a large scale to power various local, on-device AI features in first-party apps, such as the offline voice transcription on the Voice Memos app.
While MLX is heavily lauded as a win for the open-source community, the secret to its staggering performance—the actual microcode that sends instructions to the hardware—remains behind a proprietary wall.
“The GPU kernels are the actual microcode that sends the instructions to the different hardware units, and that stuff is not open-source,” Tarun explains, noting that these kernels are bundled exclusively into macOS and iOS updates.
Having said that, the framework could grow by helping developers deploy more powerful large-language and vision models directly on iOS. Unlike mature frameworks that feature highly advanced, dynamic quantisation techniques, MLX currently relies on basic quantisation.
“If you are building a laptop or desktop program, it doesn't matter; your computer has so much memory,” Tarun says, noting that developers can easily grab pre-converted MLX models from Hugging Face and start building instantly.
“But if you want to build an app that runs on your phone using MLX, it’s a lot more work because you have to manually do the work of compressing the models in a really intelligent way.”
The Road Ahead
MLX is more relevant than ever, given that on-device AI is considered a ‘bear case’ by industry voices, amidst the rapid growth of high-performance GPUs and data centres.
“We’re moving towards a world where inference is increasingly personalised, latency matters more than raw throughput, and privacy constraints push workloads on device,” says Gupta. “Apple already controls the hardware, the OS. MLX fits as the missing piece: a developer-facing abstraction over that stack.”
Besides, Apple is also experiencing healthy growth for its Mac devices. Across all segments, Mac grabbed a 15.7% share of the US PC market in the last quarter of 2025, according to data from Omdia. It achieved 11.2% growth in 2025, compared to an industry average growth of just 3.3%.
“Right now, MLX is mostly used by hackers and early adopters. But if Apple leans in, MLX could become the default runtime for on-device models in the Apple ecosystem or even the backbone for consumer-facing AI features,” Gupta declares.
Your reaction