Microsoft wants to make its own AI chips. According to a report by The Information, the company is currently engaged in the development of its AI chips, called Athena, a project the tech giant has been working on since 2019. Currently, the chips are being tested by a small group of employees from Microsoft and OpenAI. But what we don’t know yet is if and when Microsoft plans to bring the chip out for commercial use. 

So far, to run its AI models, Microsoft has been relying on chips available in the market, which is dominated by Nvidia. To help OpenAI train its GPT series large language models (LLMs), the Redmond-based company also built a supercomputer for OpenAI, powered by Nvidia A100 chips.

Given the rapid advancement in the field, AI companies such as Google, Apple and Amazon have already developed their AI chips. Google, in fact, has built a supercomputer to train its models with its TPUs (Tensor Processing Units). Amazon also has its Trainium and Inferentia processor architectures. Hence, it’s no surprise that Microsoft is walking down the same lane. 

Optimising cost 

It is estimated that just training an LLM like GPT-3 could have cost OpenAI around USD4 million. Additionally, OpenAI spends approximately USD 3 million per month to sustain ChatGPT. Besides, GPUs used to run these models are also very expensive. Nvidia, the leading producer of GPUs for the AI industry, sells its primary data centre chips for around USD10,000. Moreover, its H100 GPUs are being sold at USD40,000 on eBay. Reportedly, Microsoft’s supercomputer utilises tens of thousands of Nvidia’s H100 and A100 data centre GPUs.

One of the primary reasons for Microsoft to build its own chips is to ​​reduce its costs. As per the report, Athena could possibly reduce the cost per chip by a third in comparison to Nvidia.

Besides, Microsoft also wants to reduce its reliance on Nvidia. Hence, building chips in house could mean Microsoft can design the chips, its architecture, and compatibility according to their own needs. As per the report, Microsoft has designed Athena for both training as well as running its AI models.

Given that Microsoft is aiming to introduce AI-powered features in Bing, Office 365, and GitHub, the migration could prove advantageous for the company in terms of cost. “As needs expand and diversity of use expands as well, it’s important for Microsoft and the other hyperscalers to pursue their own optimised versions of AI chips for their own architectures and optimised algorithms (not CUDA-specific),” Jack Gold, of J Gold Associates, told VentureBeat. 

On device AI

Typically, AI models are run on the cloud. However, recent advancements have shown that it’s possible to run AI models entirely on devices. Very recently, a group of Qualcomm engineers managed to run text-to-image AI model Stable Diffusion on an Android device. 

Currently, there are laptops in the market that have chips designed to help train AI models. Microsoft too, makes premium laptops with hi-end hardware under the Surface brand. Microsoft is also reportedly designing its own ARM-based processors for its Surface laptops; however, these chips have not yet been released. 

Hence, we could possibly see a future generation of Surface laptops that run a ChatGPT like model completely on the device. 

Competition with Nvidia?

Currently, Nvidia is the leading supplier of AI chips holding more than 90% of the enterprise GPU market. While Nvidia is focussed on building GPUs, Microsoft’s focus lies elsewhere. It wants to bring its AI products to the enterprise. 

Currently, Microsoft is building Athena for its own internal use. Hence, it’s unlikely that Microsoft will be in competition with Nvidia in the chips market. For now, Microsoft’s aim is to lower its cloud operating costs. However, down the line, Microsoft could use Athena to enhance its cloud services and devices, offering better performance and lower costs than its competitors. 

A challenge for developers?

At present, it is uncertain if Microsoft will offer these chips to its customers using the Azure cloud service. If Microsoft decides to run Azure on Athena, the newer chips might pose a challenge for developers because the chips may differ from existing ones in terms of their design, performance, power usage, and compatibility with existing software and frameworks. 

Initially, developers may face challenges in migrating to the new AI chips due to these differences and may need to adapt their workflows and code accordingly. Transitioning their existing code to the new platform could prove to be challenging.  Further, developers might require detailed documentation and resources on the new AI chips, including tutorials, sample code, and compatibility guidelines.

While the report suggests that Microsoft’s Athena chips may be available within a year, it may take some time for them to become widely available.