Microsoft on Monday, Oct. 12 unveiled Maia 300, the third generation of the AI chip it designs itself, and said its Maia systems will become the primary hardware for inference across its products and services. Inference means running a trained model to answer requests, as opposed to training it, and it is what happens each time someone asks Copilot a question. Microsoft says it will expand Maia to deployments drawing gigawatts of power starting in 2027, the year Maia 300 is due to go into service.
Microsoft's pitch is efficiency. A data center has a fixed amount of power and cooling, and Microsoft says it designed Maia so that every watt and every dollar delivers more AI output. It expects Maia 300 to produce 40% more tokens, the small chunks of text, often parts of words, that AI models read and write, for each megawatt of power. Its comparison is with leading systems built on GPUs, advanced AI chips, the hardware AI models are trained and run on, which other companies make. Microsoft also says a full rack, the cabinet that holds the chips in a data center, of 96 Maia 300 chips turns out up to seven times as many tokens in the same time as a rack of the current Maia 200, which holds half as many chips.
The current chip is already at work. Microsoft says Maia 200 now runs Microsoft Copilot, with GitHub Copilot and its Foundry service for developers to follow by the end of the year, and that the chip runs OpenAI's models, Microsoft's own MAI models and popular open models, which anyone can download and run. Maia 200 is being installed in the United States, Europe and Australia, and Microsoft says it gets up to 1.7 times as much output per watt as leading GPU systems for the AI built into Teams and Copilot.
According to a news report, the shift would let Microsoft rely mainly on its own chips for running AI, instead of ones from Nvidia and AMD. Microsoft's own post names neither company, and says its future will not be defined by a single piece of silicon. Mustafa Suleyman, chief executive of Microsoft AI, said his team needs a chip platform it owns and designs alongside its models, according to the same report.
The efficiency figures are Microsoft's own, and the 40% is a forecast for a chip that is not yet in service. Microsoft has not said which GPU systems it compared Maia with, how much of its AI work will still run on other companies' chips, or whether its cloud customers will be able to rent Maia directly. It says it will share more at its Ignite conference in San Francisco next month.