Microsoft's Maia 300 AI Chip Aims To Make AI Cheaper And Faster

As Microsoft continues to build-out multi-gigawatt cloud and AI infrastructure deployments, its custom silicon remains a major focus. Today Microsoft reiterated that its Maia 200 is production-proven and deployed across regions in the United States, Europe, and Australia, and that the next-generation Maia 300, which reportedly boosts AI inference performance significantly, is coming in 2027.

For those unfamiliar, the Azure Maia 200 is actively running limited Microsoft Copilot workloads, OpenAI models, internal MAI models, and many open-source frameworks. Microsoft claims Maia 200, which is designed for real-time inference and model development, offers 1.7x higher token throughput per watt for core AI features in Microsoft Teams, Copilot, and Cowork use-cases compared to leading GPUs.

Azure Maia 300 Will Be An Inference Powerhouse

According to Microsoft, the next-generation Azure Maia 300 is a ground-up redesigned, inference-first architecture tailored specifically for memory-intensive, long-context reasoning and latency-critical agentic tasks. The company is claiming Maia 300 will offer 40% more tokens per megawatt than competing GPU solutions, with a full 96-accelerator rack configuration pushing up to 7x the rack-scale token throughput of Maia 200. A full Maia 300 rack will feature 96 accelerators within a 167-kilowatt power envelope, with 27.6 TB of aggregate HBM capacity and 1.7 PB/s of aggregate HBM bandwidth per rack.


Like a number of other architectures announced over the last few months, efficient, optimized data movement was a major design focus of the Maia 300. The architecture combines AI-assisted kernel optimization and on-die SRAM, which will act as a “software-managed scratchpad” to optimize access to memory across compute and the fabric layer, to minimize the need to access the HBM.

The Maia 300: Big Memory Bandwidth And Optimized Data Movement

The Maia 300 features 8 stacks of HBM4 memory to deliver an impressive 18.0 TB/s of peak memory bandwidth within a stout 950-watt Thermal Design Power (TDP). It also leverages a modular chiplet architecture separating compute, memory controllers, and I/O blocks. The Maia 300’s compute chiplets are built on TSMC's advanced N2P process node, but information about the other chiplets wasn’t mentioned in Microsoft’s announcement.

The Maia 300 also features advanced low-precision support for various datatypes, including enhanced MXFP4, to compress memory footprints and lower bandwidth consumption. The chip features high-speed tensor-parallel interconnects as well, with a custom transport protocol over standard Ethernet-based infrastructure. According to Microsoft, this allows seamless scale-up networking from single-racks to multi-gigawatt many-rack datacenter deployments.

By co-designing and optimizing the silicon, software stacks, and cloud infrastructure, Microsoft's believes its Maia architecture will be a highly-efficiency foundation for next-generation frontier AI within Azure.