Microsoft’s Maia 300 is a 950-watt accelerator with eight HBM4 stacks and 18.0 TB/s of peak bandwidth, and Microsoft plans to scale the Maia family to multi-gigawatt deployments from 2027 as the main inference system for its own products. The announcement came the same day NVIDIA and AMD published rack designs for the Open Compute Project, so this roundup covers three ways to build the rack around the silicon.
Microsoft’s Maia 300 rack holds 96 accelerators and 27.6 TB of HBM in a 167-kilowatt envelope
Maia 300 is built for long-context, decode-heavy inference. Its compute chiplets use TSMC’s N2P node, with memory control and I/O on separate chiplets, and it relies on a software-defined dataflow design and enhanced MXFP4 to cut data movement and KV-cache size. Production deployments begin in 2027. A full rack carries 96 accelerators with 27.6 TB of HBM and 1.7 PB/s of aggregate bandwidth.
Dividing that out gives about 288GB of HBM per accelerator. The power split is less obvious: 96 accelerators with a 950-watt TDP come to 91.2kW, about 55% of the 167-kilowatt rack figure, and the post does not itemize what draws the rest. Both numbers are arithmetic on Microsoft’s figures, and a TDP is a design limit, not a measured draw.
The efficiency claims are Microsoft’s projections for the new part. It expects Maia 300 to deliver 40% more tokens per megawatt than leading GPU-based systems, and a full 96-accelerator configuration to reach up to 7x the rack-scale token throughput of Maia 200. Maia 200 is the part in service today: it runs Copilot, with GitHub Copilot and Microsoft Foundry to follow by the end of the year, and Microsoft puts it at up to 1.7x more tokens per watt for features in Teams, Copilot and Cowork. The post does not name the GPU system either comparison uses.
NVIDIA’s MGX 1.2 specification supports racks up to 250kW with a liquid-cooled 5,000A busbar
NVIDIA contributed Vera Rubin rack and compute tray designs to the Open Compute Project through the third generation of its MGX architecture. In its OCP post, NVIDIA lists what MGX 1.2 adds: blind-mate liquid cooling at rack scale, interoperable rack-scale interconnect, a liquid-cooled busbar, native 19-inch equipment support, fanless liquid-cooled compute trays and configurable I/O bays. The architecture supports racks up to 250kW.
That is a ceiling for the specification, not a Vera Rubin rack’s draw. The post gives no power figure for a Vera Rubin NVL72 rack, so it likely cannot be set against Maia 300’s 167-kilowatt rack. A separate Vera CPU rack design extends MGX to tool calling, orchestration and sandboxed execution for AI agents.
AMD’s Helion rack comes in a CPU-only version with up to 32,768 cores and an EPYC plus Instinct MI350P version
AMD introduced Helion as an open rack blueprint based on an OCP specification design. The press release describes two example builds. The CPU-only rack uses 6th Gen EPYC processors for up to 32,768 cores, aimed at hyperscalers that run agents on dedicated capacity and connect to separate inference infrastructure. The all-in-one rack combines EPYC CPU nodes, Instinct MI350P GPU nodes and general-purpose compute for enterprises. AMD says the blueprint allows liquid-cooled and air-cooled versions, and Helion includes a token router that directs an agent’s model calls across frontier and open-source models.
AMD plans to contribute the specification to OCP. Rack configurations are under development with ecosystem partners and are planned for November 2026; Cisco, Dell, Hon Hai, HPE, Lenovo, QCT, Supermicro and Wiwynn each gave a supporting statement. No power or performance figures were given.