Microsoft Reveals Customized 128-Core Arm Datacenter CPU, Large Maia 100 GPU Designed for AI
Microsoft‘s AI-first initiatives have introduced it to customized chips. At Microsoft Ignite, the corporate’s convention for companion builders, Microsoft introduced two customized chips designed in its Redmon, Washington silicon lab: the Azure Maia 100 AI Accelerator and Azure Cobalt 100 CPU. Each chips are being produced on “the newest course of node” at TSMC, Microsoft advised Tom’s {Hardware}.
These chips will present up “early subsequent yr” in Microsoft’s knowledge facilities, which can begin by operating Microsoft’s Copilot, Azure OpenAI and different companies. When requested what number of chips might be out there, a spokesperson solely mentioned that Microsoft is within the “first phases of deployment” with “preliminary servers” on-line in its knowledge facilities. The corporate will nonetheless supply companies to prospects utilizing chips from companions like Nvidia and AMD.
Microsoft is not releasing a lot in the best way of spec particulars, making it tough to check the brand new choices to Nvidia and AMD choices.
When requested for extra particulars, Microsoft advised Tom’s {Hardware} that Maia 100 is manufactured on a 5-nanometer course of and boasts 105 billion transistors, “making it one of many largest chips that may be made with present know-how.”
“Maia’s servers are designed with a totally customized Ethernet-based community protocol with mixture bandwidth of 4.8 terabits per accelerator to allow higher scaling and end-to-end workload efficiency,” Microsoft added.
What we do know is that the Maia 100 AI Accelerator – a GPU – is designed for big language fashions. Microsoft says that OpenAI “supplied suggestions” in regards to the Maia 100 designs.
“We had been excited when Microsoft first shared their designs for the Maia chip, and we’ve labored collectively to refine and check it with our fashions,” OpenAI CEO Sam Altman said in Microsoft’s weblog submit. “Azure’s end-to-end AI structure, now optimized right down to the silicon with Maia, paves the best way for coaching extra succesful fashions and making these fashions cheaper for our prospects.”
Brian Harry, a Microsoft technical fellow heading up the Maia workforce, mentioned that Maia 100 was made particularly for Microsoft’s present infrastructure, and that it may “yield large good points in efficiency and effectivity.”
In the meantime, the Cobalt 100 CPU is Arm-based, which Wes McCullough, company vice chairman of {hardware} product improvement mentioned within the weblog submit, “is designed with energy effectivity in thoughts.”
“Azure Cobalt makes use of a licensed design from Arm primarily based on ARM Neoverse CSS that’s custom-made for Microsoft’s functions,” Microsoft advised Tom’s {Hardware}. “The 64-bit 128-core chip represents efficiency enhancements of as much as 40% over present generations of Azure Arm servers.”
The corporate additionally states that designing its personal chips permits it to extend cooling in knowledge facilities and to spice up the capability in present amenities. The corporate mentioned that there have been no server racks that would “home the distinctive necessities of the Maia 100 server boards,” which required constructing racks from scratch.
“These racks are wider than what sometimes sits within the firm’s datacenters,” Microsoft’s weblog submit reads. “That expanded design supplies ample area for each energy and networking cables, important for the distinctive calls for of AI workloads.”
The corporate additionally wanted to provide you with a liquid cooling answer to relax these chips throughout advanced AI duties, however did not have knowledge facilities arrange for “giant liquid chillers.” So Microsoft is utilizing what it calls a “sidekick” that may sit subsequent to a rack of Maia 100 servers, ship chilly liquid to chilly plates hooked up to the Maia 100 chips, after which ship the nice and cozy liquid again to the sidekick to take away the warmth and return the liquid to the rack once more.
Microsoft says it has shared the customized rack and cooling designs with business companions.
“All of the issues we construct, whether or not infrastructure or software program or firmware, we are able to leverage whether or not we deploy our chips or these from our business companions,” mentioned Pat Stemen, companion program supervisor on Microsoft’s Azure {Hardware} Techniques and Infrastructure workforce. “This can be a alternative the client will get to make, and we’re attempting to offer the most effective set of choices for them, whether or not it’s for efficiency or price or some other dimension they care about.”
The corporate is already engaged on second-generation variations of each the Maia AI Accelerator and Azure Cobalt CPU.
Along with the brand new {hardware}, Microsoft additionally introduced Azure Increase, which it says could make storage and networking processes faster by transferring them from host servers to {hardware} and software program designed particularly for these functions.
Microsoft additionally introduced additional partnerships with each AMD and Nvidia. On the Workforce Purple facet, Microsoft will add AMD MI300X digital machines to its Azure platform. For Ndvidia, there is a preview of the brand new NC H100 v5 Digital Machine Collection designed for Nvidia’s H00 Tensor Core GPUs.
Many particulars about Maia and Cobalt weren’t out there when this was written. However we now have reached out to Microsoft with additional questions, so anticipate extra protection right here quickly. However one factor is for sure: In a crowded panorama of bespoke, cloud-focused knowledge heart CPUs with gamers like Meta, Oracle, and Google, this transfer from Microsoft is prone to have an enormous affect. Outdated-school knowledge heart mainstays like Intel, AMD, and Nvidia, in addition to comparative newcomers like Ampere, are prone to be impacted by these bulletins and should make daring strikes of their very own.