Microsoft has unveiled Maia 300, its third-generation in-house artificial intelligence (AI) inference chip, as the company prepares to expand its custom silicon infrastructure across Azure data centers beginning in 2027. The new accelerator is designed to handle growing demand for AI services, including Microsoft Copilot, coding assistants, enterprise applications and autonomous AI agents. The company expects Maia 300 to improve computing efficiency while helping it manage the cost and power requirements of large-scale AI operations.
The announcement marks another step in Microsoft’s effort to develop more of its own AI hardware rather than relying entirely on processors supplied by Nvidia and other semiconductor companies. Microsoft said its Azure Maia family will expand to multi-gigawatt deployments from 2027, with Maia 300 designed to become the primary inference system for its products and services. The company’s performance figures are based on its own stated comparisons, and actual results will depend on the workloads and deployment configurations involved.
Key Takeaways
- New AI chip: Maia 300 is the third generation of Microsoft’s custom Azure Maia accelerator family.
- Production timeline: Microsoft plans to begin production deployments in 2027.
- Higher efficiency: The company expects 40% more tokens per megawatt than leading GPU-based systems.
- Rack-scale performance: A configuration with 96 accelerators is designed to deliver up to seven times the token throughput of Maia 200.
- Advanced architecture: The chip uses a multi-chiplet design, HBM4 memory and high-speed networking to support demanding AI inference workloads.
- Strategic objective: Microsoft aims to improve AI infrastructure economics and reduce its dependence on external accelerator suppliers.
What Is Microsoft Maia 300?
Maia 300 is a custom AI accelerator developed by Microsoft for inference workloads. Unlike general-purpose processors, AI accelerators are designed to handle the mathematical operations required to run machine-learning models efficiently.
Inference occurs when a trained AI model processes a prompt and generates an answer, completes a coding task or performs another operation. As AI assistants and agents handle increasingly complex tasks, inference infrastructure must support longer contexts, larger models and more simultaneous requests.
Microsoft designed Maia 300 around these requirements. Its architecture targets long-context reasoning, low-latency conversational applications and sustained, multi-step agentic workloads. These applications can require models to process substantial amounts of information, generate many tokens and interact with tools repeatedly.
The chip is part of Microsoft’s broader Azure Maia program, which combines custom silicon with software, networking, server racks and data-center infrastructure. Rather than optimizing a processor in isolation, the company says it designs these components together to improve overall performance and energy efficiency.
Maia 300 Specifications and Expected Performance
Microsoft has disclosed several technical details about the new accelerator, including its memory architecture, power envelope and rack-level configuration.
| Specification | Maia 300 details |
|---|---|
| Generation | Third-generation Azure Maia |
| Primary purpose | AI inference |
| Production deployment | Planned for 2027 |
| Peak memory bandwidth | 18.0 TB/s |
| Memory technology | Eight HBM4 stacks |
| Chiplet manufacturing | Compute chiplets use TSMC’s N2P process |
| Thermal design power | 950 watts |
| Full rack configuration | 96 accelerators |
| Rack power envelope | Approximately 167 kW |
| Aggregate rack HBM capacity | 27.6 TB |
| Aggregate HBM bandwidth | 1.7 PB/s |
| Claimed efficiency improvement | 40% more tokens per megawatt than leading GPU-based systems |
| Claimed rack throughput | Up to 7x Maia 200 |
Source: Microsoft’s official Maia 300 announcement and architecture documentation. Performance figures are company-reported specifications and expectations, not independent benchmark results.
Why Memory Bandwidth Matters
AI inference performance depends on more than the raw processing power of a chip. Models must continually access their parameters and intermediate information, including the key-value cache used to retain context during generation.
As prompts become longer and models perform more complex reasoning, moving data between memory and computing units can become a significant bottleneck. High memory bandwidth helps supply data to the processor more quickly.
Maia 300 uses eight HBM4 memory stacks with a stated peak bandwidth of 18.0 terabytes per second. Microsoft says the design also uses on-chip SRAM as a software-managed scratchpad to reduce pressure on high-bandwidth memory.
The objective is to keep computing resources occupied with useful work instead of waiting for data to arrive. Whether this advantage translates into better performance depends on the model, software implementation and inference workload.
How Maia 300’s Chiplet Architecture Works
Microsoft is using a multi-chiplet design that separates computing functions, memory controllers and input/output components into specialized sections. These components are integrated into a broader processor system.
The design allows different parts of the chip to be optimized for their specific responsibilities. Microsoft says its compute chiplets use TSMC’s N2P manufacturing process, while the wider architecture is designed to balance performance, manufacturing cost and data movement.
Chiplet architectures can provide flexibility when building complex processors, although they also introduce engineering challenges involving communication between components, packaging, power delivery and thermal management.
Microsoft has also developed a software-defined dataflow architecture intended to coordinate how data moves through computing units, memory and networking resources. This approach is particularly relevant to inference systems, where performance can be constrained by data movement as well as arithmetic operations.
The company is positioning Maia 300 as a complete computing platform rather than simply a replacement processor. Its performance depends on how the chip works with the software stack, memory, network and physical infrastructure surrounding it.
Microsoft Targets 40% More AI Output per Megawatt
One of Microsoft’s central claims is that Maia 300 will deliver 40% more tokens per megawatt than leading GPU-based systems.
A token is a small unit of text processed or generated by an AI model. Depending on the model and language, tokens may represent words, parts of words, punctuation or other text elements.
Tokens per megawatt is an efficiency measure intended to show how much AI output can be produced for a given amount of power. Higher output at the same energy consumption can help cloud providers serve more requests without increasing electricity use proportionally.
For Microsoft, this matters because inference demand is expanding across consumer and enterprise products. Every AI-generated response consumes computing resources, and sustained growth in usage can increase both infrastructure costs and electricity requirements.
However, the 40% figure should be treated as Microsoft’s stated comparison. The company has not named all the specific GPU systems used for the comparison in its announcement, and performance can vary substantially between models and workloads.
A chip that performs particularly well on one inference workload may not deliver the same advantage on another. Independent benchmarks and production data will be important in assessing Maia 300’s real-world efficiency after deployment.
Maia 300 vs Maia 200: What Has Changed?
Maia 300 builds on Microsoft’s experience deploying Maia 200 in production environments. Microsoft says Maia 200 is already serving workloads that include Copilot and supporting the development of its own AI models.
| Feature | Maia 200 | Maia 300 |
|---|---|---|
| Status | In production | Production deployment planned for 2027 |
| Primary focus | Large-scale AI inference | More demanding, memory-intensive inference |
| Memory architecture | Earlier-generation design | Eight HBM4 stacks |
| Rack configuration | Earlier-generation system | 96 accelerators |
| Efficiency claim | Up to 1.7x more tokens per watt for certain built-in AI capabilities versus leading GPU infrastructure | 40% more tokens per megawatt versus leading GPU-based systems |
| Rack-level throughput | Baseline for Microsoft’s comparison | Up to 7x Maia 200 in a full 96-accelerator configuration |
The figures in this table describe different comparisons and should not be combined into a single performance ranking. Microsoft’s Maia 200 efficiency claim applies to specified built-in AI capabilities, while the Maia 300 figures describe the company’s expected system-level performance.
Microsoft’s experience with Maia 200 also provides information about the workloads its infrastructure must support. The company says that experience influenced Maia 300’s memory, computing and networking design.
How Maia 300 Could Affect Microsoft Copilot and Azure
Microsoft intends to use its Maia infrastructure across a growing portfolio of AI services. The company has identified coding assistance, cloud-scale model serving, AI development and latency-sensitive conversational applications among the workloads its custom hardware is designed to support.
Microsoft Copilot and Teams
AI assistants need to respond quickly while serving many users at once. More efficient inference infrastructure could help Microsoft handle higher request volumes and reduce the computing cost associated with individual interactions.
Microsoft Teams and other productivity applications could also benefit from improved capacity for AI features. The actual user experience, however, will depend on software, model selection and how Microsoft allocates its infrastructure.
GitHub Copilot and Microsoft Foundry
Coding assistants can involve multi-step tasks, long context windows and repeated model calls. Microsoft expects Maia systems to support more efficient coding experiences through GitHub Copilot.
Microsoft Foundry, its platform for developing and deploying AI applications, could also benefit from additional inference capacity and support for diverse model architectures. The company has said GitHub Copilot and Microsoft Foundry workloads are expected to follow Maia 200’s initial production uses.
Dragon Copilot and Enterprise AI
Microsoft is also targeting healthcare and other enterprise applications. Dragon Copilot, for example, is designed to support clinical workflows and reduce administrative burdens.
More efficient inference could help scale such applications, but chip performance alone does not determine their usefulness. Reliability, security, regulatory requirements and integration with existing workflows remain important factors.
Microsoft’s Strategy to Reduce Dependence on Nvidia
Nvidia has a central role in the AI accelerator market, supplying processors and supporting software used across the industry. Microsoft’s custom-chip initiative gives it another option for meeting the computing requirements of its cloud services.
Developing its own silicon could provide Microsoft with greater control over the processor design, memory system, networking and software stack. It may also help the company negotiate its overall infrastructure costs and allocate different workloads to the hardware best suited to them.
The move does not mean Microsoft will stop buying Nvidia processors. Large cloud providers often use multiple types of accelerators because different workloads have different requirements. Nvidia’s software ecosystem, installed base and performance across a wide range of AI applications remain relevant considerations.
Custom silicon also requires substantial investment, engineering expertise and manufacturing capacity. Microsoft must ensure that its hardware performs well in production and that software developers can use it effectively.
The competitive landscape is expanding as major cloud providers develop their own processors. Google has its Tensor Processing Units, while Amazon has developed Trainium and Inferentia. Microsoft’s Maia program is part of this broader shift toward custom AI infrastructure.
The Data Center and Energy Challenge
Microsoft’s planned Maia 300 rollout also highlights the energy demands associated with large-scale AI infrastructure.
The company says a full Maia 300 rack containing 96 accelerators will operate within a power envelope of approximately 167 kilowatts. At that scale, improving the amount of useful AI output produced for each unit of electricity becomes an important consideration.
Yet better efficiency does not automatically reduce total energy consumption. If lower operating costs encourage greater AI usage, overall electricity demand can continue to rise. Data-center operators must also manage cooling, networking, power delivery and the physical space required for increasingly dense systems.
Microsoft plans to expand Maia deployments to multi-gigawatt scale beginning in 2027. The scale of that ambition illustrates how the next phase of AI competition will involve not only chips, but also electricity supply, cooling systems, manufacturing capacity and data-center construction.
The Bigger Picture
Maia 300 reflects a broader change in the AI industry: competition is increasingly focused on the cost and efficiency of running models at scale, not just on building more powerful models. As AI assistants become embedded in everyday software and businesses adopt multi-step agents, inference infrastructure becomes a major determinant of service economics.
Microsoft’s approach is to optimize the entire system, combining custom processors, memory, software and networking. If its performance claims hold up under independent testing and real production workloads, Maia 300 could give the company more flexibility in scaling AI services while managing energy consumption and hardware costs. However, the extent of any competitive advantage will depend on implementation, software compatibility and the performance of competing accelerators.
Looking Ahead
Microsoft plans to begin Maia 300 production deployments in 2027, with further expansion of its Azure Maia infrastructure. The next milestones will include deployment scale, independent performance comparisons and evidence of how effectively the chips support real-world workloads across Copilot, coding tools and Azure AI services. The company’s stated targets provide an indication of its ambitions, but production results will offer a clearer view of the chip’s performance and economics.
The announcement also signals that custom silicon will remain an important part of cloud providers’ AI strategies. Microsoft is not abandoning third-party accelerators, but it is seeking greater control over a critical part of its infrastructure. As AI usage grows, the ability to deliver reliable model inference at competitive cost and within data centers’ power limits will help shape the next stage of competition among cloud and semiconductor companies.
Get the day’s top stories in your inbox
One concise email. No spam, unsubscribe anytime.
