OpenAI has published the first detailed benchmark results for Jalapeño, its custom artificial intelligence inference chip developed in collaboration with Broadcom, showing substantial gains in performance per watt and response latency compared with the Nvidia systems used in the company’s tests. The results were presented at the Hot Chips conference and are based on SemiAnalysis’ public InferenceX benchmark, covering several large language models.
The results mark an important step in OpenAI’s effort to build more of its AI infrastructure in-house. Jalapeño is designed specifically for inference, the stage at which trained AI models generate responses for users, rather than for training models. OpenAI says the chip can combine high throughput with low latency, addressing a trade-off that often forces conventional systems to prioritize either serving many users efficiently or delivering very fast responses.
OpenAI’s Jalapeño Delivers Higher Performance Per Watt
OpenAI tested Jalapeño across GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T using the InferenceX benchmark. The company compared its system with commercial Nvidia hardware and measured performance at different operating points, rather than relying on a single peak-performance figure.
Across the three models, OpenAI reported that Jalapeño delivered between 1.5 times and 1.9 times more AI work per watt at peak throughput. End-to-end latency was between 1.7 times and 3.6 times lower than the comparison systems.
Key Jalapeño Benchmark Results
| Model | Performance Per Watt Advantage | Lower End-to-End Latency | Comparison System |
|---|---|---|---|
| GPT-OSS 120B | ~1.9x | ~1.7x | Nvidia Blackwell |
| DeepSeek R1 670B | ~1.7x | ~3.6x | Nvidia Blackwell |
| Kimi K2.5 1T | ~1.5x | ~3.4x | Nvidia Blackwell |
The figures are particularly relevant for large-scale AI services because power consumption is becoming a major component of data-center operating costs. A chip that can process more AI work with less power could allow an operator to serve more requests within the same energy budget.
OpenAI says Jalapeño also achieved strong results at lower-latency operating points, suggesting that the chip is not optimized only for maximum throughput.
Jalapeño Targets The Bottlenecks In AI Inference
The architecture is designed around the specific requirements of large language model inference. OpenAI says the chip minimizes data movement and communication delays, two factors that can become increasingly important as models grow larger.
During inference, the system must move model data and maintain information about the ongoing conversation or task. The key-value cache, commonly known as the KV cache, is particularly important because it stores information needed to generate subsequent tokens.
Jalapeño is designed so that model state, including the KV cache, can remain local while the system activates the appropriate combination of compute, memory and networking for different phases of an inference workload.
This approach is intended to reduce unnecessary movement of data between components.
Why Latency Matters More For AI Agents
Low latency is becoming increasingly important as AI applications evolve from simple chatbot interactions toward agents that perform multiple steps on behalf of users.
An agent may need to generate a response, call a tool, analyze the result and generate another response before completing a task. Delays at each stage can accumulate.
OpenAI therefore argues that a system capable of maintaining high throughput while reducing token latency can be particularly valuable for interactive AI applications.
Jalapeño’s Results Across Large Models
The benchmark results span models with substantially different sizes and architectures. GPT-OSS 120B is the smallest of the three models tested, while Kimi K2.5 1T represents a much larger model with roughly one trillion parameters.
The performance gains remained significant across the range.
DeepSeek R1 670B
For DeepSeek R1 670B, OpenAI reported approximately 1.7 times higher peak mixed tokens-per-second per kilowatt than the comparison system.
The company measured 19,641 mixed tokens per second per kilowatt for Jalapeño versus 11,781 for the comparison system. End-to-end latency was approximately 1.65 seconds compared with 5.99 seconds.
At the lowest reported time-between-tokens measurement, Jalapeño reached about 1.43 milliseconds versus 5.90 milliseconds for the comparison system.
Kimi K2.5 1T
The Kimi K2.5 1T results were also notable because of the model’s size.
OpenAI reported approximately 18,195 mixed tokens per second per kilowatt for Jalapeño against 11,862 for the comparison system, representing about a 1.5-times advantage.
End-to-end latency was approximately 1.56 seconds versus 5.31 seconds.
| Metric | Jalapeño | Comparison System |
|---|---|---|
| Mixed TPS/kW | 18,195 | 11,862 |
| End-to-end latency | 1.56 sec | 5.31 sec |
| Minimum TBT | 1.44 ms | 5.48 ms |
| Peak advantage | ~1.5x | — |
These results suggest that OpenAI’s architecture is intended to remain effective as model sizes increase rather than being optimized around a single workload.
The Chip Is Designed With A 700-Watt Rating
OpenAI lists Jalapeño’s package power rating at 700 watts. However, the company said measured sustained power remained at or below 550 watts on the workloads tested.
This distinction matters when evaluating performance-per-watt claims because AI accelerators can operate at different power levels depending on workload and configuration.
OpenAI’s methodology uses published chip power ratings to normalize comparisons. The company argues that performance per unit of power is more useful for evaluating modern AI infrastructure than comparing individual chips solely on raw throughput.
Jalapeño Versus Conventional AI Infrastructure
| Factor | Jalapeño Approach | Traditional Accelerator Approach |
|---|---|---|
| Primary workload | AI inference | Broad AI workloads |
| Design focus | LLM serving | General-purpose acceleration |
| Optimization | Latency + throughput | Varies by system |
| Package power rating | 700W | Depends on accelerator/system |
| Sustained power in tests | ≤550W | Depends on workload |
| Development partner | Broadcom | Multiple hardware vendors |
| Commercial availability | Internal OpenAI deployment | Commercial systems available |
The comparison should nevertheless be viewed in context. The benchmark results reflect specific configurations and workloads, and Jalapeño is not yet broadly available as a commercial accelerator.
OpenAI Is Building A Full AI Hardware Stack
Jalapeño represents a broader strategic shift for OpenAI. The company has historically depended heavily on external hardware suppliers for the computing infrastructure needed to train and run its models.
By designing its own inference silicon, OpenAI can tailor hardware to the characteristics of its models and products.
The company previously announced Jalapeño in June, describing it as an LLM-optimized inference processor developed with Broadcom. OpenAI said the chip was designed and brought to production in nine months, with its own AI models helping accelerate the development process.
OpenAI has also described Jalapeño as the beginning of a multigenerational hardware platform rather than a one-off chip.
Jalapeño Is Not A Replacement For Nvidia Yet
Despite the strong benchmark results, OpenAI’s chip does not immediately eliminate its dependence on Nvidia or other external hardware providers.
TechCrunch reported that OpenAI expects Jalapeño to reach deployment in very small volumes toward the end of 2026, with more significant deployment expected in 2027. That creates an important qualification around the benchmark results: the Nvidia hardware available when Jalapeño reaches large-scale deployment could be significantly newer than the systems used in the current comparison.
The current results therefore demonstrate the potential of OpenAI’s architecture, but they do not establish how Jalapeño will compare with every future generation of commercial AI accelerators.
Why Inference Chips Are Becoming Strategic
The economics of AI are increasingly shifting toward inference as millions of users interact with models and businesses deploy AI agents at scale.
Training frontier models requires enormous computing resources, but once a model is deployed, every user request generates an ongoing inference workload. Improvements in energy efficiency and latency can therefore have a direct impact on the cost and responsiveness of AI products.
This is encouraging major technology companies to develop specialized chips. Google and Amazon have already developed custom AI accelerators, while Microsoft has also pursued proprietary silicon. OpenAI’s move places it within this wider industry trend.
For OpenAI, the potential advantage is not simply owning a chip. It is the ability to coordinate models, software, memory, networking and silicon around the workloads generated by its products.
The Bigger Picture
Jalapeño’s first benchmark results show why custom inference hardware is becoming strategically important for AI companies. OpenAI is reporting substantial improvements in both performance per watt and latency, two metrics that directly affect the economics and user experience of large-scale AI services.
The more significant development, however, is OpenAI’s move toward a full-stack approach. By designing hardware around its own models and serving software, the company can potentially optimize bottlenecks that are difficult to address with general-purpose commercial accelerators. Whether those gains translate into lower costs and better products at production scale will become clearer once Jalapeño is deployed more broadly.
Looking Ahead
OpenAI expects only limited Jalapeño deployment toward the end of 2026, with larger-scale deployment planned for 2027. The company will therefore have to prove that the benchmark advantages can be maintained under real-world workloads involving different model sizes, utilization levels and customer demand. The economics of manufacturing, data-center integration and software optimization will also determine how valuable the chip becomes.
The competitive landscape will continue to evolve before Jalapeño reaches significant scale. Nvidia and other accelerator suppliers are developing newer generations of hardware, while competing technology companies are also building custom silicon. OpenAI’s current results provide evidence that specialized inference hardware can deliver meaningful gains, but the long-term test will be whether Jalapeño and its successors can turn those technical advantages into durable improvements in AI cost, speed and availability.
Get the day’s top stories in your inbox
One concise email. No spam, unsubscribe anytime.
