OpenAI has released the first measured results for Jalapeño, its own custom inference chip[1]. Benchmarked against leading commercially available systems, the chip delivered 1.5 to 1.9 times more AI work per unit of power and 1.7 to 3.6 times lower end-to-end latency. Throughput and low latency usually pull against each other, but OpenAI says a single architecture handled both without forcing that tradeoff.
Three public models measured on a public benchmark
The testing was done on InferenceX, a public benchmark from SemiAnalysis that measures the full process of serving an AI request. OpenAI compared Jalapeño with leading commercially available AI systems across the tested operating range, from high-throughput serving to highly interactive, low-latency use[1].
Three models were covered: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. All are public models, and they include work developed both inside and outside OpenAI. Across all three, Jalapeño produced a better combination of performance per watt and latency, placing it on the Pareto frontier, the line beyond which one metric cannot improve without sacrificing the other[1].
In numbers, that means 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency. For highly interactive workloads, the gap widened to 2.1 to 4.1 times higher performance. On Kimi K2.5 1T, the largest model tested, Jalapeño delivered roughly 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency[1].
OpenAI adds that the advantage widened further in internal testing on its frontier models, suggesting the architecture pays off more as workloads grow larger and more demanding. That figure sits outside the public benchmark, however, and cannot be reproduced by anyone else.
Rated at 700 watts, measured at 550 watts or below
OpenAI argues that systems should be compared per unit of power rather than per chip. Results were normalized using each accelerator's published chip power rating. Jalapeño itself is rated at 700 watts, though its measured sustained power stayed at or below 550 watts on the workloads tested[1].
These are still OpenAI's own measurements, published by OpenAI. Using an open benchmark helps, but the specific products behind the phrase "leading commercially available AI systems" are not named, so the figures deserve that caveat.
Chip, memory, and network designed as one
Jalapeño started from a single question: what hardware would you build if its primary job were serving modern and future language models, especially interactive agents[1]?
Language-model inference moves through two phases with different bottlenecks. Prefill, when the system processes a prompt, is compute-intensive. Decode, when the system generates the response token by token, is constrained more by memory bandwidth. On top of that, every hop of data between cores and chips adds latency, leaving processing units idle while they wait. A system that excels at only one phase tends to lose that advantage waiting for data.
Jalapeño is therefore built to minimize data movement and communication delay. Model state, including the KV cache used while generating a response, can be explicitly placed and kept local, while the system activates the right combination of compute, memory, and networking for each inference phase. The network is treated as part of the architecture: its large domain keeps the entire workload inside one connected system, which helps the complete request stay fast and efficient from beginning to end[1].
AI shows up in both the design and the programming
The development timeline is unusually short. Jalapeño went from initial design to tapeout in nine months, with OpenAI models used to explore implementations and shorten design, measurement, and verification loops. AI also helped optimize the chip's arithmetic circuits, letting the team fit more compute performance into the chip on schedule[1].
The chip was also designed as a clear, predictable programming target for humans and AI alike. Engineers describe work through local tensors, explicit communication, and predictable synchronization, and AI then optimizes how that work is mapped, placed, scheduled, and coordinated across the system.
As an example, the team used Codex with GPT-Astra to bring three open-weight models that were not part of the original production plan to high performance within two months. For selected GPT-OSS attention and mixture-of-experts blocks, which route work to different specialist sub-models, AI-generated implementations ran 1.5 to 1.8 times faster than existing human-expert-written implementations[1]. Those figures apply to the selected blocks, not to full models.
Deployment this year, with NVIDIA hardware still in the mix
Jalapeño was announced on June 24, 2026 as a joint development with Broadcom. The design went from initial concept to manufacturing tapeout in nine months, with Celestica contributing board, rack, and system expertise[2].
OpenAI plans to begin deploying Jalapeño within its own compute infrastructure by the end of the year. The roadmap assumes multiple generations: Gen 2 is deep in development and Gen 3 is taking shape[1]. For now, the work continues on production qualification, software maturity, preparation for operating at scale, and performance validation across more models.
This is not a wholesale move away from third-party silicon. OpenAI states that it will continue to widely deploy accelerators from NVIDIA and other partners for both training and inference[1]. Meeting growing demand simply requires more compute from every available source.
Summary
OpenAI has published the first measurements for its own inference chip, Jalapeño, showing 1.5 to 1.9 times better performance per watt and 1.7 to 3.6 times lower end-to-end latency on the public InferenceX benchmark. The gains are attributed to designing the chip, memory, network, and rack-scale system together around language-model inference, where prefill and decode stress different resources. Deployment inside OpenAI's infrastructure begins this year, though the numbers are the company's own and the comparison systems are unnamed. How the later generations land, and whether anyone reproduces the results independently, is what to watch next.
Source[1]: https://openai.com/index/jalapeno-first-results
Source[2]: https://openai.com/index/openai-broadcom-jalapeno-inference-chip/
