OpenAI benchmarks its Jalapeño chip at 1.9x more AI work per watt

OpenAI published the first benchmark results for Jalapeño, its custom inference chip, on Tuesday. The numbers land ahead of the Nvidia systems it was measured against. Across three open weight models, the company reports 1.5 to 1.9 times more AI work per watt at peak throughput, plus 1.7 to 3.6 times lower end to end latency. For highly interactive workloads, it claims 2.1 to 4.1 times higher performance.

Those figures come from InferenceX, a public benchmark from SemiAnalysis. It measures the full process of serving an AI request, and it publishes every data point through a public GitHub Actions run. OpenAI compared Jalapeño against Nvidia’s GB200 on GPT-OSS 120B, and against the GB300 on DeepSeek R1 670B and Kimi K2.5 1T.

The results were normalised by each accelerator’s published chip power rating, which is where much of the gap comes from. Jalapeño is rated at 700 watts, against 1,200 for the GB200 and 1,400 for the GB300. OpenAI says measured sustained power stayed at or below 550 watts on the workloads tested. The company is explicit about preferring that frame: “Although performance is sometimes reported per chip, we believe the more useful standard is performance per unit of power.”

Here are the headline figures from the results appendix, at a nominal 8k input and 1k output.

Model	Compared with	Peak mixed TPS per kW	End to end latency
GPT-OSS 120B	GB200	85,448 vs 44,960	1.03s vs 1.80s
DeepSeek R1 670B	GB300	19,641 vs 11,781	1.65s vs 5.99s
Kimi K2.5 1T	GB300	18,195 vs 11,862	1.56s vs 5.31s

The bottom line is that the results show a very, very significant performance advance over state of the art.

Richard Ho, OpenAI’s head of hardware, via TechCrunch

In a briefing with reporters, Ho said Jalapeño offers the “best of both worlds”, according to The Verge, which lists him as OpenAI’s hardware vice president. That framing matters because inference systems normally trade latency against throughput. The mechanism OpenAI describes for dodging that trade is a full stack one.

Language model inference splits into a compute heavy prefill phase and a memory bandwidth bound decode phase. Communication between cores and chips adds delay on top, leaving processing units idle while they wait. Jalapeño’s answer is to keep model state local, including the KV cache used while generating a response, then activate the right mix of compute, memory and networking per phase.

The chip itself is an ASIC built with Broadcom, and OpenAI says its own models helped design it. That took the team from initial design to tapeout in nine months. On selected GPT-OSS attention and mixture of experts blocks, AI generated implementations ran 1.5 to 1.8 times faster than the human expert versions. OpenAI flags that those figures cover the selected blocks, not the full model.

The catch is timing, and it’s the part the benchmark can’t settle. Ho told reporters Jalapeño would deploy at the end of 2026 “in very small volumes,” with more significant deployment in 2027, and OpenAI hasn’t said how many chips that means. He also told The Verge that OpenAI doesn’t expect to replace its entire chip lineup with Jalapeño.

The blog post says the company plans to keep widely deploying accelerators from Nvidia and other partners. That fits the scale of its existing commitments, including the $105bn of leases Nvidia guaranteed at an Ohio data centre.

The benchmark that made Jalapeño look good is already running it against a newer Nvidia system.

Rundowns AI

TechCrunch made the sharper point. By the time Jalapeño reaches full deployment, the competition may have moved. SemiAnalysis is already surfacing a Jalapeño comparison against Nvidia’s Vera Rubin NVL72 on the InferenceX front page, so these GB200 and GB300 baselines won’t stay current for long. Even so, custom silicon aimed at the cost of serving is a familiar move, and it’s the same logic behind Microsoft building its own models to cut inference costs.

Which means the thing to watch is whether the second generation lands before the first one ships in volume. OpenAI says Gen 2 is deep in development and Gen 3 is taking shape. Production qualification, software maturation and validation across more models are all still in progress. Until outside parties run Jalapeño themselves, every number above is OpenAI’s own measurement on someone else’s harness.