OpenAI showed the first benchmarks for its in-house inference chip at the Hot Chips conference. According to the reported results, the chip, called Jalapeño, outperforms Nvidia's Blackwell and Rubin platforms on both throughput per watt and token latency.

The chip's scope is limited and worth stating plainly: it handles inference only, meaning it runs models but does not train them. It is also not tuned to OpenAI's own models — it is a general-purpose LLM inference accelerator.

The reported numbers

OpenAI's published results are these:

  • Work per watt — 1.5 to 1.9 times more at peak throughput across the three models tested.
  • End-to-end latency — 1.7 to 3.6 times lower than the best commercially available systems.
  • Interactive workloads — 2.1 to 4.1 times higher performance.

The results come from SemiAnalysis's public InferenceX benchmark. OpenAI provided the numbers; SemiAnalysis verified some runs on-site in the lab. The models tested were GPT-OSS 120B, Deepseek R1 670B and Kimi K2.5 1T.

Richard Ho, OpenAI's head of hardware, said in a press call: "The bottom line is that the results show a very, very significant performance advance over state of the art. Jalapeño can serve more AI work per unit of power, while also returning responses more quickly."

What is being compared matters

SemiAnalysis points out that the fairer comparison is not Blackwell but Nvidia's newer Vera Rubin platform, since both use HBM4 memory. Even there, Jalapeño squeezes out more output tokens per megawatt — and does so without adopting optimizations like multi-token prediction that Nvidia's accelerator uses. On total cost of ownership per token, the two come out roughly even.

SemiAnalysis CEO Dylan Patel summarized his assessment: "Usually first generation chips aren't competitive, but OpenAI is beating Nvidia Blackwell and even Rubin."

The caveats

Several points call for reading the numbers carefully. Nvidia and AMD have published results with larger models that have not been tested on Jalapeño yet. More importantly, there is timing: Rubin systems are already shipping to customers, while Jalapeño reportedly has not moved beyond engineering samples.

Ho's own timeline confirms this: the chip will deploy at the end of 2026 "in very small volumes," with more significant deployment in 2027. So the benchmark was run against today's Nvidia system, but by the time Jalapeño reaches full deployment the competition may have advanced considerably.

Designed in nine months

Jalapeño was developed with Broadcom. Design work began in mid-2024 and the final design went to fabrication in November 2025. The full cycle took about sixteen months, but OpenAI says only nine months passed between the first chip design and the finished blueprint heading to the factory.

The company says it used its own models during development: older model generations helped with chip design, while newer ones sped up programming and optimization.

Why it matters

SemiAnalysis reads this as a sign that Nvidia's much-discussed "CUDA moat" may no longer hold: "The CUDA moat is potentially dead given how fast OpenAI can bring up new models on their silicon."

OpenAI CFO Sarah Friar presents the chip as part of a broader compute strategy in which data centers, chips, models, the developer platform, products and devices all work as one integrated system. Friar says Jalapeño complements rather than replaces existing partnerships with Nvidia, AMD, AWS, Cerebras and CoreWeave.