OpenAI's Jalapeño chip claims big efficiency and latency wins over Nvidia's GB300 in first benchmarks

OpenAI unveiled benchmark results at Hot Chips showing its custom Jalapeño inference ASIC, co-developed with Broadcom, delivers 1.5 to 1.9 times more throughput per kilowatt and 1.7 to 3.6 times lower latency than Nvidia's GB200 and GB300 rack systems on the InferenceX suite. The 700W chip outperformed Nvidia parts rated at 1,200W and 1,400W, with the widest margins at low-latency settings, though OpenAI noted the comparison used package TDP and that utility-power and multi-token prediction scenarios narrow the gap. Jalapeño is slated for deployment in OpenAI's data centers later this year, but it was not tested against Nvidia's upcoming Vera Rubin and does not handle model training.