OpenAI's Custom Chip Beats Nvidia in Key Efficiency Measures

OpenAI's custom chip, Jalapeño, has been unveiled as a major competitor to Nvidia's top-end Blackwell accelerators. The 700-watt inference chip is designed specifically for running AI models and outperforms Nvidia's GB300 in key efficiency and latency measures.

In tests conducted by SemiAnalysis, Jalapeño delivered 1.5 times to 1.9 times more throughput per kilowatt and 1.7 times to 3.6 times lower end-to-end latency than the GB200 and GB300 rack systems. OpenAI's chip was tested against Nvidia's accelerators at a lower wattage, with Jalapeño using only 550 watts of power.

The results are significant because they show that OpenAI's custom chip can perform as well as or better than Nvidia's hardware in certain scenarios. However, it's worth noting that the chip does not train models, and its performance against other Nvidia platforms like Vera Rubin is still unknown.