OpenAI Claims Custom 'Jalapeño' Chip Outperforms Nvidia GB300 in Inference Efficiency

For years, the artificial intelligence industry has operated under a simple assumption: Nvidia's hardware is the undisputed king of both training and running large language models. OpenAI, the creator of ChatGPT, is one of Nvidia's largest customers, relying on the chipmaker's infrastructure to power its rapidly expanding data centers. Yet at the Hot Chips conference at Stanford University on Tuesday, OpenAI presented a direct challenge to that dominance. The company unveiled benchmark results for "Jalapeño," its first custom-built AI processor, claiming the new chip significantly outperforms Nvidia's current-generation GB300 systems in both speed and power efficiency.[1][2]

The announcement highlights a growing tension in the AI sector: while companies like OpenAI depend heavily on Nvidia, the massive electricity costs of running AI models are forcing them to develop their own specialized hardware. However, the move is not a complete break from Nvidia. OpenAI's hardware chief, Richard Ho, clarified that the company will continue to purchase Nvidia GPUs for training its models, a computationally intensive process where Nvidia remains unchallenged. Jalapeño, instead, is designed exclusively for inference—the stage where a trained model actually processes user requests and generates responses.[2][7]

Inference is where the bulk of long-term artificial intelligence costs accumulate. Every time a user asks ChatGPT a question or requests a summary, the model must retrieve data from memory and compute an answer, consuming significant power in the process. OpenAI developed Jalapeño in partnership with Broadcom to target this specific operational bottleneck. According to the company's published results on the public InferenceX benchmark, Jalapeño delivered 1.5 to 1.9 times more AI work per watt at peak throughput compared to Nvidia's GB200 and GB300 systems.[1][5]

The efficiency gains are rooted in the chip's architecture. Jalapeño is built on TSMC's N3P process and utilizes HBM4 memory, offering 15.4 terabytes per second of bandwidth. By keeping the model's state close to the processor and minimizing data movement, the chip reduces the power draw typically associated with transformer-based inference. While Jalapeño is rated at 700 watts, OpenAI reported that its sustained power consumption remained at or below 550 watts during the tested workloads, compared to Nvidia systems rated at 1,200 to 1,400 watts.[1][2][4]

Speed is the other critical factor in evaluating inference hardware, particularly for applications that require real-time user interaction. For highly interactive workloads that demand extensive back-and-forth communication, OpenAI reported that Jalapeño achieved 2.1 to 4.1 times higher performance than the Nvidia comparison systems. End-to-end latency was reduced by 1.7 to 3.6 times across the three large language models tested in the benchmark suite: OpenAI's own GPT-OSS 120B, DeepSeek R1 670B, and Moonshot AI's massive 1-trillion-parameter Kimi K2.5 model.[1][5][6]

The development cycle for the Jalapeño processor was unusually rapid for custom silicon. OpenAI stated that the engineering team moved from initial design to manufacturing tape-out in just nine months. To achieve this accelerated timeline, the company utilized its own artificial intelligence models, including Codex, to write kernel code and streamline the design and verification processes. This full-stack approach—designing the models, the serving software, and the physical silicon in tandem—allows OpenAI to optimize every layer of its infrastructure based on the exact real-world workloads its data centers handle daily.[1][5][6]

Despite the impressive benchmark numbers, industry analysts note several important caveats. Jalapeño was tested against Nvidia's GB200 and GB300 systems, but it was not compared to Nvidia's newer Vera Rubin architecture, which also utilizes HBM4 memory and is currently beginning to ship to customers. Furthermore, OpenAI normalized the benchmark results using the published power ratings of the accelerators, rather than the actual power drawn by the Nvidia chips during the tests, which could affect the exact efficiency margins.[1][2][3]

Looking ahead, OpenAI plans to begin deploying the Jalapeño processor in small volumes within its own data centers by the end of 2026, with a broader production ramp scheduled throughout 2027. A second-generation version of the chip is already in fabrication, promising an additional 25 percent improvement in performance per watt. Up to 2,048 Jalapeño chips can be connected across a 16-rack scale-up domain, supporting OpenAI's ambitious goal of building massive, gigawatt-scale data centers that can handle the next generation of artificial intelligence demand.[4][7]

The introduction of the Jalapeño processor signals a broader structural shift in the artificial intelligence infrastructure market. As the economics of inference become central to long-term profitability, major AI laboratories are increasingly seeking autonomy over their hardware stacks to control costs. While Nvidia's data center revenue continues to soar and its training hardware remains indispensable, the success of custom silicon like Jalapeño suggests that the future of AI inference may be driven by highly specialized, in-house processors designed to make computing power more abundant and affordable.[6][7]