OpenAI and Broadcom Unveil 'Jalapeño' Custom AI Chip to Slash Inference Costs

On Wednesday, OpenAI and semiconductor giant Broadcom officially unveiled "Jalapeño," a custom-built artificial intelligence processor designed to fundamentally alter the economics of running large language models. The highly anticipated silicon represents OpenAI’s first major foray into proprietary hardware, marking a strategic pivot from relying entirely on off-the-shelf graphics processing units (GPUs) to designing its own vertical stack. Early engineering samples of the chip are already operating in OpenAI’s labs, running advanced machine learning workloads like the GPT-5.3-Codex-Spark model at target power and performance levels.[1][2]

The primary objective of the Jalapeño architecture is dramatic cost reduction. According to Broadcom CEO Hock Tan, early testing indicates that the new accelerator delivers cost savings of roughly 50 percent compared to the typical AI GPUs currently dominating the market. For a company that processes billions of queries daily across ChatGPT, its API, and enterprise deployments, halving the cost of compute could translate to billions of dollars in saved operational expenditures, while simultaneously allowing the company to lower prices for developers.[1][4]

To understand Jalapeño’s significance, it is crucial to distinguish between the two phases of artificial intelligence compute: training and inference. Training is the computationally massive, months-long process of teaching a model how to understand language and patterns, a domain still overwhelmingly ruled by Nvidia’s versatile GPUs. Inference, however, is the everyday execution phase—the split-second process of the finished model generating a response to a user’s prompt. Jalapeño is an Application-Specific Integrated Circuit (ASIC) purpose-built exclusively for this inference phase, trading the broad flexibility of a GPU for hyper-optimized efficiency in a single task.[1][2][5]

Richard Ho, the head of OpenAI’s hardware program, emphasized that Jalapeño was architected from the ground up to address the specific bottlenecks that plague large language model inference at scale. Traditional hardware often struggles with the costly movement of data between compute cores and memory banks. The new custom chip mitigates this by balancing compute, high-bandwidth memory (HBM), and networking resources to minimize data travel, thereby maximizing throughput while maintaining the low latency required for real-time conversational agents and autonomous AI workflows.[3][5]

Perhaps the most remarkable aspect of the Jalapeño project is its blistering development timeline. In the semiconductor industry, the cycle from initial schematics to a manufacturing-ready "tape-out" typically spans several years. OpenAI and Broadcom managed to compress this entire process into a mere nine months. This unprecedented speed was achieved through a deep software-hardware co-development strategy, but it also relied on a novel catalyst: artificial intelligence itself.[2][6]

OpenAI engineers actively utilized their own prior-generation AI models to accelerate complex parts of the chip’s design and optimization process. By deploying AI to assist in routing billions of microscopic transistors and simulating hardware performance, the team bypassed traditional engineering bottlenecks. This creates a powerful feedback loop: the very models served to users are now helping to design the physical infrastructure required to run the next generation of models even faster and more efficiently.[2][6]

The physical realization of the Jalapeño chip relies on a global coalition of hardware partners. The silicon itself is being fabricated by Taiwan Semiconductor Manufacturing Company (TSMC), the world’s leading foundry for advanced nodes. Once manufactured, Canadian electronics manufacturing services company Celestica will integrate the chips into specialized server boards and rack systems. Finally, the completed server racks are slated for initial deployment by the end of 2026 within massive data centers operated by Microsoft and other strategic infrastructure partners.[1][6]

For enterprise customers, the arrival of custom inference silicon cannot come soon enough. As businesses integrate generative AI deeper into their daily operations, they are encountering what industry analysts call "explosive cost switches" associated with tokenomics. Every word generated by an AI model consumes a fraction of a cent in compute power, known as a token. At enterprise scale, these token consumption costs are compounding rapidly, creating a financial barrier to deploying autonomous AI agents that require thousands of background reasoning steps.[3]

By drastically reducing the cost of inference, Jalapeño fundamentally changes the economics of converting electrical watts into AI tokens. This efficiency gain provides OpenAI with significant margin expansion. More importantly, if the enterprise AI market descends into a price war—a likely scenario given the aggressive moves by rivals like Anthropic and Google—OpenAI will have the financial bandwidth to slash its API pricing without destroying its own path to profitability.[3]

The scale of the planned deployment underscores the massive capital requirements of the generative AI era. Broadcom and OpenAI are targeting the rollout of gigawatt-scale data centers—facilities that consume as much electricity as a mid-sized city. Broadcom CEO Hock Tan recently noted that demand from OpenAI and other hyperscalers is so intense that his previous projection of deploying 1.3 gigawatts worth of custom AI chips next year may prove to be overly conservative.[1][4]

Despite the launch of Jalapeño, OpenAI is not abandoning its relationship with Nvidia. The startup recently participated in a massive funding round that secured 10 gigawatts of computing systems from Nvidia, heavily utilizing the next-generation Vera Rubin platform for future model training. However, by offloading the high-volume, lower-margin inference workloads to its own custom Broadcom silicon, OpenAI is diversifying its supply chain and reducing its vulnerability to the pricing power of a single dominant hardware vendor.[2][4]

Jalapeño represents merely the opening salvo in a long-term hardware strategy. OpenAI and Broadcom have explicitly framed this release as the first step in a multi-generation compute platform. The companies have already established a roadmap for future chip iterations, with the next major architectural leap planned for 2028, followed by an annual release cadence. As AI models evolve from text generators into autonomous digital workers, the underlying silicon will continue to co-evolve, ensuring that the physical infrastructure can support the software's expanding ambitions.[1][6]

Ultimately, the push for custom silicon is about democratizing access to frontier intelligence. By controlling the full stack—from the foundational model weights down to the physical routing of data on the silicon die—OpenAI aims to make advanced AI more abundant and reliable. Whether it manifests as a faster response in a consumer app, a cheaper API call for a startup, or a more capable reasoning agent for medical researchers, driving down the cost of compute is the prerequisite for integrating artificial intelligence into the fabric of the global economy.[6]