OpenAI's Jalapeño: The Chip That Ends the GPU Hype?

OpenAI's Jalapeño chip, built with Broadcom, hit tape-out in 9 months — and frontier labs now treat custom inference silicon as table stakes.

What Jalapeño actually is

OpenAI's first custom AI chip, codenamed Jalapeño, was built in partnership with Broadcom and is targeted at LLM inference, not training. Greg Brockman pushed performance per watt as the headline metric — at frontier scale, power is the bottleneck, and better perf/W means more tokens per dollar and fewer agents waiting on merchant GPU allocations. Use cases named explicitly: ChatGPT, Codex, API traffic, and future agent products.

The numbers (community reverse-engineered, unofficial)

Reverse-engineering estimates put Jalapeño near a reticle die with roughly 260 GB of HBM3E, 7.1–7.4 TB/s bandwidth, and around 10 PFLOPS in FP4. The design-to-tape-out cycle was 9 months, unusually fast for a high-performance ASIC, and OpenAI's own models reportedly accelerated the timeline — AI designing the silicon that runs AI. The chip looks TPU-like in community teardowns.

Same-day moves across the stack

Chris Lattner announced Qualcomm is acquiring Modular, with Mojo's open-sourcing still on track. Nvidia pushed NeMo Automodel, promising 3.4–3.7× higher training throughput on MoE models via expert parallelism, DeepEP, and Transformer Engine kernels. SkyPilot launched unified inference endpoints across clusters. Modular claimed open-source inference stacks can beat proprietary providers on latency.

Local optimization caveat

John Durbin reported 30–50% real-world decode gains from custom deep flash draft and speculator models. Powerful, but expensive enough that full local-agent inference is 5–10 years out.

Where this leaves you

The AI race is shifting from best model to best factory. Anthropic, Alibaba, MiniMax, and others will be forced to answer. If you build agents, decide now where inference will live in two years — the answer is about to look very different.

Sections referenced

• OpenAI, Broadcom, Jalapeño
• Chris Lattner, Qualcomm, Modular, Mojo
• Nvidia, NeMo Automodel, DeepEP, SkyPilot
• John Durbin, deep flash, draft, speculator