OpenAI unveils its first custom chip, built by Broadcom

Original Article ↗ Hacker News Discussion ↗

OpenAI has announced “Jalapeño,” its first custom AI inference chip co-designed with Broadcom and manufactured by TSMC, aiming for substantially better performance-per-watt and roughly 50% lower costs than typical GPU-based deployments. Commenters see this as part of a broader shift in which major AI labs and cloud providers build their own accelerators to escape Nvidia’s pricing and power constraints, though many note the lack of concrete specs and question how much is Broadcom IP versus genuine OpenAI hardware innovation. The thread also touches on AI-assisted chip design, impacts on competitors like Cerebras and Google’s TPU efforts, and whether highly specialized hardware—up to and including models baked directly into silicon—can pay off before the next generation of models arrives.

Chip purpose and positioning

  • Jalapeño is framed as an inference‑only accelerator, not for training.
  • Goal is substantially better performance‑per‑watt and ~50% lower cost vs “typical” AI GPUs, with first deployment targeted for end of 2026.
  • Many commenters note inference now dominates total spend; optimizing ongoing token costs is seen as more strategic than training speed.

Broadcom, TSMC and industry structure

  • Chips are fabbed at TSMC and co‑designed with Broadcom, which already partners with Google on TPUs and with other hyperscalers on custom ASICs.
  • Broadcom is described as a giant in ASIC and interconnect IP, with strong allocation agreements at foundries and memory vendors.
  • Some warn OpenAI that Broadcom is an aggressive, customer‑unfriendly operator (citing VMware/CA/Symantec experiences).

Impact on Nvidia and other accelerators

  • Many see this as part of a broader move: Google TPUs, Amazon/Anthropic Trainium, Microsoft Maia, Meta/Broadcom ASICs.
  • Consensus: Nvidia likely remains dominant for general‑purpose training, but loses share on hyperscaler inference.
  • Debate on how much this threatens Cerebras: some think OpenAI’s own chip will crowd them out; others note Cerebras targets niche/training and already has an OpenAI partnership.

Technical debates

  • Strong focus on memory bandwidth and architecture: unified SoCs (Apple), LPDDR vs HBM, and tradeoffs between prefill latency vs raw bandwidth.
  • Clarification that the PR photo shows a wafer with 50–60 dies, not a wafer‑scale engine like Cerebras.
  • Extended side discussion on “baking weights into silicon” (e.g. Taalas): huge speed/efficiency potential but inflexible and KV‑cache‑limited; seen as better for stable, smaller models or edge/robotics use.

AI‑assisted chip design claim

  • Press language says OpenAI models “accelerated” design and optimization.
  • Some think this just means engineers used LLMs for HDL, testbenches, scripts, email, etc.
  • Others note OpenAI hiring for AI‑for‑chip‑design, but several remain skeptical nine months is enough for a from‑scratch, AI‑designed 3nm chip.

Business strategy, risk, and moats

  • Viewed as necessary to cut OpenAI’s massive inference costs and dependence on Nvidia, and to justify a future IPO.
  • Skepticism about vague metrics (“substantially better,” “typical GPUs”) and lack of concrete specs.
  • Some argue Nvidia’s true moat is software; for a single internal model family, that matters less.
  • Concerns raised about concentration of ultra‑fast, custom hardware inside a few labs, limiting smaller companies to slower, metered APIs.

Cultural and miscellaneous reactions

  • The “Jalapeño” name provokes mixed reactions (cringe, regional cliché, or just another arbitrary codename).
  • A few blame massive RAM purchases and custom chips for high memory prices.
  • Some see this as inevitable “if you care about software, build hardware”; others as scope creep and pre‑IPO hype.