OpenAI unveils its first custom chip, built by Broadcom

Original Article ↗ Hacker News Discussion ↗

Chip purpose and positioning

  • Jalapeño is framed as an inference‑only accelerator, not for training.
  • Goal is substantially better performance‑per‑watt and ~50% lower cost vs “typical” AI GPUs, with first deployment targeted for end of 2026.
  • Many commenters note inference now dominates total spend; optimizing ongoing token costs is seen as more strategic than training speed.

Broadcom, TSMC and industry structure

  • Chips are fabbed at TSMC and co‑designed with Broadcom, which already partners with Google on TPUs and with other hyperscalers on custom ASICs.
  • Broadcom is described as a giant in ASIC and interconnect IP, with strong allocation agreements at foundries and memory vendors.
  • Some warn OpenAI that Broadcom is an aggressive, customer‑unfriendly operator (citing VMware/CA/Symantec experiences).

Impact on Nvidia and other accelerators

  • Many see this as part of a broader move: Google TPUs, Amazon/Anthropic Trainium, Microsoft Maia, Meta/Broadcom ASICs.
  • Consensus: Nvidia likely remains dominant for general‑purpose training, but loses share on hyperscaler inference.
  • Debate on how much this threatens Cerebras: some think OpenAI’s own chip will crowd them out; others note Cerebras targets niche/training and already has an OpenAI partnership.

Technical debates

  • Strong focus on memory bandwidth and architecture: unified SoCs (Apple), LPDDR vs HBM, and tradeoffs between prefill latency vs raw bandwidth.
  • Clarification that the PR photo shows a wafer with 50–60 dies, not a wafer‑scale engine like Cerebras.
  • Extended side discussion on “baking weights into silicon” (e.g. Taalas): huge speed/efficiency potential but inflexible and KV‑cache‑limited; seen as better for stable, smaller models or edge/robotics use.

AI‑assisted chip design claim

  • Press language says OpenAI models “accelerated” design and optimization.
  • Some think this just means engineers used LLMs for HDL, testbenches, scripts, email, etc.
  • Others note OpenAI hiring for AI‑for‑chip‑design, but several remain skeptical nine months is enough for a from‑scratch, AI‑designed 3nm chip.

Business strategy, risk, and moats

  • Viewed as necessary to cut OpenAI’s massive inference costs and dependence on Nvidia, and to justify a future IPO.
  • Skepticism about vague metrics (“substantially better,” “typical GPUs”) and lack of concrete specs.
  • Some argue Nvidia’s true moat is software; for a single internal model family, that matters less.
  • Concerns raised about concentration of ultra‑fast, custom hardware inside a few labs, limiting smaller companies to slower, metered APIs.

Cultural and miscellaneous reactions

  • The “Jalapeño” name provokes mixed reactions (cringe, regional cliché, or just another arbitrary codename).
  • A few blame massive RAM purchases and custom chips for high memory prices.
  • Some see this as inevitable “if you care about software, build hardware”; others as scope creep and pre‑IPO hype.