Nvidia pursues $30B custom chip opportunity with new unit
Nvidia’s move into a $30B market for custom AI and compute chips prompts debate over whether buyers truly need bespoke silicon or simply cheaper, high‑VRAM GPUs for workloads like large language models. Commenters highlight Nvidia’s dominant software ecosystem, aggressive product segmentation, and pricing power, contrasting it with AMD’s ROCm efforts and a wave of startups and proposals such as OpenAI’s reported multi‑trillion‑dollar chip ambitions. Many see custom chips as a logical extension of Nvidia’s platform strategy, while questioning whether meaningful gains over general‑purpose GPUs justify the enormous capital being proposed for new fabs and architectures.
Consumer GPUs, VRAM, and AI Workloads
- Several commenters want consumer GPUs with very large VRAM (e.g., 128GB) to run bigger open‑source LLMs locally.
- Others note this is far beyond what graphics workloads need; such specs effectively turn a card into a compute accelerator.
- Discussion argues Nvidia keeps consumer VRAM low to avoid cannibalizing high‑margin datacenter cards, with big jumps in price between 24GB and 40GB models attributed to artificial segmentation and licensing.
- Physical constraints are noted: with current GDDR capacities and bus widths, ~48GB is about the max on a conventional consumer GPU without HBM, which is expensive and packaging‑limited.
Alternate Memory and Compute Approaches
- Some advocate investment in in‑memory compute / ReRAM to reduce AI energy and climate impact.
- Others dismiss this as repeatedly tried and failed, and not comparable to “just add more VRAM”.
- A middle view points out GPUs and package‑on‑package strategies already move toward higher bandwidth “near‑memory” compute.
Software Efficiency vs Hardware Abundance
- Debate over whether large resources (VRAM, RAM) encourage wasteful coding or enable productivity and new kinds of software.
- Examples range from highly optimized legacy code to modern Electron apps and slow everyday tools.
- Some argue easing “bookkeeping” is good; others say performance has regressed faster than hardware has improved.
Nvidia, CUDA, and Competitors
- Nvidia is seen as having a strong, coherent hardware–software stack (CUDA, PTX, drivers), and a culture that prioritizes working software.
- AMD’s ROCm is viewed as technically catching up for pure compute, but with poor support, limited official GPU coverage, and instability when mixing graphics and compute.
- This maturity makes Nvidia a natural player for custom AI chips versus newer startups.
Custom / Bespoke AI Chips and $7T Debate
- Custom chips are framed as a way to strip unneeded GPU features or add domain‑specific blocks (e.g., telecom instructions).
- Some argue many customers really just want cheaper, cut‑down GPUs, not true customization.
- The reported $7T AI‑chip funding figure tied to another firm is widely seen as unrealistic or purely headline‑driven; commenters note it exceeds historic projects like WWII (in inflation‑adjusted terms) and all of TSMC’s lifetime revenue.
Cloud and Pricing Dynamics
- Major clouds’ GPU pricing (AWS/Azure) is criticized as high; other providers and specialized hardware (TPUs, Gaudi) are mentioned but often priced steeply too.
- Nvidia’s GeForce Now is cited as surprisingly good value for gaming, raising questions about its long‑term economics.
- Some expect more “Hetzner‑like” cheaper GPU training providers in 2024–2025, but note hyperscalers can subsidize their own training costs.
Nvidia Strategy, Fabs, and Partnerships
- Nvidia is fabless and seen as node‑portable, using TSMC, Samsung, and potentially Intel depending on cost and product.
- A new custom‑chip unit fits a broader pivot toward IP licensing and platform dominance (e.g., prior deals with console makers and SoC vendors).
- There are mixed views on Nvidia as a partner: technically successful in consoles and datacenters, but perceived as hard to work with and highly protective of driver and software IP.