2023 was the year that GPUs stood still

Original Article ↗ Hacker News Discussion ↗

GPU buyers in 2023 are caught between rising performance demands from AI and gaming and stagnating consumer hardware, especially around VRAM capacity and price. Commenters argue that Nvidia’s dominance and aggressive product segmentation, mirrored by AMD to a lesser extent, have kept large‑memory cards expensive while mid‑range and low‑end options lose value, pushing some users toward alternatives like Apple Silicon or dedicated AI boxes. There is broad agreement that software ecosystems (CUDA vs. ROCm vs. Intel’s stack) and artificial constraints matter as much as raw silicon, and that future gains may come as much from smarter software and model optimization as from bigger, faster GPUs.

GPU Memory Needs and Local AI

  • Many participants argue desktop GPUs need far more VRAM (48–96 GB) to run large (70B+) models and local multimodal “personal AI” workloads.
  • Current consumer caps (24 GB, rare 48 GB prosumer) are seen as an artificial segmentation to protect high-margin enterprise GPUs.
  • Others suggest model optimization, quantization, and better software tooling partly offset limited VRAM, but concede big models remain memory-bound.

Apple Silicon and Unified Memory

  • Apple’s M-series with 80–192 GB shared memory is praised as uniquely capable for big local models, low power use, and portability.
  • Drawbacks: high price, weaker gaming and raw GPU bandwidth vs Nvidia, spotty support in many popular AI tools, and closed GPU ecosystem.
  • Some think UMA was a forward-looking design for post-graphics GPU workloads; others say it’s just a big shared DRAM pool.

CUDA, ROCm, and Ecosystem Lock-In

  • Strong consensus that Nvidia’s CUDA ecosystem is the dominant moat.
  • Many AI frameworks and toolchains embed hand-written CUDA kernels, making AMD/Intel support fragile or slower even when PyTorch “runs.”
  • AMD is criticized for years of poor ROCm support, limited officially supported SKUs, and inconsistent tooling; recent MI300/ROCm pushes are seen as late but promising.
  • Intel is viewed as improving fast with Arc drivers; some see its continued GPU investment as strategically unavoidable despite losses.

Pricing, Segmentation, and Midrange GPUs

  • Ongoing argument whether midrange GPUs (e.g., RTX 4060/4070, RX 7600/7800XT) are still good value.
  • One camp: performance-per-dollar at $250–600 is reasonable given wafer costs; “doom and gloom” is emotional and driven by nostalgia.
  • Other camp: capacities, die sizes, and features at those price points feel “lower tier” than in past generations; the real “mainstream” has crept to $500–700.
  • There’s frustration that larger-memory SKUs exist as workstation cards at huge markups.

Future of Consumer GPUs and AI Demand

  • Some predict high-end hardware will keep scaling for hyperscalers while consumer GPUs stagnate or shift to integrated APUs, with discrete cards becoming niche.
  • Debate over whether the AI boom sustains for 5–10 years or is partially a bubble; most think AI workloads won’t disappear even with copyright regulation.
  • Console VRAM (~12 GB usable) is seen as a ceiling for AAA game assets; extra PC VRAM may instead be used for AI upscaling and richer simulations.