Jim Keller criticizes Nvidia's CUDA, x86

Original Article ↗ Hacker News Discussion ↗

A prominent chip designer’s claim that Nvidia’s CUDA software stack and the x86 architecture are “swamps, not moats” prompts debate over whether messy, backward‑compatible platforms are a strength or a liability. Commenters contrast CUDA’s dominant ecosystem and robust libraries with AMD’s weaker ROCm support and driver issues, while also examining how vendor lock‑in, hardware availability, and AI workloads shape GPU adoption. The thread broadens into x86 vs. ARM and RISC vs. CISC design trade‑offs, with many arguing that practical compatibility and decades of tooling matter more than ISA purity.

Clarifications about Triton and context

  • Several comments note the article confused OpenAI’s Triton language with Nvidia’s Triton Inference Server; the former is a Python-like GPU programming language, the latter is a model-serving system and not a CUDA replacement.
  • It’s emphasized that the critic works at a direct Nvidia competitor, which readers argue should have been clearly foregrounded as potential bias, though others say his opinions may be what led him to compete in the first place.

CUDA: moat, “swamp,” and developer experience

  • Many argue that when people say “CUDA is a moat” they really mean the whole CUDA ecosystem (libraries like cuDNN, cuBLAS, TensorRT), not just the low-level language.
  • There’s broad agreement that most users access CUDA through higher-level frameworks (PyTorch, TensorFlow), rarely writing custom kernels; a few report doing hand-optimized CUDA/PTX and finding it powerful but difficult and time-consuming.
  • Some see the ecosystem’s complexity and backward compatibility as a “swamp” that nonetheless functions as a very effective moat.

AMD, ROCm, and non-Nvidia GPUs

  • Multiple anecdotes describe AMD’s software stack as unstable or underperforming, with drivers seen as the main problem rather than hardware capability.
  • ROCm is described as roughly analogous to CUDA+libraries, with HIP used as an abstraction layer, but support is fragmented and historically limited to specific “pro” or newer cards.
  • Several note that despite supply constraints on Nvidia GPUs, the lack of mature, painless software support keeps large users on Nvidia.

x86, ARM, and ISA debates

  • Discussion revisits x86 as a “swamp” that evolved into a kind of compressed encoding for internal micro-ops; over time, the cost of supporting old CISC features shrank relative to total transistor budgets.
  • Commenters push back on simplistic “x86 has a RISC core inside” narratives, explaining that almost all modern CPUs (including RISC-V) translate instructions into µops; “RISC vs CISC” is framed as largely obsolete.
  • ARM’s rise is debated: some highlight Apple’s M-series and AWS Graviton; others point out weaker SIMD (outside SVE), mixed real-world performance, and strong x86 ecosystem inertia.

Platform “swamps” and backward compatibility

  • Several argue that “swampiness” (complex, messy, backward-compatible ecosystems) is a common trait of successful platforms: x86, Windows, PDF, the web, Linux, even capitalism and democracy.
  • Backward compatibility is seen both as user-friendly and as a barrier to competitors, reinforcing moats even when the underlying design is imperfect.