Llm.c – LLM training in simple, pure C/CUDA

Original Article ↗ Hacker News Discussion ↗

Project scope and intent

  • Repo is a ~1000-line C implementation of transformer training, currently CPU/OpenMP-based; CUDA is planned but not yet present.
  • Many see it as a minimal, didactic reference showing that LLM training internals are conceptually simple compared to large frameworks.
  • Others note that while the core algorithm fits in little code, production systems need far more functionality and engineering.

Python, overhead, and frameworks

  • Debate over Python’s performance: core tensor ops are in optimized low-level code, so runtime/memory overhead is often modest.
  • Counterpoints: hot Python loops and suboptimal data structures can dominate runtime; “using the wrong primitive” can yield huge slowdowns.
  • Some argue Python’s container variety invites performance traps; others say that’s a programmer, not language, problem.
  • Larger criticism targets ecosystem complexity and dependency bloat (PyTorch, CPython) versus small, focused C code.

CUDA, GPU stacks, and alternatives

  • CUDA SDK and associated libraries are seen as huge and complex (multi‑GB installs, very large shared objects).
  • Some emphasize that CUDA size is also a burden for Python stacks; GPU complexity is largely independent of language.
  • CUDA remains dominant in speed and tooling; alternatives (AMD ROCm, TPUs, Vulkan-based projects, Rust frameworks, etc.) exist but are less mature or harder to use.
  • Several comments describe frustrating experiences with AMD drivers and ecosystem stability.

Hardware and VRAM discussions

  • Desire for expandable GPU memory (e.g., SO‑DIMMs) to reduce VRAM constraints for LLMs.
  • Others explain technical and economic reasons against modular GPU memory: latency, bandwidth, power, cooling, and product segmentation.

Educational value and resources

  • Many view the project as an excellent learning tool that demystifies LLM training.
  • Multiple replies point to associated tutorials and video series that build similar models step by step in higher-level languages, then in C.

Transformers and other domains

  • Discussion notes that transformers are generic “array in, array out” modules; with appropriate tokenization they can handle time series, images, and audio.
  • References are given to time-series transformer work, alongside skepticism that these always beat strong classical baselines like gradient boosting.