Five Years of Tinygrad

Original Article ↗ Hacker News Discussion ↗

Project goals & status

  • Commenters ask what tinygrad has actually achieved in five years and what it can do now.
  • Cited goals from its site: run standard ML benchmarks/papers 2× faster than PyTorch on a single NVIDIA GPU and perform well on Apple M1; ETA mentioned as next year.
  • It already powers an automotive driver-assist stack and can run NVIDIA GPUs on Apple Silicon via external enclosures.
  • Mission is framed as “commoditizing the petaflop” and enabling efficient LLM training on non‑NVIDIA hardware.

Potential impact & competition

  • Some see tinygrad as a potential alternative backend to PyTorch/TensorFlow, especially for edge and non‑CUDA hardware.
  • Others argue PyTorch could neutralize it by adding an AMD backend to its own compiler stack, leaving tinygrad’s main work (AMD codegen) as a feature PyTorch could adopt.
  • tinygrad maintainers respond that they welcome being used as a backend and already provide PyTorch and ONNX frontends.

Code size, complexity, and style

  • The low line count is polarizing: some see ~19k SLOC with zero dependencies as evidence of low incidental complexity; others complain it feels like code‑golf and is hard to read.
  • A linked optimization file becomes a focal point: critics find it dense; defenders say GPU compilation is inherently complex and the code is readable, “2D”, and appropriate for a small, expert team.
  • There’s debate over whether fewer lines actually imply simplicity; several note that autoformatters trade away information density for consistency.

Language & ecosystem comparisons

  • Discussion branches into Mojo vs CPython, Julia’s suitability as a Python successor, 1‑based indexing, multiple dispatch, metaprogramming, and trust in Julia’s correctness.
  • Some argue Mojo’s divergence from Python semantics weakens its pitch; others say Mojo’s aim is acceleration of Python‑like code on specialized hardware, not replacing CPython.

Organization, hiring & funding

  • Hiring via paid bounties and contributions is praised as highly productive and more meaningful than LeetCode interviews, but also criticized as potentially underpaying skilled work.
  • The company is small, mostly remote, with periodic meetups in Hong Kong and some physical offices.
  • Funding comes from VC, AMD contracts, and a hardware division selling multi‑GPU boxes (~$2M/year revenue); commenters debate whether this can support a team of engineers.

“Elon process”, TRIZ, and attribution

  • The blog’s reference to an “Elon process” (remove dumb requirements, “the best part is no part”) triggers pushback.
  • Several note these ideas predate that figure (e.g., TRIZ, classic design aphorisms); some dislike marketing that centers a celebrity rather than original sources.
  • There’s broader meta‑discussion about separating technical achievements from controversial public personas, and about not derailing threads into personality politics.

NVIDIA, AMD & market dynamics

  • Many see real value in helping AMD and other vendors compete with CUDA, calling this potentially worth a lot of money and technologically important.
  • Some believe open‑source software and models, plus strong inference on commodity hardware, are the realistic path to “owning” NVIDIA’s current dominance.

Hiring bounties, AI, and the future of coding

  • The bounty‑as‑interview model is contrasted with multi‑stage corporate interviews; some find it fairer, others see it as exploitative if undercompensated.
  • There’s concern that AI coding agents will flood bounties with low‑quality patches, shifting value from coding to task specification and verification.
  • One commenter speculates that as LLMs make both writing and understanding large codebases easier, huge legacy projects (LLVM, Linux, Chrome) may be harder to justify vs. focused, smaller stacks like tinygrad.

Community sentiment

  • Enthusiasts praise the openness, clear technical mission, tiny stack, and hardware/software co‑design and express strong hope that tinygrad succeeds in pushing back against “rent‑everything” compute.
  • Skeptics question the marketing emphasis (celebrity references, line counts), code ergonomics vs. PyTorch, and the founder’s public political writings, with some saying they’ll stick with mainstream frameworks for now.