A ChatGPT clone, in 3000 bytes of C, backed by GPT-2 (2023)

Original Article ↗ Hacker News Discussion ↗

A tiny 3,000‑byte C program that runs GPT‑2 and mimics a ChatGPT-style interface is prompting debate over what really constitutes complexity in modern AI systems. Commenters note that while the code is impressively small, almost all the “magic” lives in the hundreds of megabytes of model weights and the vast training data behind them, so the tool is more educational and artistic than practically useful. The thread broadens into questions of whether such minimal implementations demystify AI, how they relate to earlier chatbots like ELIZA, and what they imply for future AGI in terms of code size versus data and embodiment.

Project scope and implementation

  • Thread clarifies this is a tiny C program (~3000 bytes, minified) that runs inference on an existing GPT‑2 TensorFlow checkpoint, not a full ChatGPT or training setup.
  • Unobfuscated, readable C source is linked and only modestly larger; the minified form is mainly for IOCCC/code‑golf style aesthetics.
  • Most of the “magic” resides in the downloaded ~475 MB model file, not in the code itself.
  • Prior similar IOCCC work using LSTMs is referenced; this project updates the idea to transformers/GPT‑2.

Output quality and “ChatGPT clone” debate

  • Multiple people who ran it report highly repetitive and low‑quality dialog (e.g., repeating “I am a computer model trained by OpenAI”, nonsensical math like “2+2= bird”).
  • Some argue that calling it a “ChatGPT clone” is misleading since GPT‑2 is not instruction‑tuned for chat and the author themselves notes the output is objectively poor.
  • Others are impressed it is even somewhat conversational given GPT‑2’s original training and recall older GPT‑2 outputs (e.g., fairy tales) that were weird but often coherent.
  • Several compare it unfavorably to classic rule‑based chatbots like ELIZA.

Purpose and value of the tiny implementation

  • Supporters frame it as:
    • A “demake” or low‑res homage, showing the core mechanism in minimal code.
    • An educational piece that demystifies transformers and shows that inference logic is conceptually simple.
    • A kind of technical art or craftsmanship, akin to IOCCC entries or mountain‑climbing: done for challenge and joy, not utility.
  • Critics question the practical usefulness: quality is poor, model/training dominate cost, and binary size doesn’t imply performance gains.

Models, data, and “size of intelligence”

  • Discussion distinguishes:
    • Engine code vs. model weights vs. training data, using video‑game “engine/assets” analogies.
  • Several note that the core math for LLMs is small; complexity lies in huge datasets and billions of learned parameters.
  • A broader debate emerges about whether AGI could be expressed in tens of thousands of lines of code, and whether focusing only on stateless math ignores embodiment and I/O.

Broader reflections on AI, art, and responsibility

  • Some see such projects as inspiring examples of “intelligence as a new fundamental layer” and a way to push open experimentation (including tiny frameworks and low‑level drivers).
  • Others worry that technology pursuits like AI can become harmful “perversions” if pursued without social responsibility, while defenders emphasize individual joy in building things.