A ChatGPT clone, in 3000 bytes of C, backed by GPT-2 (2023)

Original Article ↗ Hacker News Discussion ↗

Project scope and implementation

  • Thread clarifies this is a tiny C program (~3000 bytes, minified) that runs inference on an existing GPT‑2 TensorFlow checkpoint, not a full ChatGPT or training setup.
  • Unobfuscated, readable C source is linked and only modestly larger; the minified form is mainly for IOCCC/code‑golf style aesthetics.
  • Most of the “magic” resides in the downloaded ~475 MB model file, not in the code itself.
  • Prior similar IOCCC work using LSTMs is referenced; this project updates the idea to transformers/GPT‑2.

Output quality and “ChatGPT clone” debate

  • Multiple people who ran it report highly repetitive and low‑quality dialog (e.g., repeating “I am a computer model trained by OpenAI”, nonsensical math like “2+2= bird”).
  • Some argue that calling it a “ChatGPT clone” is misleading since GPT‑2 is not instruction‑tuned for chat and the author themselves notes the output is objectively poor.
  • Others are impressed it is even somewhat conversational given GPT‑2’s original training and recall older GPT‑2 outputs (e.g., fairy tales) that were weird but often coherent.
  • Several compare it unfavorably to classic rule‑based chatbots like ELIZA.

Purpose and value of the tiny implementation

  • Supporters frame it as:
    • A “demake” or low‑res homage, showing the core mechanism in minimal code.
    • An educational piece that demystifies transformers and shows that inference logic is conceptually simple.
    • A kind of technical art or craftsmanship, akin to IOCCC entries or mountain‑climbing: done for challenge and joy, not utility.
  • Critics question the practical usefulness: quality is poor, model/training dominate cost, and binary size doesn’t imply performance gains.

Models, data, and “size of intelligence”

  • Discussion distinguishes:
    • Engine code vs. model weights vs. training data, using video‑game “engine/assets” analogies.
  • Several note that the core math for LLMs is small; complexity lies in huge datasets and billions of learned parameters.
  • A broader debate emerges about whether AGI could be expressed in tens of thousands of lines of code, and whether focusing only on stateless math ignores embodiment and I/O.

Broader reflections on AI, art, and responsibility

  • Some see such projects as inspiring examples of “intelligence as a new fundamental layer” and a way to push open experimentation (including tiny frameworks and low‑level drivers).
  • Others worry that technology pursuits like AI can become harmful “perversions” if pursued without social responsibility, while defenders emphasize individual joy in building things.