Literate programming: Knuth is doing it wrong (2014)
Literate programming, popularized by Donald Knuth, aims to interleave prose and source code so software can be read like a book, but many engineers question how well this scales to modern, collaborative projects. Commenters contrast Knuth’s typeset, book-oriented approach with today’s needs for navigable code, live tooling, tests, and hyperlinked docs, arguing that good structure, naming, and “literate” tests often deliver more practical value than fully woven narratives. Others see promise in lighter-weight variants—Jupyter-style notebooks, org-mode, tools like nbdev, and automatic docs from comments or tests—and speculate that future IDEs or LLMs could generate richer, on-demand explanations without imposing Knuth’s heavy workflow.
Usefulness and Scope of Literate Programming
- Many see “classic” literate programming (LP) as excellent for small, idea‑dense systems and educational books (e.g., compilers, TeX‑like systems, PBRT), where a lot of concept fits into relatively little code.
- Several argue most business code is boilerplate and integration, so the hard part is system composition, not explaining localized algorithms; LP is seen as overkill there.
- Some view LP as primarily about making intent and domain concepts clear (good naming, staged explanations), not about fancy typesetting.
Narrative vs Navigability
- A recurring theme: linear narrative and optimal code navigation are often orthogonal.
- Readers approach a codebase with different questions and need different “paths” (newcomer vs maintainer vs feature work). A single perfect ordering is considered impossible.
- Some highlight LP’s potential when combined with hyperlinked, indexed documents, but others argue modern IDEs, docs generators, and hypertext already cover much of what LP aimed for.
Tooling and Modern Variants
- Many variants are discussed: WEB/CWEB, noweb, literate Haskell, Emacs org‑mode, Markdown‑based tanglers, notebooks (Jupyter, Pluto, Marimo), nbdev, doctests, shell‑style literate tests, and custom DSLs for embedding docs in code.
- Notebooks are praised for mixing narrative and code but criticized for execution order, version control, and integration; newer tools (Pluto, Marimo) try to decouple layout from execution.
- Some propose “literate testing” where authoritative docs are generated from tests, not implementations.
Writing Burden and Commit Messages
- A major obstacle: most programmers are not motivated writers. Getting good commit messages is already hard, let alone full LP.
- One side claims rich commit messages are rarely read and not worth the time at high commit rates; others counter they aid future maintainers, reduce interrupts, reveal flaws while writing, and serve as self‑promotion.
Knuth‑Style LP and Scalability
- Knuth’s fragment‑based style (arbitrary named code chunks woven into programs) is praised for solo, book‑oriented work but widely seen as untenable for large, multi‑developer or very large codebases.
- Several note that his tooling partly addressed Pascal’s historical limitations; modern languages and tools reduce the need for such preprocessing.
Future Directions and LLMs
- Some imagine IDEs or LLMs generating “literate views” or reorganized narratives automatically, but others warn current models still hallucinate and mostly describe “what” rather than “why.”