Show HN: Next-token prediction in JavaScript

Original Article ↗ Hacker News Discussion ↗

Project overview & goals

  • Project is a JavaScript “next-token prediction” library using text corpora to build a language model.
  • Exposed capabilities: predict the next token(s), complete phrases, and generate top‑k completions.
  • Intended use cases: autocomplete, autocorrect, spell checking, search/lookup; author also explores chatbots and “next‑pixel prediction.”

Is this an LLM? Core dispute

  • Many commenters argue this is not a large language model: it’s a trie / n‑gram / Markov-style language model, not a neural network with learned weights and matrix multiplications.
  • They stress that LLMs are artificial neural networks; tries and Markov chains are a different class of next‑token algorithms.
  • The project author repeatedly clarifies it “isn’t an LLM” but a library to build models, while also claiming it can be used to build LLMs and initially advertised “build fast LLMs from scratch.”
  • After pushback, references to LLMs in the title/description were removed or toned down.

Implementation details & terminology debates

  • Internals: nested JavaScript objects (trie-like) storing observed sequences and bigram counts; described as a language model, sometimes compared to Markov chains and n‑grams.
  • The library’s “embeddings” are frequency- and structure-based feature vectors (low-dimensional), not semantic high‑dimensional vectors used in modern LLMs.
  • Commenters point to Markov chains, n‑gram models, and PPM as related, older techniques and say this is a naïve n‑gram LM.
  • Some argue attention mechanisms and richer embeddings are key to modern LLM capabilities and are missing here.

Scalability, performance, and language choice

  • Critics say a trie over large corpora will have huge fan‑out and memory usage and won’t scale like neural LLMs, especially for unseen text.
  • There’s a contentious side thread on JavaScript vs Python performance, time complexity of data structures, and V8 internals; several commenters correct misconceptions about O(1) operations and process/threading.

Capabilities vs modern LLMs

  • Commenters note this approach can autocomplete or mimic training text but cannot generalize or answer novel questions like current LLMs.
  • The author proposes adding an NLP layer (e.g., parts‑of‑speech analysis) to select more contextually appropriate completions for a chatbot.

Meta: reception and learning resources

  • Thread mixes encouragement for experimentation with strong skepticism about claims and terminology.
  • Multiple commenters recommend more rigorous learning resources (e.g., classic NLP textbooks, Karpathy tutorials, transformer examples) to better understand modern LLMs.