What happens when an LLM never sees material beyond fifth grade?
Limiting a language model’s training data to U.S. K–5 curriculum material produces an AI that can mimic a very well-read child but still hallucinates and confidently answers far beyond its supposed knowledge level. Commenters probe how well the grade-level filtering worked, whether such a model can ever reliably say “I don’t know,” and what this implies about current LLMs more broadly—namely that their capabilities are tightly bounded by pretraining data rather than emerging abstract “intelligence.” Some see this as a valuable research probe into knowledge ceilings and humility in models; others argue it mainly confirms that today’s systems remain sophisticated autocomplete rather than true reasoners.
Scope and setup of the experiment
- Model is trained on text filtered to roughly “≤5th grade” / K–5 U.S. curriculum.
- Filtering pipeline: start from an educational web corpus, apply word age-of-acquisition thresholds (discard samples where >5% of words exceed age 12), remove higher-level math symbols, then train a classifier to refine the split.
- Some commenters want more transparency: examples of included data and human spot‑checks of grade level; dataset is not yet released.
Leakage and data-quality concerns
- Several responses suggest the model clearly knows concepts beyond 5th grade (Rayleigh scattering, quantum entanglement, Schrödinger’s cat jokes, basic Python, etc.).
- People speculate the filter is imperfect; “5% advanced words allowed” and adult-written children’s material may smuggle in higher-level knowledge.
- Others argue many of these explanations do appear in kid science books, so they may still be within a plausible K–5 corpus.
Behavior, answer quality, and “fifth‑grade intelligence”
- Users report wildly wrong or incoherent answers (e.g., five-times table, square root of −1, impact of asbestos, maximizing a function, sorting algorithms causing an infinite loop).
- Some see it as “just a weak model,” not a principled fifth‑grader analogue; a real child would often say “I don’t know.”
- Others liken it to an omnivorous reader of every elementary text, with perfect recall but little genuine reasoning.
Hallucinations, inability to say “I don’t know,” and RLHF
- Large subthread debates why LLMs rarely say “I don’t know”:
- One view: base models imitate text where people almost never end with “I don’t know.”
- Counter‑view: instruction tuning and RLHF can explicitly train models to express uncertainty, but this competes with benchmarks that reward confident answers.
- Discussion of “know‑it‑all” behavior: LLMs as sycophantic, rarely subjectively refusing ideas; others note newer frontier models do push back or correct users more often.
- Some connect RLHF to reduced variance: fewer refusals may maximize benchmark scores and engagement.
Capability ceilings and implications for AGI
- Key claim from the paper (as paraphrased in the thread): scaling, instruction tuning, RL, and in‑context tricks amplify what the curriculum teaches but do not push beyond it; pretraining filter sets an effective capability ceiling.
- Many find this result important: suggests intelligence is closely bounded by training data scope, challenging hopes that sheer scale yields qualitatively new knowledge from limited corpora.
- Some see this as confirming “stochastic parrot” critiques; others remain interested in whether stronger reasoning training, memory, and long‑horizon tasks could still yield novel discoveries even from a constrained curriculum.
Alternative training ideas and potential uses
- Proposals include:
- Strict curriculum-based training across grades with checkpoints at each grade level.
- Curiosity-driven growth: when the model admits ignorance, a larger model fetches and adds new data.
- Training on highly verified, well‑balanced corpora instead of noisy web mixtures.
- A few see niche applications: a constrained K–5 model for home automation or safer assistants, though concerns about persistent hallucinations remain.