Rich Sutton on AI creativity and discovery

Interpretations of the Core Argument

• Many readers parse the argument as: creativity = variation + evaluation + selective retention, and claim that bare generative models mostly do variation.
• Some think the critique only applies to pretraining-era LLMs, not to modern systems with reinforcement learning and tools.
• Others say the talk is less “anti‑LLM” and more a call to embed evaluation and retention directly into AI workflows.

Novelty vs Quality (“Good and Novel”)

• The claim that LLM outputs are either “novel or good, but never both” is widely challenged.
• Commenters argue random exploration can still land on high‑quality ideas; the premise that randomness and goodness are mutually exclusive is disputed.
• Several note that recombining learned “pieces” (styles, abstractions, procedures) can yield genuinely new, useful compositions.

Role of Evaluation, Feedback, and Harnesses

• Strong agreement that closed‑loop systems (generation + test + refinement) underpin real discovery.
• Coding agents, math systems with proof checkers, and systems using compilers or theorem provers are cited as examples where evaluation is “neurosymbolically closed” and AI makes novel, validated advances.
• Many emphasize that the “agentic harness” around LLMs—tools, tests, self‑play, RL with verifiable rewards—is crucial, and the talk underplays this ecosystem.

Limits, Training Dynamics, and RLVR

• Debate over whether reinforcement learning with verifiable rewards truly expands beyond the base model’s distribution or just “mode‑seeks.”
• Some highlight continual-learning ideas (e.g., periodically reinitializing underused neurons) as a way to maintain variation and plasticity.
• Others argue current planners and search procedures remain “dumb” and are the true bottleneck, not the models.

Human vs Machine Creativity and Evaluation

• Several reject any special metaphysical barrier preventing AI from evaluating or discovering; humans also rely on tools and real‑world experiments.
• Others stress that humans have built‑in goals, embodiment, and long evolutionary history shaping evaluation, which AI lacks.
• Art and emotion: some say AI can create art if art is defined by evoking experiences in a viewer; others insist human intent and lived experience are essential.

Broader Reflections and Skepticism

• Some appreciate the framing of creativity as variation–evaluation–retention but find the conclusions unsurprising or underspecified.
• Concerns arise about over‑reliance on authority and dismissal of new empirical evidence from modern LLMs.
• Several see today’s models as already highly useful “discovery assistants,” even if they never reach paradigm‑shifting scientific genius.