FunSearch: Making new discoveries in mathematical sciences using LLMs

Original Article ↗ Hacker News Discussion ↗

DeepMind’s new “FunSearch” system uses large language models to iteratively generate and evolve short Python programs, guided by an automatic scoring function, to tackle hard combinatorial problems like cap sets and bin packing. Commenters are split on whether the breakthrough reflects genuine new mathematical insight from the LLM or chiefly the power of an evolutionary search that keeps only the best code variants, noting that similar ideas have long existed in genetic programming and inductive program synthesis. The exchange highlights broader questions about how to fairly credit LLMs in hybrid systems, how much they really “reason,” what experiments are needed to prove their value over traditional methods, and what this line of work implies for future scientific discovery and AI capabilities.

Overall reaction to FunSearch

  • Many find the result striking: LLM-guided search produced state-of-the-art solutions for cap sets and bin packing, beating prior computational solvers.
  • Others see it as an incremental improvement on genetic programming / heuristic search rather than a revolution or “since fire” moment.

What the LLM is actually doing

  • Broad agreement that the heavy lifting is the evolutionary search + evaluator; the LLM mainly proposes code edits instead of random mutations.
  • Supporters argue the LLM is crucial because it:
    • Generates syntactically correct, plausible-looking programs.
    • Avoids a “cold start” where most random programs have zero fitness.
    • Serves as an “expert-ish” code mutator, a general problem-solver at the programming level.
  • Skeptics note:
    • The LLM sees only code snippets and type signatures; it is not a domain expert in combinatorics.
    • Ablations compare against an extremely weak random mutation baseline, not against strong, existing genetic programming systems.
    • It remains unclear how much advantage comes specifically from the LLM vs. sheer compute and search.

Novelty, extrapolation, and “stochastic parrot” debate

  • Some argue this counters the view that LLMs merely regurgitate training data, citing high-dimensional extrapolation and evidently new mathematical constructions.
  • Others point to evidence of memorization and emphasize that generating many bad candidates plus a filter is not the same as genuine reasoning.
  • There’s consensus that LLMs can be practically useful even if they lack “true understanding.”

Scope, limitations, and extensions

  • FunSearch works best when:
    • There is a fast, rich evaluator.
    • The problem can be framed as evolving a small code kernel in a fixed skeleton.
  • The method is explicitly ill-suited to problems like proof generation, where a rich numeric score is unclear.
  • Several comments wish the system evolved formal proofs (e.g., in Lean) or integrated more symbolic reasoning and classical program synthesis.
  • Some note missing practical pieces: no reference implementation of the LLM interface in the repo and no clear third-party replications yet.

Broader implications

  • Seen by some as validation of “LLM + tools” or neurosymbolic hybrids as the right path.
  • Others stress this is impressive but far from singularity; it is mostly a powerful search-acceleration trick.