What sort of maths are LLMs good at?

Original Article ↗ Hacker News Discussion ↗

Large language models are starting to assist with formal mathematical proofs and counterexample search, yet they still fail at many everyday reasoning tasks and messy real-world problems, such as interpreting job ads or handling open-ended logic puzzles. Commenters contrast LLMs’ strength at pattern matching, code and theorem-proving within well-specified systems with their weakness in general reasoning benchmarks, spatial understanding and reliably following nuanced instructions. This leads to debate over whether current systems qualify as “general intelligence,” how much progress is driven by brute-force search plus verification, and whether future models might ever produce genuinely new, elegant mathematical ideas rather than incremental or brute-force results.

Scope of LLM Capabilities

  • Many commenters distinguish between:
    • Strong performance on formal, closed-domain tasks (math, code, formal proofs, Lean).
    • Weak, inconsistent performance on open-ended “normal world” tasks (job search, general reasoning, commonsense).

Real‑World Task Failures vs. Prompt Quality

  • One commenter reports frontier models failing a seemingly simple agentic task: finding freelance IT consulting gigs in a specific country, conflating:
    • Freelance vs permanent roles.
    • Job location vs agency location.
  • Others push back that:
    • The described prompt is hard to parse; actual prompt quality is unknown.
    • Data quality and scraping ambiguity (job boards, agencies) likely matter.
  • There is a broader point that end‑to‑end LLM agents for messy real‑world workflows remain brittle and need decomposed, verifiable pipelines.

Reasoning Benchmarks and Limits

  • Cited benchmarks (General365, ARC‑AGI‑3) show:
    • Frontier models score far below humans on diverse reasoning tasks when background knowledge is controlled.
    • Example puzzles are extremely hard even for humans; some argue 60%+ accuracy is impressive, others see it as evidence against “AGI”.
  • Another line of criticism: LLMs are easily distracted by irrelevant context; longer, more verbose prompts can hurt.

Math, Proof, and “Brute Force” vs Insight

  • Discussion of recent LLM‑assisted mathematical results emphasizes:
    • Heavy use of generate‑and‑test, many sub‑agents, thousands of scripts/commands.
    • Some see this as glorified brute‑force search; others frame it as iterative heuristic search akin to evolution.
  • LLMs are viewed as:
    • Excellent at grinding through cases, searching for counterexamples, and producing long formal proofs when paired with verifiers/SAT solvers.
    • Not yet clearly capable of producing new, deeply “elegant” concepts (e.g., analogues of Fourier transform).

Intelligence, AGI, and Trust

  • One camp argues that:
    • Broad world knowledge + simulated reasoning + cross‑domain transfer already meets their personal bar for “AGI”.
  • Critics respond that:
    • Failure on child‑level abstraction tasks and practical brittleness contradict this.
    • True AGI would need to be trusted with high‑stakes engineering (bridges, planes, warfare) without human safety nets, which is far from current reality.
  • Several note that human expertise is still required both to guide LLMs and to verify results; lay users rarely achieve breakthroughs.

Meta‑Observations

  • LLMs are framed as powerful generators of “plausible human‑like reasoning traces” that can upgrade stochastic search, but:
    • They remain error‑prone on systematic, multi‑step reasoning, especially at long chains or in concurrency/temporal logic.
    • There is concern about a coming flood of machine‑generated proofs and results that only a tiny expert community can verify.