Vote on which of Hacker News' challenges for AI have been met
A community-built site catalogs years of Hacker News predictions about what AI “will never do” and lets users vote on whether those challenges have now been met, revealing sharp disagreement over how far current models have actually come. Commenters debate shifting goalposts, reliability vs. one-off demos, and ambiguous benchmarks like the Turing test, business automation, full‑stack app generation, and solving open math problems. Underneath is a broader split between those who see present systems as already transformative and those who argue that frequent errors, opaque training, and narrow success criteria mean many bold claims remain unproven.
Scope and ambiguity of “challenges”
- Many “AI challenges” from past HN comments are vague or compound (“do X and don’t do Y”), making yes/no voting hard.
- Key qualifiers like “arbitrary”, “business tasks”, “reliably”, “entirely”, “common applications” change the difficulty a lot.
- Several people say at least a third to three-quarters of items can’t really be judged, or are mis-summarized on the site (e.g., “and” turned into “or”, or challenges that were observations, not predictions).
Capabilities vs reliability
- Broad agreement that current LLMs can:
- Write a lot of useful code, sometimes entire small apps or niche tools.
- Do “business-speak” tasks (summaries, emails, translations) and complex research/data-wrangling workflows with big productivity gains.
- Help with taxes in simpler systems, and fill in official forms when guided.
- Generate coherent, if not great, books and long-form text, especially with planning and tooling.
- Major disagreement is about reliability: models still hallucinate, mis-cite sources, get technical details wrong, and can’t safely be trusted on high‑stakes or legally binding tasks without review or tests.
Turing test and “intelligence”
- Some argue that, for average users in short text chats, frontier models already pass the Turing test.
- Others insist a “proper” Turing test must be long, adversarial, and domain‑expert‑led; by that standard, current models still show clear tells (cooperation, causal/counterfactual failures, stylistic tics).
- Several conclude the original Turing test was simply too easy or conceptually flawed, especially given human limitations.
Research, math, and benchmarks
- Multiple commenters cite AI contributions to research‑level math problems (e.g., Navier–Stokes, Jacobian-related work), but debate whether that counts as “AI solved it” versus “AI‑augmented humans”.
- Some feel their long‑standing “AI will never do X” math benchmarks have now been met; others note controversy around provenance and degree of AI contribution.
Goalpost shifting and sentiment
- Historical pattern: early challenges were Turing test / code / simple actions; newer ones demand physical control, scientific breakthroughs, or near‑zero failure rates.
- A table of yearly “yes” fractions shows declining agreement over time, interpreted by some as moving goalposts, by others as simply harder questions.
- Thread reveals a split: one camp sees denial of obvious progress as “ostriching”; another sees boosters as overstating current capabilities and ignoring reliability and edge cases.