80% of AI Projects Crash and Burn, Billions Wasted Says Rand Report

Original Article ↗ Hacker News Discussion ↗

Link & availability

  • Original blog repeatedly returned database errors; several users relied on archived copies.
  • Multiple commenters linked directly to the underlying RAND report instead.

Definition and reliability of the “80% fail” figure

  • RAND focuses on machine-learning-based projects inside existing organizations, excluding pure “prompt engineering” wrappers around pretrained LLMs.
  • The 80% number does not originate in RAND itself; it traces to a business article citing unspecified executive surveys (83–92% failure) without primary data.
  • Several commenters flag this as methodologically weak and advise skepticism about the exact percentage.

Reported causes of AI project failure

From RAND summary and discussion:

  • Problem selection: stakeholders misidentify or miscommunicate what the AI system is supposed to solve; same root cause as many failed software projects.
  • Data: organizations often lack sufficient, appropriate, or high-quality data; many hoard user data instead of generating the domain-specific documentation/expert material LLMs actually need.
  • Shiny-object syndrome: teams prioritize “latest tech” and AI branding over solving concrete user problems.
  • Infrastructure & MLOps: inadequate data pipelines, deployment infrastructure, and too few data engineers; ML specialists end up maintaining brittle data code.
  • Overreach: some projects tackle problems that are currently too hard for AI, or where any misprediction is too costly.

How bad is 80%? Comparisons and ambiguity

  • Thread notes claims that “non‑AI IT projects” fail at roughly half that rate, but other sources in the discussion say 60–70% of software projects in general fail.
  • Some see a 20% success rate for bleeding-edge tech as quite good; others say it’s worse than mature IT and may still be overstated because many projects haven’t failed yet.

Hype, management behavior, and internal politics

  • Many anecdotes of executives demanding “AI everywhere” for optics, often ignoring technical staff and basic ROI analysis.
  • Others describe the opposite: AI and LLMs blocked over security/compliance fears, or leaders being ultra‑conservative on spend.
  • Several compare this wave to earlier fads (blockchain, NoSQL, microservices, 3D movies), with “insert new tech into everything” mandates and predictable waste.

Data, cost, and feasibility constraints

  • Training serious models is described as prohibitively expensive; code is easy, data and compute are hard.
  • A common failure mode: AI used to paper over missing process, tooling, or documentation; without those, LLMs add little value.

Is the money “wasted”?

  • “Billions wasted” is debated: funds mostly turn into salaries, cloud bills, and hardware.
  • From a VC or portfolio view, high failure is acceptable if a few “black swan” wins pay for the rest; for enterprises seeking incremental automation, repeated failures hurt more.

Where AI appears to work

  • Coding assistance is repeatedly cited as a genuine productivity booster for some engineers, especially for tedious refactors and boilerplate, though others remain unconvinced or concerned about quality.
  • Commenters stress that many so‑called “AI products” are thin chat‑bot or LLM API wrappers with little differentiated value, contributing to the high apparent failure rate.