90% of US working hours are spent on tasks not covered well by AI benchmarks
A new analysis claims that around 90% of U.S. work tasks aren’t represented in current AI benchmarks, prompting debate over how far today’s models can really go in automating “real” jobs. Commenters argue that LLMs are well suited to tightly scoped, verifiable digital tasks like coding, but struggle with the messy, political, and physical aspects that dominate most work, from stakeholder wrangling to janitorial labor. Many see near‑term gains as modest and uneven, raising concerns about job quality, inequality, and whether automation will primarily cut costs for employers rather than broadly improve living standards.
Scope of AI Benchmarks vs Real Work
- Thread agrees benchmarks mostly cover digital, tightly-feedback-loop tasks, not the messy, interpersonal, or physical majority of work.
- Several note benchmarks often target high-wage “AI-exposed” roles (law, coding) and miss support roles (legal secretaries, clerks, food service, janitorial).
- Some argue even “covered” occupations are only benchmarked on low‑value sub-tasks, not the highest‑leverage work.
What Software Engineers Actually Do
- Many report that only a small fraction of their time is coding; most is:
- Stakeholder wrangling, politics, expectation management, meetings.
- Coordination across teams, integration, and dealing with organizational hierarchy.
- There’s a strong view that coding was never the main bottleneck; AI accelerates an already-fast part.
Automation Priorities and Constraints
- Consensus: AI hits tasks that are digital, verifiable, and have scalable synthetic training data (code, math, some back-office work).
- Robotics and physical manipulation are seen as the last barrier; debate over whether robots will economically beat low‑wage labor anytime soon.
- Some expect eventually “everything representable as a program” gets automated; others cite slow, gritty progress in real‑world automation.
Economic and Social Effects
- Sharp disagreement on whether tech has produced “disinflation” in necessities (housing, healthcare, food) vs mainly making luxuries cheaper.
- Many expect AI to first erode high-paid white‑collar wages, deepening inequality and hollowing out the middle class.
- Some frame this as reducing “unearned returns” for high‑IQ professions; others see it as further wealth concentration toward billionaires.
- Fears of high structural unemployment, especially for young/entry‑level workers; calls for tax and social policy changes (UBI, taxing capital/AI more).
Bullshit Jobs, Hidden Work & the Doorman Fallacy
- Debate over how many jobs are “bullshit” vs performing hard‑to‑measure but real coordination, safety, and customer experience functions.
- Doorman/attendant examples used to show automation often ignores intangible value until it’s lost.
Productivity, Bottlenecks, and AI Hype
- Amdahl’s Law and Baumol’s Cost Disease invoked: speeding the automatable 10–20% doesn’t transform outcomes if approvals, alignment, and QA dominate.
- Individuals report significant personal speedups (e.g., coding, scripting) but little visible macro productivity or headcount change yet.
- Concern that AI is overhyped as job‑killer and under‑evaluated in domains lacking good benchmarks.