Too much efficiency makes everything worse (2022)

Original Article ↗ Hacker News Discussion ↗

Pursuing maximum efficiency or optimizing hard for a single metric can make systems brittle and counterproductive, from global supply chains and standardized testing to social media and corporate management. Commenters connect this pattern to Goodhart’s and Campbell’s laws and to overfitting in machine learning: once a proxy measure becomes the target, behavior shifts to game the metric rather than improve underlying outcomes. Many argue that deliberately preserving “slack,” robustness and multiple goals — even at the cost of short‑term efficiency — is essential for resilience in economies, organizations, and public policy.

Scope of “too much efficiency”

  • Many see the core issue not as “efficiency” itself but:
    • Over‑optimizing on narrow or flawed metrics.
    • Confusing proxies (test scores, GDP, publication counts) with underlying goals (learning, wellbeing, scientific progress).
  • Several argue this is better framed as Goodhart’s/Campbell’s law: once a metric is targeted, it degrades and can backfire.

Overfitting vs bad metrics

  • Some agree the overfitting analogy from ML is illuminating: optimizing a proxy beyond a point yields diminishing, then negative returns.
  • Others say the post conflates:
    • Overfitting (too complex a model to limited data),
    • Wrong metrics (mis-specified objective),
    • Excessive efficiency (sacrificing robustness).
  • There is debate whether “too much efficiency” is even the right phrase; many see the real problem as mis-specified or static objectives.

Efficiency vs robustness and slack

  • Strong theme: highly optimized systems are brittle:
    • Just‑in‑time supply chains, COVID-era shortages, global shipping disruptions.
    • Queuing theory: as utilization → 100%, wait times → ∞; you need slack.
    • Error-correcting codes and other technical systems that fail catastrophically at capacity limits.
  • Slack is framed as:
    • Necessary for resilience, experimentation, and long-term survival.
    • Social/organizational analogs include “lazy” workers/ants, backup staff, mixed forests vs monocultures.

Political, economic, and social angles

  • Discussion touches on:
    • Capitalism optimizing for efficiency at the firm level while system-level resilience is offloaded to turnover and bailouts, possibly creating an “inefficient Nash equilibrium”.
    • Planned economies and overcentralization as “too efficient” in theory but fragile in practice.
    • Concerns about metrics like GDP and standardized tests as overused proxies that distort education and policy.
  • Some argue the cure is not more control or more metrics, but loosening control and accepting inefficiency as a feature.

Mitigations, alternatives, and open questions

  • Proposed mitigations/parallels from ML and systems theory:
    • Build robustness/slack into optimization criteria.
    • Inject noise or randomness (e.g., sortition in politics).
    • Use multiple metrics, change goals over time, favor simplicity and “good enough” models.
  • Skeptics note:
    • These mitigations themselves can be gamed or over‑optimized.
    • Formalizing these ideas across ML, economics, and politics is promising but historically difficult; complex human systems resist clean mathematical treatment.