Fable 5 – Median thinking declined in August

Original Article ↗ Hacker News Discussion ↗

Claims that Anthropic’s Fable 5 model has quietly degraded over weeks, seemingly “thinking” less and performing worse, have reignited worries that AI vendors are cutting inference costs after launch while still selling the same branded model. Commenters debate how much of the perceived decline is real versus user bias or changing workloads, and whether token-level telemetry and third‑party benchmarks show systematic under‑delivery relative to advertised “reasoning” capabilities. The thread broadens into calls for transparency, regulation, and greater use of open or self‑hosted models to avoid opaque A/B tests, hidden quantization, and shifting service quality.

Perceived Model Degradation

  • Many users report that Fable 5 (and earlier Opus versions, plus other vendors’ models) feel clearly worse weeks after launch: less proactive, more shallow, more “eager to please,” worse at code navigation and tool use.
  • Complaints span domains (distributed systems, embedded, general coding, math, brainstorming); some say day‑to‑day “mood swings” make models feel like slot machines.
  • Several note heavier guardrails and system-prompt changes post‑safety interventions as a possible source of “nerfed” behavior.

Possible Technical / Economic Causes

  • Hypotheses: lower test‑time compute, runtime quantization, reduced “thinking tokens,” degraded KV cache resolution, load shedding under high demand, secret routing to cheaper models, or auto‑mode effort downgrades.
  • Some suggest capacity constraints and training of new models lead to reallocating compute away from existing ones.
  • Others propose “planned obsolescence” / “Shepard tone” strategies: launch at full quality, slowly cheapen inference, then release a “new” model that mainly restores original behavior.

Measurement, Benchmarks, and Methodology

  • The original “thinking token” analysis is heavily debated:
    • Supporters say the multi‑week, multi‑project corpus and token trends strongly suggest systematic shifts.
    • Critics argue it’s post‑hoc on changing workloads, not a fixed benchmark; day‑to‑day variance could be user‑side.
    • Comparing coding sessions to ARC‑AGI‑2 reasoning-token counts is seen by many as inappropriate, since those tasks are deliberately extreme.
  • External trackers (e.g., repeated coding benchmarks) generally show small fluctuations; some see real dips, others say they’re within expected noise.

Trust, Transparency, and Regulation

  • Strong concern that providers run opaque “black boxes,” can change quantization, effort, or even underlying models without disclosure.
  • Debate whether this is outright fraud or covered by ToS that allow modifying or degrading services.
  • Multiple calls for:
    • Independent “weights and measures”‑style oversight,
    • SLAs or metrics for “intelligence,”
    • Proofs of model/quantization identity per request.

User Perception, Non‑Determinism, and Alternatives

  • Several argue humans are bad at judging stochastic systems and are subject to hedonic adaptation; the model may be constant while expectations rise or projects become more complex.
  • Others insist the change is too large and abrupt to be explained by vibes alone.
  • Frustration is driving interest in open‑weight and self‑hosted models (Qwen, GLM, DeepSeek, etc.) for stability and control, even if frontier capability is slightly lower or hardware costs are high.