GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

Original Article ↗ Hacker News Discussion ↗

Observed bug & symptoms

  • GPT‑5.5 (especially Codex usage) shows “reasoning_output_tokens” clustering at specific values: 516, 1034, 1552, etc. (≈518 apart).
  • These clustered runs are strongly associated with wrong answers on complex reasoning tasks.
  • The phenomenon appears common on GPT‑5.5, less on 5.4, and almost absent on 5.2/5.3.
  • Some users report seeing identical spikes in Codex CLI and desktop logs and even in encrypted reasoning segments.

User experiments & impact

  • Simple reproducible puzzles show two modes:
    • ~516 reasoning tokens → wrong solution.
    • Thousands of reasoning tokens → correct solution.
  • Several report ~40% of runs hitting the 516-token “short‑circuit” state.
  • Others share histograms of past sessions confirming sharp spikes at 516 + 518·n.
  • Some see similar effects on 5.4; others say 5.4 high remains reliable.

Hypotheses about root cause

  • Inference / harness bug rather than model weights:
    • Batch-based or chunked reasoning (≈512‑token blocks) as a throughput optimization.
    • Misconfigured “adaptive thinking” / reasoning budget cutoff.
    • Interaction with an internal ## Intermediary updates system prompt.
  • Some reject “intentional nerf” narratives; others think cost‑cutting optimizations may have degraded quality.

Model versions & perceived quality

  • Multiple users feel GPT‑5.5 has regressed vs its launch and vs 5.3‑codex, citing more “stupid” or shallow outputs.
  • Some migrated from 5.5 back to 5.4, or to Claude, or to local/other cloud models; others did the opposite after Claude regressions.
  • A benchmarking site linked in the thread does not show clear degradation, leading to debate about what’s actually being measured.

Business, trust, and openness

  • Concern that subscription “frontier” models are silently changed over time; calls for refunds or more transparency.
  • Counterpoint: ToS allows backend optimizations; consistent behavior is only guaranteed if you self‑host models.
  • Appreciation that Codex is open source and has a public issue tracker, but frustration that a prior related issue was closed without explanation.
  • Discussion of using custom harnesses (e.g., Pi, OpenCode), CI/CD integration, and local models to avoid opaque server‑side changes.