GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

Original Article ↗ Hacker News Discussion ↗

Users report a reproducible bug in OpenAI’s GPT‑5.5 Codex where internal “reasoning” tokens cluster at fixed thresholds (around 516 + 518n), often coinciding with incorrect answers on complex problems. Many perceive a sharp decline in reliability compared to earlier GPT‑5.x versions, speculate that aggressive cost‑saving optimizations or adaptive reasoning limits are misconfigured, and contrast this with the relative stability of older models or local setups. The incident is fueling broader concerns about opaque server‑side changes, vendor lock‑in, and the need for more transparent, controllable AI tooling.

Observed bug & symptoms

  • GPT‑5.5 (especially Codex usage) shows “reasoning_output_tokens” clustering at specific values: 516, 1034, 1552, etc. (≈518 apart).
  • These clustered runs are strongly associated with wrong answers on complex reasoning tasks.
  • The phenomenon appears common on GPT‑5.5, less on 5.4, and almost absent on 5.2/5.3.
  • Some users report seeing identical spikes in Codex CLI and desktop logs and even in encrypted reasoning segments.

User experiments & impact

  • Simple reproducible puzzles show two modes:
    • ~516 reasoning tokens → wrong solution.
    • Thousands of reasoning tokens → correct solution.
  • Several report ~40% of runs hitting the 516-token “short‑circuit” state.
  • Others share histograms of past sessions confirming sharp spikes at 516 + 518·n.
  • Some see similar effects on 5.4; others say 5.4 high remains reliable.

Hypotheses about root cause

  • Inference / harness bug rather than model weights:
    • Batch-based or chunked reasoning (≈512‑token blocks) as a throughput optimization.
    • Misconfigured “adaptive thinking” / reasoning budget cutoff.
    • Interaction with an internal ## Intermediary updates system prompt.
  • Some reject “intentional nerf” narratives; others think cost‑cutting optimizations may have degraded quality.

Model versions & perceived quality

  • Multiple users feel GPT‑5.5 has regressed vs its launch and vs 5.3‑codex, citing more “stupid” or shallow outputs.
  • Some migrated from 5.5 back to 5.4, or to Claude, or to local/other cloud models; others did the opposite after Claude regressions.
  • A benchmarking site linked in the thread does not show clear degradation, leading to debate about what’s actually being measured.

Business, trust, and openness

  • Concern that subscription “frontier” models are silently changed over time; calls for refunds or more transparency.
  • Counterpoint: ToS allows backend optimizations; consistent behavior is only guaranteed if you self‑host models.
  • Appreciation that Codex is open source and has a public issue tracker, but frustration that a prior related issue was closed without explanation.
  • Discussion of using custom harnesses (e.g., Pi, OpenCode), CI/CD integration, and local models to avoid opaque server‑side changes.