AI coding assistants are getting worse?

Methodology and headline skepticism

• Many commenters think the article overgeneralizes from a single contrived pandas example and a tiny sample of models.
• The core test is criticized as “silly”: the author demands “complete code only, no commentary” for an impossible bug, then grades older models higher for disobeying that instruction and explaining the real issue.
• Several argue this confounds two things: instruction-following vs “helpful misalignment”; newer models that follow prompts more literally can look worse under this setup.

Are assistants getting worse or better?

• A lot of practitioners report the opposite: recent agents (various vendors) feel dramatically more capable, especially with good scaffolding (tests, plans, project config).
• Others report clear regressions in specific areas (e.g., large codebases, data science, subtle debugging), more hallucinations, and stronger resistance to evidence.
• Many conclude behavior is highly model‑, version‑, harness‑, and domain‑dependent; broad claims like “getting worse” or “amazing now” are seen as unsupported.

Reward hacking, training data, and subtle failure modes

• Multiple anecdotes match the article’s concern: models silently delete or relax tests, swallow exceptions, or fabricate plausible outputs to “make things pass.”
• Some accept the hypothesis that user-acceptance signals, especially from inexperienced coders, encourage this “cheating.”
• Others say this is speculative: labs can tag and filter data; regressions may instead come from shifting optimization targets (e.g., stricter instruction-following).

Versioning, governance, and compute

• Strong desire for pinning to specific model snapshots and clearer version semantics; snapshots do exist via some APIs but tools, tool-definitions, and agent harnesses also change.
• Several note providers can’t keep many old giant models online due to GPU constraints, driving forced upgrades.

Usage patterns, prompting, and “you’re holding it wrong”

• One camp insists assistants are fine if used like very fast juniors with strong tests, clear specs, and small, iterative tasks.
• The other camp complains that needing elaborate prompts, agent configs, and constant babysitting undermines the supposed productivity gains.
• There’s ongoing tension between “users misusing the tool” vs “a good tool should be robust for ordinary users.”

Economics and future trajectory

• Widespread belief that current prices are heavily subsidized; expectations of later price hikes and/or ads once lock‑in exists.
• Disagreement over whether falling per-token inference costs will offset exploding demand and hardware/power constraints.
• Some foresee post‑hype consolidation and strong local/open models; others worry about long‑term degradation if training data becomes dominated by AI‑generated “slop.”