Opus 5 is currently #1 on Artificial Analysis Intelligence Leaderboard
Opus 5 from Anthropic has briefly taken the top spot on the Artificial Analysis AI leaderboard, but many engineers argue its small performance edge over rivals like GPT‑5.6 Sol and Kimi K3 doesn’t justify its significantly higher cost. Commenters highlight that real-world value depends more on intelligence-per-dollar, reasoning settings, and task domain (e.g., coding vs. knowledge retrieval) than on a single “best model” score. A major pain point is Anthropic’s aggressive safety filters and automatic model downgrades—especially in biology, security, and debugging workflows—which some say make cheaper, less restricted or open models more practical despite slightly lower benchmark scores.
Overall leaderboard reaction
- Opus 5 tops ArtificialAnalysis’ “Intelligence Index,” but the margin over GPT‑5.6 Sol and other frontier models is small.
- Several commenters argue the more interesting charts are “intelligence vs cost/time” rather than the single headline score.
- Some see Opus 5’s position as validation of Anthropic’s technical progress; others say GPT‑5.6 Sol looks more attractive given similar performance at lower cost.
Cost, efficiency, and reasoning settings
- Many emphasize that reasoning level (“low/medium/high/max/xhigh”) radically changes cost and behavior.
- Opus 5 High is reported to score similarly to Sol Max with roughly comparable cost/time; Opus Max is much more expensive and sometimes overthinks.
- Some argue it’s usually better to use a smarter model at lower reasoning than a cheaper one at max.
- Several highlight Kimi K3 and other non‑US models as strong cost–performance options, though K3 is not yet fully represented in the AA charts.
Model roles and multi‑model workflows
- OpenAI’s Sol/Luna/Terra “stack” is discussed: Luna praised as an efficient workhorse/tool‑caller, Terra as a good conversationalist, Sol for delicate synthesis.
- Some propose pipelines: e.g., Sol or Opus for planning/architecture, cheaper or smaller models for implementation, multiple models for cross‑review.
User experiences with Opus 5 vs Fable, Opus 4.8, and Sol
- Mixed reports: some say Opus 5 is a clear generational leap, especially for game or complex design tasks; others find it more superficial or “lost” than Opus 4.8 or Sol.
- Complaints include overbuilding, weak UI design, or poor behavior when continuing old sessions; others praise less verbosity and more focused answers than 4.8.
- Several note that performance is highly codebase‑ and task‑dependent.
Guardrails, censorship, and model downgrades
- The most intense thread concerns Anthropic safety filters, especially on Fable and, to a lesser degree, Opus 5.
- Many report benign prompts (biology, medicine, chemistry, security, performance, even certain words like “cell,” “microbes,” “segfault,” “login”) triggering hard refusals or automatic downgrades to Opus.
- Users in biology, medicine, radiology, security, and reverse‑engineering describe the models as effectively unusable because of over‑broad classifiers.
- Some appreciate safety concerns; others argue risk is overstated and that overblocking harms science and productivity.
- There is debate over whether silent/automatic downgrades are acceptable; some point out settings exist to disable auto‑switching, but others dislike the entire pattern.
Benchmark usefulness and limitations
- Several criticize single “best model” leaderboards as misleading; real workloads vary, and domain‑specific strengths (e.g., coding vs knowledge vs agents) matter.
- Others defend benchmarks as still highly informative for narrowing choices, as long as users look at multiple metrics (cost, hallucinations, domain scores).
- Some call for stack‑specific or community‑maintained evals (e.g., for particular languages/frameworks).