Qwen3.8 Max now ranked as the best overall model by agentic index

Original Article ↗ Hacker News Discussion ↗

Qwen 3.8 Max, a large open-weights Chinese language model, briefly appeared as the top “agentic” model on the Artificial Analysis leaderboard, prompting scrutiny of how these composite benchmarks are constructed and updated. Commenters compare its capabilities, cost, and token-hungry reasoning style to rivals like Anthropic’s Opus, OpenAI’s GPT 5.6, Kimi K3, and DeepSeek, noting that on raw “intelligence” scores it still trails some proprietary models and is expensive to run. A recurring theme is that Chinese frontier and open models are now close enough in quality that trade-offs like price, latency, ability to self-host, and interaction style may matter more than small differences in benchmark scores.

Benchmarks & “Best Model” Claim

  • Artificial Analysis has multiple indices:
    • Agentic Index (GDPval‑AA v2, Tau³‑Banking) where Qwen3.8 Max briefly topped, now #2 after a methodology update.
    • Coding Agent Index (DeepSWE, Terminal‑Bench, SWE‑Atlas‑QnA) where Qwen3.8 Max is only partially tested and lags some OpenAI models on shared benchmarks.
  • Several commenters see the initial “best overall” framing as clickbaity and note the win is narrow and benchmark‑specific.
  • The methodology update (including a new grader model) landed while people were watching, causing visible score/order changes and feeding suspicion about leaderboard neutrality.

Model Quality & User Experience

  • Many report Qwen models, especially 3.6/3.8 large variants, as excellent for troubleshooting, agentic workflows, and general coding; some compare them favorably to Anthropic and OpenAI on complex debugging.
  • Others find Qwen/Chinese models sloppy or inconsistent: changing formats mid‑session, omitting requested items, or needing more explicit prompts.
  • Raw “intelligence index” leaderboards still often put Opus 5 and other proprietary models ahead; some distrust any benchmark where Opus beats alternatives they find stronger in daily use.

Cost, Tokens & Economics

  • Qwen3.8 Max is open‑weights but huge (~trillion‑scale), so mostly cloud‑only and nearly as expensive per task as top proprietary models.
  • Commenters stress that reasoning‑heavy models can emit many tokens, making mid‑priced models effectively as costly per task as high‑end ones.
  • Others highlight cheaper strong options (e.g., DeepSeek v4 Flash, Grok 4.5, GPT‑5.6 Luna after price cuts) and argue cost‑per‑quality is now a primary differentiator.

Local & Smaller Models

  • Strong interest in upcoming Qwen 3.8 27B and existing Qwen 3.6 27B/35B A3B as “local default” candidates on prosumer GPUs and Apple silicon.
  • Trade‑offs discussed: 27B seen as smarter but slower and heavy on KV cache; 35B A3B is faster and “dumber” but adequate for pair‑programming.

Anthropic / Claude vs Others

  • Many describe Opus 5 in coding agents as verbose, over‑planning, “token‑burning,” and reluctant to execute work; some revert to older Opus 4.6/4.8 or switch to Qwen/DeepSeek/OpenAI.
  • Others report Opus 5 as a clear upgrade in code quality if you adjust prompting and output‑style settings, but still complain about its prose and personality.
  • Subscription vs API pricing for Anthropic is seen as oddly skewed; some feel credits vanish extremely fast in agentic swarms.

China, Distillation & Bias

  • Commenters argue China has effectively “caught up”; differences between frontier models now feel small and use‑case/brand/personality driven.
  • Experiences with DeepSeek, GLM, Kimi, Qwen vary widely: some find them game‑changing and far better value; others report worse reliability than US models.
  • Debate over whether Chinese models are heavily distilled from Western outputs and whether that’s ethically distinct from training on web data.
  • Some raise concerns about censorship and narrative shaping in Chinese models; others counter that all major vendors (Chinese and Western) embed their own biases.

Agentic Harnesses & Communication

  • Multiple people stress that benchmarks miss a key factor for coding agents: how clearly the model explains what it did.
  • Opus 5, in particular, is criticized for dense jargon, invented terminology, and meandering updates that make human supervision harder, despite strong raw capability.
  • Several suggest we may be at a “good enough” intelligence plateau where speed, cost, and communication quality matter more than tiny benchmark deltas.