Qwen3.8-2.4T

Original Article ↗ Hacker News Discussion ↗

A new 2.4‑trillion‑parameter Qwen3.8 model has been released as open weights, impressing many with benchmark results near top proprietary models while raising practical questions about how to run a ~5 TB system in the real world. Commenters debate quantization strategies, hardware requirements, and whether extremely large 1‑bit or FP8 variants are worthwhile compared with smaller but better-optimized models like DeepSeek V4 Flash, GLM 5.2, or the upcoming Qwen3.8‑27B. There is also concern that the open variant is intentionally limited—shipping without vision, reduced context length, and no QAT quantization—making it harder to self-host than rivals such as Kimi K3 despite similar performance.

Overall release & positioning

  • Qwen3.8-2.4T is a ~2.4T-parameter MoE model with ~5TB bf16 weights (≈2.5TB fp8), positioned around Opus 4.8–Fable 5 benchmark levels.
  • It’s framed as a rival to Kimi K3 (2.8T-A100B) but is harder to serve at launch due to bf16/fp8 only and lack of QAT-ready 4-bit weights.

Scale, performance & comparisons

  • Claims: 1-bit quantized variant (~397GB, 95B active MoE) performs around “Opus 4.5 level,” with some arguing very large models degrade less under heavy quantization.
  • Others push back: 1-bit comparisons to full-precision models are called misleading; KL divergence is criticized as an inadequate proxy for capability.
  • Multiple commenters say DeepSeek v4 Flash and Kimi K3 already reach Opus 4.5–4.6 territory at far smaller size and cost.
  • For coding, some users report GLM 5.2 outperforming both Kimi K3 and Qwen 3.8 despite higher token usage, making it more economical for them.

Hardware requirements & quantization

  • Running full bf16 (≈4.9TB) or fp8 is seen as cluster-only; consumer setups must rely on quants.
  • Debate over guidance for a fixed RAM budget: older rule “bigger model at lower bits > smaller at higher bits” is now muddied by MoE, QAT, MTP/DFlash, KV efficiency.
  • Rough consensus:
    • 4-bit is often the lowest “safe” precision for standard models.
    • Sub‑4‑bit is usually poor unless the model is explicitly trained for it (few large models are).
  • Many users focus on upcoming 27B Qwen3.8 for local use (fits on high-end GPUs with Q4 quants), noting a “dead zone” in the 120–300B range for open releases.

Features & limitations

  • Open-weight 2.4T model lacks vision, has 250k context, and excludes some “thinking”/tooling features present in the proprietary Qwen3.8-Max (1M context, built-in tools, non-thinking mode).
  • Some see this as a step back from Qwen 3.5’s more complete open releases.
  • New reasoning_effort parameter (low/medium/xhigh) is meant to trade off speed vs depth, but users report loopiness can persist even at low.

Ecosystem, tooling & providers

  • Unsloth offers early quants and detailed docs; many praise their quality and responsiveness, though a few report issues with specific gguf builds.
  • Llama.cpp and similar tools can quantize without QAT but may sacrifice optimality; providers will likely favor fp8 or high-quality calibrated nvfp4 until good Q4 QAT exists.
  • Serving options mentioned include Alibaba’s own cloud, OpenRouter, DigitalOcean, Fireworks, OpenCode integrations; Bedrock is seen as drifting toward mostly proprietary US labs.

Chinese competition & policy context

  • Several commenters link Qwen’s open releases to Chinese industrial policy encouraging open models and tech transfer, framing it as both geopolitical strategy and a practical driver of openness.
  • There’s broad sentiment that Chinese open-weight releases are pressuring Western labs, preventing a fully closed “metered intelligence” future.

Hardware trajectory & future models

  • Estimates for sub‑$10k hardware to run such models unquantized range from ~5 years to ~2040, with bandwidth (HBM vs DDR) cited as a key limiter, not just capacity.
  • Others expect future smaller (<400B) models to reach similar intelligence long before commodity hardware can comfortably host 2.4T bf16.