Qwen 3.8 27B is excellent, but it defaults to overthinking things

Original Article ↗ Hacker News Discussion ↗

Qwen 3.8 27B, a high‑end open local language model, is being praised for near–frontier‑level reasoning but criticized for “overthinking” by default, burning vast numbers of internal “thinking” tokens and running 5–10x slower than alternatives for many tasks. Users compare it with terser models like Muse Glimmer and Gemma 4, debate the trade‑off between model size and test‑time compute, and note that token efficiency directly impacts both latency and operating cost for agents. Many report good results after turning reasoning down or off, adding thinking budgets or LoRAs, or switching templates—highlighting both how powerful current local models have become and how much their usefulness depends on harness and configuration choices.

Model quality vs. overthinking

  • Qwen 3.8 27B is widely praised as a major leap over 3.6 and unusually strong for a local dense model, with some users benchmarking it near recent frontier models for reasoning and coding.
  • Main complaint: default “xhigh” reasoning mode produces huge thinking traces (often 10–30k+ tokens), making it 5–10x slower and very token‑inefficient for interactive work.
  • Overthinking often shows up as deep rabbit holes, excessive edge‑case analysis, and over‑engineered solutions (e.g., ornate SVG/HTML for simple prompts).

Reasoning modes, templates, and mitigations

  • Users report big gains from:
    • Turning reasoning off or using “low”/“medium” instead of “xhigh”.
    • Imposing hard reasoning budgets in llama.cpp (--thinking-budget, --thinking-message) or custom proxies that cut off thinking after N tokens and tell the model to proceed.
  • Some harnesses/templates default to xhigh; others (community “fixed” templates) set “medium” and expose a dropdown to tune reasoning effort.
  • LoRAs like “ThinkingCap” are being tested to reduce thinking tokens ~40–50% while keeping quality, with mixed but promising early results.

Comparisons to other models

  • Muse Glimmer 30B: much terser “thoughts,” lower token usage, faster in practice despite being slower in raw tok/s; some see it as less capable than Qwen 3.6/3.8.
  • Gemma 4 12B is cited as a standout small model (fast, multimodal, token‑efficient), though allegedly hurt by misconfigured templates and settings in some toolchains.
  • Several users note that many current open and proprietary models “overthink” due to training incentives and benchmark optimization.

Hardware, performance, and local vs cloud

  • Qwen 3.8 27B runs on higher‑end consumer GPUs and Apple Silicon, but speed and context size are heavily constrained by memory bandwidth and VRAM.
  • Users share real numbers: ~20–50 tok/s on decent GPUs or M‑series Macs at moderate context; long contexts and heavy reasoning can drive generation down to ~1 tok/s.
  • For some, electricity plus hardware makes cloud models (e.g., Luna/Terra/Sol, GPT‑5.x) cheaper and faster; others see local as preferable for privacy, agentic workflows, and long‑running jobs.

Broader views on “thinking”

  • Many like visible reasoning for debugging, safety, and deeper planning; others find it cognitively exhausting and prefer hidden or minimal thinking.
  • There is debate over whether chain‑of‑thought tokens reflect true internal reasoning vs. a “mumbling” substrate, and whether future models should move more toward latent‑space reasoning.