Qwen 3.6 27B is the sweet spot for local development

Original Article ↗ Hacker News Discussion ↗

Model quality & use cases

  • Many users find Qwen 3.6 27B the first “local” model that feels genuinely strong for coding, planning, and general reasoning, especially compared to older Llama variants.
  • 27B dense is widely seen as the sweet spot: better quality than Qwen 3.6 35B-A3B (MoE), which is faster but “dumber” and more prone to loops.
  • Some argue Gemma 4 31B QAT is stronger than Qwen 3.6 in quantized 24GB scenarios, especially for reasoning and C/C++ work; others still prefer Qwen for agentic coding.
  • Smaller models (Qwen 3.5/3.6 9B, Gemma 4 12B, Ornith 9B) work well for documentation, design, cleanup, and “second pair of eyes” tasks but struggle with complex coding/agentic workflows.
  • Specialized models (Qwen3-coder, AgentWorld, DeepSeek v4 Flash, GLM 5.2, Minimax) are mentioned as better for tools/agents or frontier-level work, but often too heavy for typical local rigs.

Hardware requirements & performance

  • Qwen 3.6 27B can run:
    • On 24–32GB VRAM GPUs (3090, 4090, 5090, 7900 XTX, R9700) with 4–6 bit quants and usable context; ~40–100 tok/s reported.
    • On 48–128GB Apple Silicon (M1–M5), often fully in unified RAM at 4-bit; speeds ~10–40 tok/s dense, much higher for 35B MoE.
  • M-series laptops can run these but get hot and loud; several advise using a Mac Mini/Studio or other headless box as a server.
  • Strix Halo and DGX/RTX Spark boxes give large RAM with moderate bandwidth; better suited to MoE and long context than to dense models vs high-end GPUs.
  • Bandwidth and KV-cache settings (e.g., fp16 vs quantized KV) strongly affect long-context quality and speed. Below Q4 or heavily quantized KV often degrades performance.

Local vs cloud economics

  • Strong disagreement:
    • One camp says $6–7k 128GB MacBooks or multi-GPU rigs are irrational versus cheap API access (e.g., DeepSeek v4 Flash, Qwen via OpenRouter), especially given hardware depreciation and power costs.
    • Another camp argues local is cheaper long term for heavy users, avoids token anxiety, works offline, and pays back quickly against developer salaries.
  • Some suggest waiting 1–2 years for RAM/GPU prices to normalize; others think open models and better quantization will make today’s hardware useful for years.

Developer workflows & tooling

  • Popular stacks: llama.cpp (GGUF), MLX/oMLX on Mac, LM Studio, Unsloth Studio; Ollama is criticized for technical and licensing/lock-in reasons.
  • Coding harnesses like Pi, OpenCode, Hermes, and custom agent frameworks are central; success often depends more on harness, planning, and context management than on the base model alone.
  • Users stress: give explicit scopes, point models at specific files, use plans/subagents/compaction; don’t expect unsupervised repo-wide refactors.

Skepticism, limitations, and conflicts

  • Several commenters say local models “suck” for serious development, especially on large or non-standard codebases; they revert to cloud models like Claude or DeepSeek.
  • Others report shipping real work with Qwen 3.6 27B/35B and see them as good enough for many day-to-day tasks, though still weaker than frontier models.
  • Thought loops, hallucinated tool calls, and failures on niche or legacy stacks (e.g., old PHP/WordPress, unusual architectures) are common complaints.
  • There’s disagreement over quantization impact: some claim Q8 is “virtually lossless”; others report substantial degradation in long-context, tool-heavy scenarios.

Motivations: learning, privacy, and control

  • Many value local models for:
    • Privacy and regulatory needs (sensitive code/data, government or enterprise settings).
    • Deep understanding of LLMs, hardware, and tooling by tinkering locally.
    • Independence from vendor policy shifts, censorship, or future price hikes.
  • Others see local LLMs as an expensive hobby or educational tool rather than a practical replacement for cloud AI today.