Qwen 3.6 27B is the sweet spot for local development

Original Article ↗ Hacker News Discussion ↗

Qwen 3.6 27B is emerging as a standout open-weight large language model for local software development, with many reporting it’s the first model that feels “good enough” for real coding and agentic workflows on consumer hardware. Contributors compare performance, cost, and thermals across setups—128GB M-series Macs, AMD Strix Halo, older 3090/4090-class GPUs, and emerging platforms like DGX/RTX Spark—while debating how much RAM, VRAM, and quantization you actually need. A recurring theme is the trade-off between cheap, powerful cloud models and the privacy, control, and hands‑on learning that local models enable, alongside skepticism that spending thousands on top-end laptops is justified purely for local inference.

Model quality & use cases

  • Many users find Qwen 3.6 27B the first “local” model that feels genuinely strong for coding, planning, and general reasoning, especially compared to older Llama variants.
  • 27B dense is widely seen as the sweet spot: better quality than Qwen 3.6 35B-A3B (MoE), which is faster but “dumber” and more prone to loops.
  • Some argue Gemma 4 31B QAT is stronger than Qwen 3.6 in quantized 24GB scenarios, especially for reasoning and C/C++ work; others still prefer Qwen for agentic coding.
  • Smaller models (Qwen 3.5/3.6 9B, Gemma 4 12B, Ornith 9B) work well for documentation, design, cleanup, and “second pair of eyes” tasks but struggle with complex coding/agentic workflows.
  • Specialized models (Qwen3-coder, AgentWorld, DeepSeek v4 Flash, GLM 5.2, Minimax) are mentioned as better for tools/agents or frontier-level work, but often too heavy for typical local rigs.

Hardware requirements & performance

  • Qwen 3.6 27B can run:
    • On 24–32GB VRAM GPUs (3090, 4090, 5090, 7900 XTX, R9700) with 4–6 bit quants and usable context; ~40–100 tok/s reported.
    • On 48–128GB Apple Silicon (M1–M5), often fully in unified RAM at 4-bit; speeds ~10–40 tok/s dense, much higher for 35B MoE.
  • M-series laptops can run these but get hot and loud; several advise using a Mac Mini/Studio or other headless box as a server.
  • Strix Halo and DGX/RTX Spark boxes give large RAM with moderate bandwidth; better suited to MoE and long context than to dense models vs high-end GPUs.
  • Bandwidth and KV-cache settings (e.g., fp16 vs quantized KV) strongly affect long-context quality and speed. Below Q4 or heavily quantized KV often degrades performance.

Local vs cloud economics

  • Strong disagreement:
    • One camp says $6–7k 128GB MacBooks or multi-GPU rigs are irrational versus cheap API access (e.g., DeepSeek v4 Flash, Qwen via OpenRouter), especially given hardware depreciation and power costs.
    • Another camp argues local is cheaper long term for heavy users, avoids token anxiety, works offline, and pays back quickly against developer salaries.
  • Some suggest waiting 1–2 years for RAM/GPU prices to normalize; others think open models and better quantization will make today’s hardware useful for years.

Developer workflows & tooling

  • Popular stacks: llama.cpp (GGUF), MLX/oMLX on Mac, LM Studio, Unsloth Studio; Ollama is criticized for technical and licensing/lock-in reasons.
  • Coding harnesses like Pi, OpenCode, Hermes, and custom agent frameworks are central; success often depends more on harness, planning, and context management than on the base model alone.
  • Users stress: give explicit scopes, point models at specific files, use plans/subagents/compaction; don’t expect unsupervised repo-wide refactors.

Skepticism, limitations, and conflicts

  • Several commenters say local models “suck” for serious development, especially on large or non-standard codebases; they revert to cloud models like Claude or DeepSeek.
  • Others report shipping real work with Qwen 3.6 27B/35B and see them as good enough for many day-to-day tasks, though still weaker than frontier models.
  • Thought loops, hallucinated tool calls, and failures on niche or legacy stacks (e.g., old PHP/WordPress, unusual architectures) are common complaints.
  • There’s disagreement over quantization impact: some claim Q8 is “virtually lossless”; others report substantial degradation in long-context, tool-heavy scenarios.

Motivations: learning, privacy, and control

  • Many value local models for:
    • Privacy and regulatory needs (sensitive code/data, government or enterprise settings).
    • Deep understanding of LLMs, hardware, and tooling by tinkering locally.
    • Independence from vendor policy shifts, censorship, or future price hikes.
  • Others see local LLMs as an expensive hobby or educational tool rather than a practical replacement for cloud AI today.