Granite 4.1: IBM's 8B Model Matching 32B MoE

Original Article ↗ Hacker News Discussion ↗

Model performance and benchmarks

  • Several commenters test Granite 4.1 8B and find it “impressive for 8B,” fast on commodity GPUs and good for autocomplete and small tasks, but still weaker than larger open models for coding.
  • Qwen 3.6 (especially 27B/35B MoE) is repeatedly cited as a stronger local “champion,” notably for coding and agentic workflows. Some say it “burns” Gemma and Granite; others say Gemma 4 and Qwen 3.6 are roughly comparable with different strengths (Gemma: structured extraction, world knowledge; Qwen: coding, prompt adherence).
  • One benchmark claim in the linked article is challenged: a commenter notes qwen3.5‑9B scoring far above granite‑4.1‑30B on an external benchmark, calling the article’s performance framing misleading.
  • Another thread notes Granite 8B’s strong instruction following and low hallucination compared to peers, which some consider more practically valuable than raw “intelligence.”

Dense vs MoE and model design

  • Multiple comments discuss why 8B dense might compete with a 35B‑A3B MoE: using a rule-of-thumb, that MoE’s “effective” dense size is around √(A×T), putting them near 8–10B.
  • There is debate over MoE’s benefits. Some emphasize higher world knowledge at similar active params and easier scaling; others highlight training and routing complexity and question net gains.
  • Several note a broader trend: small models tend to be dense; large frontier models increasingly use MoE.

Local usage, tools, and UIs

  • People run Granite, Qwen, Gemma locally via llama.cpp, vLLM, LM Studio, Ollama, Open WebUI, Jan, etc.
  • Small 2–4B models are used for quick autocomplete, library usage reminders, unit tests, categorization, and data extraction where speed and low resource use matter more than peak accuracy.
  • Some describe agent experiments (e.g., controlling Kakoune) to probe tool use and robustness across models.

Non-reasoning vs reasoning and RLVR

  • A key point: Granite 4.1 models are explicitly “non‑reasoning,” optimized for token efficiency and speed, especially for enterprise/local use.
  • IBM’s decision not to add reasoning/RLVR is questioned; critics find the “cost/speed” justification unconvincing and suspect IBM may not yet have strong RLVR capability.

Licensing and “open source”

  • Weights are Apache‑2.0 licensed with permissive training data; some praise IBM’s indemnification stance.
  • Others argue “true” open source for ML should include full data and training recipes, not just weights, and cite other projects as examples.

Article slop, LLM text, and trust

  • Many comments complain the linked write‑up is obvious LLM‑generated “slop,” full of cliché transitions and low signal‑to‑noise.
  • Defenders argue tool use is fine if outputs are curated; critics counter that pervasive hallucination makes such articles not worth fact‑checking.
  • Broader debate ensues on whether LLM text is uniquely untrustworthy vs human writing, and how readers should adapt their trust heuristics.

Granite 4.1 specifics and other models

  • IBM’s Granite vision 4.1‑4B for tables/semantic kv extraction is called a potential “sleeper” if benchmarks hold.
  • Compact Granite embedding models (311M, 97M) are noted.
  • One user shares a failure case where Granite 4.1 8B repeatedly bungles a simple bitmask derivation, reinforcing perceptions that small dense models still struggle on low‑level, precision logic tasks.