Google releases Gemma 4 open models

Original Article ↗ Hacker News Discussion ↗

Google’s release of the Gemma 4 open-weight models — ranging from small multimodal “E2B/E4B” variants to a 26B MoE and 31B dense model — is seen as a major step for high-quality, locally runnable AI under a permissive Apache 2.0 license. Commenters report strong real-world performance, especially from the 4B and 26B MoE models in tasks like coding, OCR, SQL, and agents, while noting rough edges around tool-calling, chat templates, and runtime support that are being rapidly patched by projects like llama.cpp, Unsloth, and LM Studio. The release intensifies comparisons with Qwen 3.5 and proprietary systems such as Gemini and GPT-OSS, renewing debate over the reliability of public benchmarks and the long-term strategy of big labs that open powerful models while still pushing larger closed variants in the cloud.

Model overview & capabilities

  • Gemma 4 adds “thinking”/reasoning traces, multimodal input (images; audio on E2B/E4B), and tool calling.
  • Lineup: small E2B/E4B (mobile‑focused, Gemma‑3n‑style architecture) plus 26B A4B MoE and 31B dense.
  • Long context (up to ~200k+ tokens on some setups) and strong performance on many public benchmarks; some users say reasoning feels notably advanced.

Licensing and variants

  • Released under Apache 2.0, seen as a major shift vs prior Gemma licenses and good for agents/BYOK.
  • Both base and instruction‑tuned (“‑it”) models are provided; “it” models are intended for assistant/chat use.

Local deployment & performance

  • Many reports of running Gemma 4 locally via llama.cpp, LM Studio, Ollama, MLX, Modular MAX, LiteRT‑LM, and others.
  • 26B A4B MoE praised for high token/sec at modest VRAM (e.g., ~150 tok/s on 4090, strong speeds on M‑series Macs) and good fit for agent frameworks.
  • 31B dense is noticeably slower but higher quality; can still run on 24–64GB setups with quantization.
  • On low‑power devices (e.g., Raspberry Pi 5) even E4B is very slow; on modern Macs it’s comfortably usable.

Quantization & tooling

  • Unsloth released GGUF “Dynamic 2.0” quants quickly; users report near‑full quality at 4‑bit with large memory savings.
  • Confusion for newcomers around model size vs quant level vs context length; tools like Unsloth Studio and llama.cpp auto‑sizing help.
  • Some interest in future QAT / NVFP4‑style variants and TurboQuant‑like KV compression.

Quality, benchmarks & comparisons

  • Consolidated benchmark tables show Gemma 4 31B roughly competitive with other large open models, but Qwen 3.5 often leads, especially on coding and some reasoning tests.
  • Several argue public benchmarks are heavily overfit/gamed; they trust human‑eval (e.g., Arena ELO) or private benchmarks more.
  • Others counter that private tests (e.g., ARC‑AGI 2) show Chinese models weaker and worry about training on test sets.

Use cases and early experiments

  • Reported uses: OCR + translation + embeddings for historical land records; PDF and table extraction; receipt/document tagging; spam filtering benchmarks; translation; RAG; code agents (Claude‑Code‑style workflows); local photo metadata and SVG/image generation.
  • Small E2B/E4B models impress some for on‑device multimodal tasks and SQL generation, but are weaker for code and complex reasoning than 26B/31B.

Concerns, bugs & limitations

  • Early issues with chat templates and tool‑calling in llama.cpp/clients caused broken behavior; fixes are landing but commenters warn against judging on day‑one bugs.
  • Some find “thinking” traces slow or theatrical and note hallucinations even when the model “pretends” to run scripts/commands.
  • Setup UX (especially on Windows) is still rough; users want simple installers and better defaults.