Gemma: New Open Models
Google’s release of the Gemma 2B and 7B “open” language models is welcomed for their strong benchmarks, local-run support (llama.cpp, Ollama, gemma.cpp), and permissive commercial use, positioning them as competitors to Llama 2, Mistral and Phi-2. Commenters probe the technical design (large 256k tokenizer, architecture choices, quantization behavior, 8k context) and note that base and instruction-tuned weights are available but training data and code are not, raising questions about how “open” these models really are. Much of the debate centers on Google’s custom license and safety/“alignment” constraints, with some seeing them as reasonable risk management and others viewing them as restrictive, politically biased, or unsuitable for serious commercial reliance.
Model performance and architecture
- Many see Gemma 7B’s benchmarks as roughly on par with Mistral 7B; 2B is viewed as weaker and beaten by Phi-2 on several public benchmarks.
- Architecture quirks draw interest: 256k‑token vocab, large FFN expansion, MQA/GQA choices, and tokenizer design closely related to Llama’s but larger and with many special tokens.
- Some argue 7B models may be near-saturated given Gemma’s 6T training tokens and only modest gains.
Availability and tooling
- Weights are downloadable (Kaggle, Hugging Face), with official JAX, Keras, PyTorch, and a standalone C++ engine.
- GGUF and llama.cpp support exist; Gemma appears in Ollama and other local UIs.
- Context length is 8k tokens. Question about extended-context or multimodal versions is answered only with “stay tuned.”
Licensing, “open” definition, and commercial use
- Weights are “open” but under custom terms, not an OSI license. No training code or datasets are released.
- Debate centers on calling this “open” vs “weights-available,” and comparisons to Meta’s and Mistral’s licenses.
- Terms include a “reasonable efforts to update” clause and a broad prohibited-use policy; some see this as a de facto kill switch and avoid Gemma for commercial use, preferring Apache/MIT models.
- Others note many startups ignore restrictive licenses in practice.
Safety, alignment, and bias
- Base (pretrained) and aligned (instruction-tuned) checkpoints are both released; users can fine-tune to change alignment.
- Some want explicit characterization of ideological/safety tuning; others argue this is hard to define or test.
- Separate but related Gemini image and history examples (e.g., racially “diversified” outputs) drive skepticism that Google’s models may be politically biased or “falsifying history.”
Quantization and local inference behavior
- Early 4‑bit Gemma 7B quantizations (e.g., in Ollama) produced nonsense text; higher-precision or different stacks (Transformers 4‑bit, gemma.cpp’s NUQ 4.5‑bit) behave better but are slower.
- Some report Gemma is slower than Llama/Mistral and finetuning may be impractical on consumer GPUs.
Community reactions and strategy
- Many appreciate the release and the team’s direct engagement on HN.
- Others see it as a PR move to promote Google’s ecosystem and erode closed-model moats, while still keeping key assets (data, training stack) proprietary.