Gemma 3n preview: Mobile-first AI

Original Article ↗ Hacker News Discussion ↗

Claims, Benchmarks & “What’s the Catch?”

  • Gemma 3n is pitched as a small, “effective 2–4B” on-device model with performance near large cloud models (e.g., Claude Sonnet) on Chatbot Arena.
  • Several commenters are skeptical: they argue Arena rankings increasingly reward style and “authoritative-sounding” answers rather than real problem-solving.
  • Others note that leaderboard or aggregate scores rarely predict performance on specific “hard tasks”; you still need task-specific evaluation.
  • A removed Aider coding benchmark initially suggested parity with GPT‑4-class models, but it was run at full float32 (high RAM use) and later disappeared from the model page, increasing skepticism.

Architecture, Parameters & Per-Layer Embeddings (PLE)

  • E2B/E4B models have more raw parameters than the “effective” count; the trick is parameter skipping and PLE caching so only part of the weights sit in RAM.
  • There’s confusion about what PLE actually is: the blog is vague, there’s no clear paper yet, and people speculate it’s some form of low‑dimensional token embedding fed into each layer and cached off-accelerator.
  • MatFormer is called out as a separate mechanism for elastic depth/width at inference, enabling “mix‑n‑match” submodels between E2B and E4B.
  • Unclear so far whether the architecture is straightforwardly compatible with llama.cpp and similar runtimes; licensing may also matter.

On-Device Usage & Performance

  • Multiple Android users report Gemma 3n running fully local via Google’s Edge Gallery app, with no network required after download.
  • Performance varies widely by device and accelerator:
    • Older phones can take many minutes per answer.
    • Recent high-end phones (Pixels, Galaxy Fold/Z) get several tokens per second, especially when using GPU; CPU is slower but still viable.
  • Vision works and can describe images and text in photos reasonably well, though speed and accuracy depend on hardware and image quality.
  • NPUs generally aren’t used yet; inference is via CPU/GPU (TFLite/OpenGL/OpenCL).

Capabilities, Limitations & Safety

  • Users report strong instruction-following and decent multimodal understanding for the size, but noticeably less world knowledge than big cloud models.
  • Examples of obvious logical/factual failures (e.g., size comparisons, object misclassification) show it’s far from Sonnet or Gemini quality.
  • Smaller models appear easier to jailbreak around safety filters (e.g., roleplay prompts).

Intelligence, Hype & Use Cases

  • Some are excited about “iPhone moment” implications: powerful, private, offline assistants, accessibility (for visually impaired users), inference caching, and local planning agents.
  • Others argue LLMs still resemble “smart search” or sophisticated memorization, not genuine understanding or reasoning; they expect hype to cool.
  • There’s a broader hope that OS-level shared models (Android, Chrome, iOS, Windows) will prevent every app from bundling its own huge LLM and ballooning storage use.