Google Gemma 4 Runs Natively on iPhone with Full Offline AI Inference

Original Article ↗ Hacker News Discussion ↗

Google’s new Gemma 4 small language models can now run entirely on-device on recent iPhones and Android phones, bringing offline AI assistants closer to everyday use. Commenters report that performance is surprisingly good on high‑end hardware but limited by RAM, thermal throttling, and battery drain, especially on phones where inference is routed through the GPU instead of dedicated NPUs. The thread also highlights broader issues: Apple’s restrictive App Store rules for apps that embed local LLMs, questions about the real usefulness and coherence of small models, and skepticism toward hype‑driven, low‑detail marketing around “edge AI.”

Performance: iPhone vs Android / Desktop

  • Multiple reports say Gemma 4 runs on both iPhone and Android with similar speed on current flagships; one person finds Pixel slightly faster than iPhone 15.
  • iPhones are reported to thermal throttle on long responses, slowing token generation after a while, whereas a newer Pixel keeps going.
  • Benchmarks from Edge Gallery on iPhone 16 Pro: ~231 tokens/s prefill, ~16 tokens/s decode, ~1.16s to first token (GPU backend, 4B model).
  • Desktop users run much larger models (e.g., 26B and 122B) on 64–128 GB RAM systems, achieving ~35–40 tokens/s and using them as daily drivers, but these are far beyond phone capabilities.

Coherence and Practical Usefulness of Local Models

  • Several users find Gemma 4 edge models (E2B/E4B), Qwen 3.5 9B/27B, and others coherent and useful for: simple commands, tone-polishing emails, moderate coding, security/OS work, and even some tax/legal-style reasoning.
  • Others remain skeptical, saying small on-device models are still weaker than top cloud models and advising caution for factual questions (e.g., pet safety).
  • On phones, heavy tasks or long contexts quickly hit thermal and battery limits; commenters expect more realistic use in short, focused interactions or tiny specialized models.

Apple Ecosystem: App Store Rules and ANE Limitations

  • Some developers report Apple blocking or slowing updates to apps that embed local LLMs, citing guideline 2.5.2 about downloading/executing new code.
  • Others note existing apps that still run Gemma locally but say Apple has been “slowly cutting them off” and may get stricter as LLMs threaten some app categories.
  • There is debate over whether Apple’s Neural Engine (ANE) is a practical target for LLMs; current Gemma demos often use the GPU instead, causing higher power draw and heat.
  • Some expect WWDC changes, with rumors of a new AI framework replacing Core ML to better support LLMs.

“Edge” vs “On-Device” Definitions

  • Commenters disagree on terminology: some insist “edge” means near-user but not on-device, others argue the user’s device is the “ultimate edge.”
  • Consensus: marketing uses the term loosely and inconsistently.

Critique of the Article and AI-Generated Content

  • Several call the article shallow “marketing slop” with no benchmarks or real detail, flagged as clickbait.
  • Multiple users suspect it is LLM-written, citing repeated rhetorical patterns; AI detectors are invoked, then challenged as unreliable and fundamentally limited.

Example Applications and Experiments

  • Projects mentioned include:
    • A visual description app for blind users using Gemma 4 E2B, reportedly faster than some cloud tools.
    • An offline “pocket vibe coder” on iPhone using Gemma 4 to generate and locally compile a TypeScript file for small interactive apps.