The Prompt API
Chrome’s new Prompt API exposes an on-device large language model (Gemini Nano) to web pages, aiming to provide shared, local AI capabilities without forcing each site to ship its own model or rely on paid cloud services. Commenters welcome the privacy and standardization potential but criticize the huge disk space requirements, slow performance on typical hardware, model quality limits, and cross-browser fragmentation that makes prompt-based features hard to test reliably. Others explore speculative uses such as AI-powered content filtering and “de-snarkifying” social media, while warning about opaque data practices and the risk that AI layers will increasingly mediate what users see online.
Model size, storage, and download behavior
- Prompt API requires large on-device models; docs say “at least 22 GB” free space, which many see as excessive for a browser feature.
- Actual model folders reported around 3–4 GB, with speculation that 22 GB is a safety threshold to allow multiple versions and avoid filling disks.
- Models are lazily downloaded on first use, cached once per browser, and shared across sites.
User experience and performance
- Several comments describe slow token generation, heating devices, and long initial download, especially on “baseline” hardware.
- Some would rather pay for fast hosted models than run a sluggish local one; others see local as “good enough” for light tasks like search or summarization.
- There’s concern that low-end models are only useful for trivial or very short interactions.
Privacy, surveillance, and abuse risks
- Some view on-device inference as privacy-preserving; others distrust Chrome/Google and fear background analysis of user data.
- Speculation about covert analytics or wiretap-adjacent uses, though others note this API isn’t required for such behavior.
- Worries about using visitors’ machines for spam or distributed computation; countered by arguments that tiny models and low payoff limit abuse.
Use cases and experiments
- Reported uses include: local search, summarizing hack-day writeups, AI subject-line generation, text adventure modification, AI-based email triage, and potential ad/cookie blockers.
- A large subthread explores “de-snarkifying” social media and comment sections: filtering aggression, summarizing long threads, and stripping clickbait.
- Some welcome this as removing “junk calories”; others fear homogenized “slop” and further detachment from unfiltered reality.
Standardization, browser ecosystem, and fragmentation
- Prompt API currently ties to specific models per browser (e.g., Gemini Nano in Chrome, other models in other browsers).
- Developers worry prompts are highly model-specific and that the API lacks introspection to adapt behavior per browser, making testing harder than with APIs like WebGL.
- Links show mixed reactions from other browser vendors; some detailed, some dismissive.
Local vs cloud models and model quality
- Comparisons claim hosted models (e.g., Gemma via APIs) are faster and more capable than in-browser Gemini Nano.
- Some expect browsers/OSes to eventually ship multiple or better models; others find the prospect of AI baked into OSes/browsers dystopian.