MicroLLM Lab – Try 7 tiny LLM's in the browser

Original Article ↗ Hacker News Discussion ↗

Tiny language models that run entirely in the browser are impressing developers with how fast and lightweight they’ve become, but their capabilities remain sharply limited compared to modern chat-style LLMs. Commenters praise the MicroLLM Lab demo for showcasing on-device inference over WebGPU and WASM, while also highlighting frequent hallucinations, poor reasoning, English-only behavior, and confusing UI design. The thread also surfaces technical challenges—especially Firefox and GPU compatibility—and points to broader efforts like Web Models API and similar projects aiming to standardize and expand local, privacy-preserving model execution on the web.

Overall impression

  • Many see the demo as a fun, eye‑opening proof of how far in‑browser, on‑device inference has come.
  • Others find the models “unusable” for assistant-style tasks, with outputs often nonsensical, repetitive, or looping.
  • Several comments compare the experience to the early GPT‑2 / “prompt engineering” era: interesting, but fragile and unreliable.

Capabilities and limitations of tiny models

  • Consensus: these are small language models (SLMs), not full assistants.
  • Suggested appropriate uses: sentiment analysis, text classification, entity extraction, simple completions.
  • For open‑ended chat, recipes, factual Q&A, or basic reasoning (e.g., 2+2, capitals, populations, card counts), users report frequent hallucinations and logical errors.
  • Some outputs are amusingly wrong (e.g., cats “not animals,” bizarre South Africa answers, absurd population numbers, strange recipes).
  • A few prompts get reasonable answers, showing that with careful prompting these models can sometimes behave coherently.
  • Several users conclude these examples are not yet practically useful as general assistants, and view the lab mainly as a capability showcase.

User experience and interface

  • Multiple users criticize the UI:
    • Text too small and dense.
    • Too much upfront, mostly ignorable, AI‑generated explanation before the interactive part.
    • Confusing footer and lack of clear examples of what SLMs are good at.
  • The author responds by:
    • Adding a large “tl;dr” and marking the longer text as optional.
    • Increasing font size and simplifying the footer.
  • Some argue the post succeeded despite the noisy layout, not because of it.

Technical implementation and performance

  • Runs models in the browser via WebGPU, with WebAssembly as a slower fallback.
  • Many report surprisingly fast token generation on modern hardware; some share specific tokens‑per‑second numbers.
  • Others hit issues:
    • WebGPU not available or buggy on Firefox/Linux (e.g., GPUShaderStage errors), requiring flags or fallback.
    • One report of macOS crashing (unclear cause).
    • Some users can’t load models or say “nothing works,” while others report everything working on mainstream browsers.
  • Suggestions include better feature detection, explicit warnings for unsupported setups, and serving models via CDN.

Related projects and broader implications

  • Several related in‑browser LLM / WebGPU demos and labs are shared, including single‑file chat apps, tiny “story” models, and other mini‑labs.
  • A proposed “Web Models API” standard is discussed, including questions about model registries and possibly P2P distribution.
  • Overall sentiment: running even small models directly in the browser, including in air‑gapped scenarios, is seen as powerful and promising despite current quality limits.