Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

Original Article ↗ Hacker News Discussion ↗

Meta’s release of Llama 3.2 — a family of small text and multimodal models aimed at edge and mobile devices — is seen as a major step for open, locally runnable AI, with users reporting surprisingly strong performance from the new 1B and 3B models and competitive vision capabilities from the 11B and 90B variants. Commenters highlight how these models enable on-device use cases, function/tool calling, long-context workflows, and GPU-light experimentation via tools like Ollama and llama.cpp, while comparing them against alternatives such as Qwen, Gemini, and Mistral. Debate centers on Meta’s partial openness (notably the EU licensing restriction on multimodal models), questions about training data, bias and alignment, and the broader strategic impact on commercial AI providers and traditional web search.

Access & Licensing Constraints

  • Some users can’t access meta.ai from certain US territories; site reports “not available in your country.” Others suggest trying Hugging Face or llama.com instead.
  • Multimodal Llama 3.2 models are explicitly not licensed to individuals or companies domiciled in the EU, which several commenters link to EU AI/data regulations.
  • Debate whether EU exclusion is cost/benefit compliance choice vs. deliberate pressure on regulators.

Model Lineup & Capabilities

  • New text-only 1B and 3B models impress many: high coherence, good instruction following, and 128K context. 1B runs on low-end hardware (e.g., Raspberry Pi 5), 3B seen as superior to earlier small models (e.g., Gemma-2-2B, Phi-3.5-mini) for some tasks.
  • Skepticism about how much knowledge 3B parameters can store; users report good factual recall but weak reasoning (e.g., simple weight/decimal comparisons).
  • Multilingual: solid German, usable in some smaller languages, but confusions and code-mixing for Greek and others.

Vision & Multimodal Performance

  • 11B and 90B vision models: some find them “legit good” for OCR and screenshots/flowcharts; others say Qwen2-VL (7B/72B) and Molmo outperform them on visual tasks and handwriting.
  • Word-search and spatial puzzles remain hard; models find some left‑to‑right words but mostly fail on diagonals/complex patterns.
  • Some users report strong refusals and safety filters on 90B for practical image-to-HTML or similar tasks.

Local / Edge Deployment

  • Many run models locally via Ollama, llama.cpp, KoboldCPP, LM Studio, Jan, Open WebUI, etc.
  • 1B/3B are popular for on-device use; people report good performance on M1/M2 Macs, modest desktops, and possibly Android/Termux, with quantization.
  • Vision models aren’t yet fully integrated into all common local stacks (e.g., llama.cpp/Ollama support is “coming soon”).

Function / Tool Calling & JSON

  • Text-only and vision models support tool/function calling for text inputs; not yet for mixed text+image prompts.
  • Several note that constraining decoding to valid JSON can yield reliable function-calling without extra fine-tuning.

Bias, Alignment & Training Data

  • Thread contains extended debate over “bias” and alignment:
    • Some want transparency on curated alignment data and worry about political/religious skew and censorship.
    • Others argue “bias” mostly reflects training corpora and market focus (e.g., US/English-centric); full “de-biasing” is expensive and low-ROI.
  • Meta is praised for technical openness and model releases, but criticized for opaque training data and default user data opt-in.

Comparisons to Other Models & Services

  • Qwen2/Qwen2-VL, Pixtral, Gemma 2, Phi‑3.5, Molmo, GPT‑4o, Claude 3.5, Gemini, and Mistral are repeatedly used as baselines.
  • Consensus: Llama 3.2 small text models are very strong at their size; vision models are competitive but not clearly best-in-class.

Reliability, Hallucinations & Evaluation

  • Users report both good practical results and glaring hallucinations (e.g., inventing fictional histories of software frameworks).
  • A hallucination leaderboard entry claims ~4–5% hallucination rates for the new vision models.
  • Commenters distrust cherry-picked vendor benchmarks and want fresher, multi-metric leaderboards that include newer open models.