MacBook M5 Pro and Qwen3.5 = Local AI Security System

Original Article ↗ Hacker News Discussion ↗

Benchmark & Workload

  • Thread centers on a benchmark of a fully local home security “agent” using Qwen3.5 models on an M5 Pro MacBook.
  • Qwen3.5-9B reportedly gets ~94% on a 96-test home-security suite, close to a GPT-5.4 cloud model, at ~25 tok/s and sub‑1s first token.
  • A 35B MoE variant is said to have faster first-token latency than tested OpenAI endpoints.
  • Some readers find the benchmark page flashy and unclear, ask for a direct link to the test suite, and question methodology (e.g., temp settings, simplistic tasks, narrow scope).

Hardware Choices & Cost Debate

  • One camp claims “barrier to entry” for usable local agents is ~$2,500 (e.g., high‑end Ryzen AI systems, M5, large RAM/context).
  • Others argue entry is closer to $400–$500 with used RTX 3060–class GPUs or a 16–32 GB Mac mini, or even <$100 with tiny models on Pi‑like boards.
  • Disagreement over what “usable” means: small models vs. large context windows (~50k–100k tokens) for coding/agents.
  • Debate on Apple Silicon vs Jetson/GPUs: memory capacity vs. prefill speed; scenario-dependent performance.

Use Cases & System Design

  • Vision + LLM “orchestration” stack: small VLM (e.g., LFM 450M) for perception and 9B Qwen for logic.
  • Goals: context-aware alerts (ignore family cars, expected visitors), richer instructions (“let in electrician, alert if they do more than X”).
  • Some question if an LLM is necessary at all vs. fixed logic, and highlight prompt-injection risks (“forget previous instructions” tricks).

Comparison to Existing Security Tools

  • Questions on how this complements or replaces systems like Frigate; consensus that it’s more of an AI layer on top of NVR-style motion/event recording.
  • Suggestions to use Coral TPU or Intel N100 + OpenVINO for efficient inference in existing setups.
  • Interest in integration with ONVIF/RTSP, UniFi Protect (including RTSPS quirks), and Home Assistant; maintainers promise fixes and open-source bridges.

Local vs Cloud & Privacy

  • Many emphasize privacy, control, and independence from cloud availability/pricing as main drivers for local AI, more than latency.
  • Others note that cloud models are still faster and more accurate; Qwen3.5 is seen as strong but significantly behind top proprietary models.

Skepticism, Security, and Productization

  • Concerns about overhyped marketing language and AI-written commentary.
  • Prompt-injection tests in the benchmark are criticized as weak.
  • Reliability for “serious” security use is questioned; some see this as early/demo-stage.
  • A major missing piece for a commercial security product is compliance (e.g., alarm certificates for insurance).

Long-Term Vision: Home AI Appliance

  • Several comments imagine a future “AI server” as a standard home appliance: local, long-lived, family-specific assistant.
  • Others doubt this due to hardware obsolescence, maintenance, cost, and the likelihood that most people will keep using cloud services instead.