Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
A 14MB “micro” language model called Needle2 aims to run locally on phones, wearables, microcontrollers, and smart home devices, turning natural language into structured tool calls (like locking doors or setting thermostats) without relying on the cloud. Commenters see strong potential for voice-controlled home automation and embedded systems, but repeated misinterpretations and odd responses highlight the limits of such a tiny model, the importance of confidence scores and calibration, and the likely need for fine‑tuning on narrow tasks. Overall, it is viewed as an intriguing step toward on-device AI, trading raw intelligence for low resource use and privacy.
Overall impressions
- Many commenters find a 14MB, on-device, tool-calling model impressive, especially running via WASM and on microcontrollers.
- Others feel the demo is underwhelming, with behavior too “dumb” for end-user expectations, but still see the concept as promising for niche, embedded uses.
Capabilities, limitations & “intelligence”
- Model is designed for tool calling, device control, and structured extraction, not general conversation or rich world knowledge.
- At this size, reasoning and understanding are clearly limited; it often misinterprets natural language (e.g., “warmer” vs. “cooler,” “dark” vs. lights off).
- Several tests show odd or inverted actions and misclassification of sentiment or context.
Confidence scores & out-of-distribution handling
- The model outputs a confidence score; authors suggest thresholding (≈60%) and escalating low-confidence cases to a larger/cloud model.
- Commenters stress that per-query scores are not the same as calibrated benchmarks and want reliability statistics over test sets.
- False positives and out-of-distribution detection (e.g., random words triggering door locks) are seen as critical usability issues.
Tool design, fine-tuning & robustness
- Performance heavily depends on clear, descriptive tool schemas. Small wording changes in tool descriptions or prompts can flip behavior from failure to success.
- The package ships with pipelines for data synthesis, augmentation, and fine-tuning on user workloads; several commenters note that a model this small essentially requires such tuning.
- There is interest in using it as a router or planner of tool-call DAGs, but this appears non-trivial without extra training.
Architecture, size, and tradeoffs
- Model is trained from scratch with custom architecture and quantization; 2-bit quantization chosen to support very constrained devices.
- Discussion highlights tradeoffs: speed and tiny footprint vs. “smarts.” Some question whether slightly larger binaries (e.g., 28MB+) would yield more useful behavior while still fitting on cheap hardware.
Use cases & integrations
- Strong interest in pairing it with speech (e.g., Whisper) for smart home, wearables, hearing aids, robotics planning layers, and Home Assistant integration.
- Some see it as a middle layer: cheap local NLP → structured tool calls → optional escalation to a frontier model.