From the creator of Redis; run LLM locally with ds4

Original Article ↗ Hacker News Discussion ↗

A new C-based LLM inference engine, ds4 (“DwarfStar”), targets a narrow set of large MoE models like DeepSeek V4 Flash and Qwen 3.8 on high-end consumer hardware, especially 96–128 GB Apple Silicon Macs and x090 GPUs. Commenters highlight its strong defaults, aggressive performance optimizations, long-context support, and SSD streaming as advantages over more generic runners like llama.cpp and Ollama, while others point to ecosystem tools such as ds4go and smaller projects inspired by it. There is also debate over its flashy marketing site, the choice of C over Rust and heavy AI-generated code, and whether this approach signals a broader shift toward powerful, fully local “frontier-like” models for serious development work.

Project focus and design philosophy

  • ds4 is a small, native inference engine focused on a few large MoE models (DeepSeek V4/4.1 Flash, GLM 5.x, Qwen3.8 Flash Next), optimized deeply for them instead of trying to support everything.
  • Emphasis is on “frontier-ish” local models, long context (up to ~1M with tricks), and good tool/agent support rather than broad model coverage.
  • The project is explicitly framed as a “working template” for big use cases that others can fork and specialize.

Hardware targets and performance

  • Primary focus is high-end consumer hardware: Apple Silicon with 96–128GB RAM, DGX Spark, and newer high-end GPUs.
  • SSD streaming is used to fit very large models or run on 64GB/96GB machines, but some say 128GB is realistically needed for Qwen3.8 Flash Next.
  • Users report strong performance on M5 Max 128GB (fast tokens, long context) and enthusiasm about upcoming M5U systems.
  • Separate effort: a Gemma-4 MoE engine for Intel Xe-LP iGPUs with ~22 tokens/s reported on an Intel Ultra 7 laptop.

Comparison to other runners

  • Some see ds4 as a vibe-coded clone of llama.cpp; others argue its value is vetted defaults and narrow, highly tuned support rather than a huge configuration matrix.
  • It’s contrasted with general tools (llama.cpp, Ollama) that can match performance but often require intensive tuning.
  • Discussion around automatic hardware-specific benchmarking as a better approach than manual config.

Ecosystem and tooling

  • A fork exposes ds4 as shared libraries with Go bindings, plus a TUI and tools (workspace, scratchpad, OpenAI-like API, MCP/tool-calling setups).
  • Another project targets older Intel GPUs; commenters wish for first-class Intel and better AMD support. There is some experimental ROCm activity.

Language choice and AI-assisted development

  • The core is written in C. Reasons cited: author familiarity, simplicity, performance, and current LLMs being better at generating C than Rust.
  • Others argue Rust is well-suited for high-performance, low-level, multithreaded code and can outperform C in some domains.
  • Debate over whether Rust’s stricter semantics help or hinder AI-assisted workflows.

Critiques, skepticism, and UX

  • Some criticize the marketing-heavy landing page as “slop” and prefer the minimal GitHub README.
  • Mixed views on the author’s past work (e.g., Redis and distributed systems debates) lead a few to treat the association as a warning.
  • Despite criticism, many are enthusiastic about ds4 as a sign that serious engineers are pushing local inference forward.