Show HN: Smart model routing directly in Claude, Codex and Cursor

Original Article ↗ Hacker News Discussion ↗

Cost, Caching, and Token Savings

  • Many commenters worry that multi-model routing increases cache misses and thus cost, especially with short cache TTLs and long coding sessions.
  • The router claims to be “cache-aware”: once a model is in use, switching requires a higher threshold so that cache loss is offset by cost or quality gains.
  • The authors report ~40% internal cost savings, arguing that:
    • Small models can fully handle many requests.
    • Using a small model for part of a task plus one cache miss is often cheaper than using a frontier model end‑to‑end.
    • Subagents with fresh contexts are natural places to route to cheaper models.
  • Some argue that beyond one planner + one executor model, extra models mostly add cache penalties and complexity.

Routing Logic, Reliability, and Use Cases

  • The router is trained on real agent traces; reward signals reflect whether a chosen model completes tasks successfully.
  • It uses embeddings of prompt + context to cluster similar tasks and adapt over time; larger models can “rescue” smaller ones that get stuck.
  • Concerns: small models may loop or fail more; ambiguous tasks are hard to route correctly; many users tune prompts to specific models, so automatic routing may misalign with their expectations.
  • Several see this as more suited to average users or agentic coding setups than to highly controlled, pre‑evaluated production flows.

Integration, Subscriptions, and Privacy

  • It can use existing Claude/Codex subscriptions (subsidized usage) and fall back to API billing; also integrates with tools like OpenCode and can route to Gemini.
  • Some worry about data privacy when using any proxy. The project can be self‑hosted; many customers reportedly still choose the hosted option.

Comparisons and Alternatives

  • Compared to other routers (e.g., vLLM Semantic Router, Sakana Fugu, Cursor “auto”), this one emphasizes agentic coding and cache awareness.
  • Some note that coding harnesses and IDEs already do their own model-aware routing and question whether a proxy-layer router breaks that control loop.

Evidence and Open Questions

  • Commenters request public benchmarks (e.g., coding benches, RouterArena) and details on switching frequency.
  • The maintainers say internal metrics show unchanged code quality and higher velocity, but external evals are not yet linked.