Best LLM for every budget, updated daily

Original Article ↗ Hacker News Discussion ↗

A web tool that charts “best” large language models by cost is drawing interest but also criticism for outdated data and using simple cost-per-million-tokens metrics. Commenters argue that cost per completed task, subscription limits, and cache pricing matter far more in practice, and many point instead to Artificial Analysis or their own benchmarks to choose models. The thread also highlights a growing split between users who rely on cheap API access to Chinese or mid-tier models like DeepSeek and Qwen, and those investing in local hardware to run open models, all trying to balance capability, latency, privacy, and real-world cost.

Overall reaction to the site

  • People appreciate a clean, visual way to compare models, but many find the data outdated or misleading.
  • Coding and math tabs lag behind current releases (e.g., still showing GPT‑5.2 as math frontier, missing or partially missing Opus 5.5 and newer DeepSeek models).
  • Several say it feels like a thin wrapper over Artificial Analysis (AA) with fewer features and less nuance.

Critique of cost metrics

  • Main complaint: cost per 1M tokens is “naive”.
  • Users argue what matters is:
    • Cost per task, not per token.
    • How many tokens a model needs to reach a given quality (verbosity, “thinking” length).
    • Cache pricing, which can dominate costs for coding and agentic workflows.
  • AA’s “cost per task” and token-use charts are frequently cited as superior; some note striking differences (e.g., some frontier models using 4–5x more tokens per task than peers).

Subscriptions vs per-token APIs

  • Many individuals pay out of pocket:
    • Some spend only a few dollars/month via DeepSeek/OpenRouter and find it cheaper than subscriptions.
    • Others burn through $100–$200/month subscriptions quickly and then fall back to cheap APIs.
  • Tradeoff:
    • Subscriptions are often far better unit value but have opaque or bursty rate limits.
    • APIs cost more per token but offer reliability, automation, and multi-vendor flexibility.
  • People want tools that factor in subscription quotas and help optimize “usage limit efficiency.”

Local vs cloud models

  • Some strongly prefer local models and don’t understand “throwing money” at SaaS.
  • Others respond that:
    • High-end models (Astra/Fable/Opus-level) are not locally feasible.
    • Hardware plus electricity can make local inference comparable or more expensive than very cheap cloud models like DeepSeek Flash.
  • Multiple users share concrete local setups (Macs with 32–128GB RAM; GPUs running Qwen/Gemma/Bonsai), using them for background coding, planning, and “chore” tasks.

Benchmarks, nuance, and desired features

  • Skepticism about any single-number ranking; error bars and aggregation make results noisy.
  • Calls for:
    • Multi-axis metrics (wordiness, persistence, “thinking ahead,” looping behavior).
    • Per-user, task-specific eval suites.
    • Tools that incorporate hardware, VRAM, local vs cloud, and actual bundle deals.
  • Some still find this site’s presentation more immediately actionable than AA, even if the underlying methodology is weaker.