The unbearable cheapness of open weight models

Original Article ↗ Hacker News Discussion ↗

Open‑weight AI models like DeepSeek are driving inference costs so low that many routine coding and knowledge‑work tasks can now be done locally or via cheap APIs, raising doubts about how frontier labs such as OpenAI and Anthropic will sustain premium pricing and huge training budgets. Commenters debate whether these firms can pivot to high‑value niches (e.g., specialized scientific or security applications, enterprise integrations, or regulatory capture) or whether AI will largely resemble a commoditized cloud VM market where brand and slight quality edges matter only in a few high‑stakes sectors. Alongside pricing, people highlight unresolved issues around data ownership, regulation, and geopolitical control over models, as well as the technical limits of scaling and synthetic data in the race toward more advanced systems.

Economics of Open-Weight vs Closed Models

  • Many argue inference is already “dirt cheap” and getting cheaper; training and RLHF/RLAIF are the real cost drivers.
  • Open-weight providers (e.g., DeepSeek, Xiaomi) are seen as lacking brand/technical moat, so they must price at commodity levels.
  • Others question sustainability of ultra-low prices, citing GPU depreciation and claims that some providers’ margins look unrealistic.
  • There’s disagreement on whether frontier labs’ inference margins are ~50% (constrained by costs) or >90% (with cash burn mostly from training and org bloat).
  • Subscriptions vs API: API tokens look expensive; flat-rate plans likely subsidized, but some think they’re still profitable due to unused quota.

Business Models and Moats for Frontier Labs

  • Concern that open-weight models will commoditize “95% of tasks” (coding, research, cowork-style work), dragging down prices.
  • Proposed survival paths:
    • Push frontier capabilities (AGI, advanced science, biotech, nuclear design) and take royalties.
    • Own the application layer (enterprise tools, vertical products) and sell “best in class, safe, compliant” to large orgs.
    • Regulatory capture: “certified/regulated models only,” federal procurement rules, copyright licenses as gatekeeping.
    • Temporarily buying up scarce hardware (RAM/GPUs) to delay competitors until IPO.
  • Skeptics doubt their app-design chops and note incumbents in verticals (law, finance, healthcare) can pair domain expertise with open weights.

Capabilities, AGI, and Data Constraints

  • Several see progress as gradual and “moat-less”: open models already nip at frontier labs, especially with good harnesses/agents.
  • Debate on whether we are compute-limited or data-limited:
    • One side: high-quality data is the bottleneck; synthetic data helps only conditionally and can cause “model collapse” if overused.
    • Other side: synthetic data plus partner datasets (biotech, engineering, etc.) can extend scaling.
  • Recursive self-improvement / “singularity” is viewed as overhyped; scaling compute dominates.

Practical Use of Open-Weight and Local Models

  • Multiple commenters report doing substantial work (coding agents, translation, niche CVE discovery, custom scripting) cheaply with local 12–35B models plus caching.
  • KV/prompt caching and agent loops can push effective token cost near zero for many workloads.
  • For many business tasks, human-level or even sub-human intelligence suffices; cost, control, and predictability matter more than absolute quality.
  • Some still find frontier coding models clearly superior for complex tasks.

Trust, Control, and Regulation

  • Strong preference from some for open weights due to: data control, auditability, avoiding arbitrary bans, and compliance predictability.
  • Others note local hosting can be “cheaper until you factor in security and liability,” which may push regulated industries toward big closed providers.
  • Expectation from several that governments (especially the US) will ban or constrain foreign/open models, tying favored domestic labs to state interests.

Space and Exotic Infrastructure (Data Centers in Orbit)

  • Long subthread on space-based data centers:
    • Optimists note SpaceX’s track record (reusable boosters, Starlink, on-orbit power and AI chips) and see orbital compute as engineering-feasible.
    • Skeptics stress economics and scaling: radiation, cooling, maintenance, launch costs, and the need for massive mass-to-orbit make large-scale deployment likely uneconomic for decades, if ever.
    • General consensus: technically possible in niche form; commercial viability at scale is unclear.

Open Models, Market Structure, and Europe

  • Some argue open weights themselves act as a moat: any new commercial model must beat strong free/cheap baselines, making new entrants (e.g., from Europe) harder to justify.
  • Others point to existing European efforts and long-run benefits of open infrastructure.

Conceptual Clarifications: “Open Weights”

  • “Open weights” = you can download the full parameter tensors and associated files (tokenizer, architecture metadata, configs) and run inference yourself.
  • Fully “open models” also publish training code and datasets; these are less common.
  • Weights are just large numeric matrices governing the model’s behavior; open weights enable fine-tuning and local deployment without revealing original training data.