IBM CEO says open weight models are now good and cheap on-prem

Original Article ↗ Hacker News Discussion ↗

IBM’s claim that open‑weight AI models are now powerful and cheap enough to run on‑premises has reignited debate over whether enterprises should self‑host AI or rely on “frontier” cloud models from OpenAI, Anthropic, and others. Commenters weigh the trade‑offs between expensive GPUs and operational complexity on one side, versus subsidized per‑token pricing, vendor lock‑in, censorship, and data‑privacy risks on the other. Many argue that rapidly improving open models are already “good enough” for most workloads and that cost, control, and long‑term pricing uncertainty will push more companies—and some power users—toward local or self‑hosted deployments.

IBM’s Role and Motives

  • Many see the CEO’s claim as “talking his book”: IBM sells on‑prem hardware and consulting, so praising open-weight on-prem AI aligns with its interests.
  • Some argue IBM could challenge NVIDIA given its hardware history, capital, and chip expertise, but others say it’s no longer the IBM of the PC era and is hampered by dysfunctional management and rent-extracting culture.
  • A few note IBM’s Granite models and Power/Z hardware are technically impressive but expensive and niche.

Quality of Open-Weight Models

  • Multiple commenters say modern open-weight models (DeepSeek v4.1 Flash, GLM 5.x, Qwen 3.8, Kimi K3.1) are “good enough” for serious coding, long-context work, and agentic workflows, sometimes close to or at “Opus 4.x level.”
  • For many human‑in‑the‑loop tasks, slight quality gaps vs. top frontier models are acceptable.

Economics: On-Prem vs Cloud

  • Self-hosters report substantial rigs (e.g., ~$25k, multi‑GPU, kilowatts of power) that let them avoid paid APIs entirely, but concede it’s not cheaper than current subsidized subscriptions.
  • Others argue cloud/API is more economical for most users: e.g., €100–200/month LLM plans or ~€200/month GPU servers, versus owning underutilized racks.
  • DeepSeek is cited as targeting hardware payback in ~10 months; at similar utilization, there’s little arbitrage in self‑hosting unless sustained API spend is high. Utilization and inference tuning are seen as the hard parts.

Privacy, Control, and Censorship

  • Strong theme: on‑prem/open-weight gives privacy, avoids sharing sensitive or NDA‑bound data, and removes provider filters, refusals, and potential reporting to authorities.
  • Local setups also avoid future price hikes, throttling, bans, or policy shifts.

Hardware, GPUs, and Ecosystem Moats

  • CUDA and NVIDIA’s software stack are viewed as a major moat; IBM or others would need years of serious ecosystem work to compete.
  • Second‑hand GPUs are expensive and scarce; RAM and VRAM are seen as the real bottlenecks. Some predict eventual sharp demand drops but not soon.

Enterprise Adoption and Frontier AI Outlook

  • Enterprises struggle to integrate AI into workflows and lack kernel/network/inference engineers, which IBM presents as a consulting opportunity.
  • Several foresee a “good enough” plateau: most work done on cheaper/open models, with frontier SOTA reserved for niche cases, challenging the economics of ever-larger frontier training runs.