Alternative clouds are booming as companies seek cheaper access to GPUs

Original Article ↗ Hacker News Discussion ↗

GPU supply and Nvidia’s incentives

  • Commenters argue Nvidia deliberately allocates GPUs to “alt clouds” rather than only hyperscalers to avoid dependence on a few giants that are building their own chips.
  • Diversified customers reduce monopsony risk and preserve demand if major clouds shift to in‑house silicon (TPU, Graviton, Azure chips).
  • Some note close relationships between Nvidia and certain alternative providers, implying not all are at arm’s length.

AMD vs. Nvidia ecosystem

  • Multiple posts say AMD’s MI250/MI300 hardware is competitive, but lack of easy, cheap cloud access and weaker tooling keeps the ecosystem Nvidia‑centric.
  • CUDA and Nvidia’s long-term software investment are seen as the real moat; AMD historically focused on gaming and lacked capital for multiple “moonshots,” but is now course‑correcting.
  • Skepticism remains about AMD’s ability to match Nvidia’s drivers and software stack.

Cloud pricing, margins, and “confusing bills”

  • Many criticize big-cloud pricing (CPU, SSD/SAN, egress, EFS, etc.) as opaque and expensive, with surprise bills common.
  • Some see AWS/GCP/Azure as a ZIRP-era phenomenon now being unwound; others counter that cloud usage and revenues are still growing and that costs can be managed with expertise.
  • There is debate whether “cloud vs. on‑prem” savings hold once you include staff and HA needs; networking, redundancy, and compliance are cited as major hidden costs.

Alternative GPU clouds: promise and caveats

  • Alt providers often offer significantly cheaper GPU hours than hyperscalers and fewer hoops to get quota.
  • Some services are praised for ease of use (simple signup, fast access, autoscaling to zero).
  • Others are criticized as “predatory” when capacity is scarce or tied to long-term contracts; defenders say this reflects marketplace dynamics and high demand.
  • Specific price examples (A100/H100) spark debate over whether offerings are subsidized by VC versus sustainable cross‑subsidy across SKUs.

Lock-in, tooling, and multi-cloud

  • Pure GPU IaaS is seen as less sticky than full AWS-style platforms; lock‑in mostly comes from data gravity and proprietary software layers.
  • Some inference platforms add abstractions that ease use but increase lock‑in, which turns off teams already invested in Triton or custom stacks.
  • New tools (e.g., multi-cloud schedulers, “single consoles” over many GPU clouds) aim to route workloads to the cheapest/available GPUs and mitigate lock‑in.

On‑prem, colo, and dedicated servers

  • Numerous commenters report large savings using dedicated servers/colo (Hetzner, OVH, others) versus big cloud, especially for bandwidth-heavy or steady workloads.
  • Others emphasize operational complexity, HA, and staffing as reasons many companies still prefer managed clouds, at least early on.

Macro outlook: boom or bubble

  • Some predict current GPU capex and alt-cloud boom will “crash big time” as AI adoption metrics disappoint and costs fall.
  • Others argue demand for powerful compute is structurally durable, even if today’s business models evolve.