SSDs have become fast, except in the cloud

Original Article ↗ Hacker News Discussion ↗

Cloud SSD storage often delivers far less throughput and higher latency than modern consumer NVMe drives, even on “storage-optimized” instances from major providers. Commenters trace this gap to architectural choices like network‑attached or heavily virtualized storage, conservative firmware and wear‑leveling, and designs that prioritize durability and live migration over raw speed. The thread broadens into a critique of cloud economics and abstractions, with many arguing that for I/O‑intensive or steady workloads, dedicated or colo hardware with local NVMe can be dramatically faster and cheaper than public cloud options.

Cloud vs. bare metal and smaller providers

  • Many argue IO‑heavy workloads are poorly served by big clouds: IOPS and throughput are expensive and throttled; dedicated servers or colo with NVMe are “way cheaper” for sustained use.
  • Others counter that TCO must include power, space, bandwidth, spares, and staff; for small teams, cloud’s operational offload can outweigh hardware savings.
  • Suggested alternatives to AWS include DigitalOcean, Hetzner, OVH, Scaleway, UpCloud (fast block storage), Entrywan, OCI, and various managed PaaS options (e.g., Supabase). Concerns raised about Hetzner’s identity‑verification freezes.
  • There’s active interest in hybrid setups: using cloud for elasticity and overflow, but running steady, IO‑heavy workloads on owned or rented metal.

Why cloud SSDs underperform

  • A recurring theme: much cloud “SSD” is network‑attached (EBS, PD‑SSD, Azure managed disks) with:
    • Higher latency than local NVMe.
    • Shared bandwidth across many tenants.
    • Extra layers (hypervisors, firmware, schedulers) that trade peak throughput for fairness, durability, and predictable latency.
  • Counterpoint: all major clouds also offer truly local/instance SSD (AWS instance store, GCP Local SSD, Azure temp/cached disks) that are PCIe‑attached but:
    • Ephemeral (data lost on stop/migration).
    • Still capped well below modern consumer NVMe speeds, likely due to virtualization and controller firmware.

Benchmarks and anecdotes

  • Multiple reports of consumer or small‑server NVMe (3–7 GB/s, ~1–1.5M IOPS, tens of µs latency) massively outperforming:
    • AWS storage‑optimized instances, which in practice deliver ~2–3 GB/s per device and ~500k 4k IOPS.
    • Azure Premium SSDs, which show 0.4–3 ms latency; Azure’s local SSD cache/temp disks can hit ~40 µs and dramatically speed databases.
  • Some find cloud CPUs also feel slower vs. similar on‑paper specs; virtualization and older generations in cheap SKUs are suspected.

Latency, architecture, and trade‑offs

  • For databases and index‑heavy workloads, random‑access latency and dependency chains matter more than raw bandwidth; networked storage hurts here.
  • Local SSD used as a cache (e.g., Rails “SSD cache”, Azure read‑caching) can be nearly as effective as RAM for many web workloads at far lower cost.
  • Debate over designing for “planet scale” in the cloud vs. starting with a single powerful box (SQLite/Postgres + lots of RAM/NVMe) and only adding distributed complexity when truly needed.

Economics, incentives, and trends

  • Several commenters claim clouds capture most of the gains from hardware advances (NVMe, newer CPUs) via pricing and throttling; performance per dollar improves slowly for customers.
  • There’s a perceived small but growing movement toward on‑prem/hybrid, especially for large, data‑intensive AI and analytics workloads.
  • Some speculate that cheap local SSDs/GPUs and privacy concerns may push more functionality back to thick clients, though mobile constraints remain a limiter.