Kubernetes on Hetzner: cutting my infra bill by 75%

Original Article ↗ Hacker News Discussion ↗

Running Kubernetes on low-cost providers like Hetzner can slash infrastructure bills by 70–80% compared with managed services on AWS or DigitalOcean, especially for compute- and bandwidth-heavy workloads. Commenters highlight trade-offs: weaker managed offerings, slower networked block storage for databases, more operational complexity (cluster upgrades, storage, networking), and occasionally rougher reliability and abuse handling. Many argue the savings justify investing in in-house ops expertise or third-party platforms built on Cluster API or Talos, while others contend that for smaller teams and less predictable workloads, managed cloud Kubernetes and simpler architectures remain safer despite the higher price.

Cost vs Operational Trade‑offs

  • Multiple commenters report infra bills on Hetzner being ~20–25% of AWS for equivalent capacity, especially when bandwidth or storage dominate costs.
  • Others emphasize TCO: self‑managing Kubernetes + storage (Ceph, etc.) can become a full‑time DevOps job and may wipe out savings, especially after outages.
  • Debate over when it becomes cheaper: some argue once cloud spend hits roughly mid–five figures per month and you have at least a couple of strong infra engineers, Hetzner/bare metal wins; others say even short outages can negate savings.

Storage and Databases on Hetzner

  • Strong consensus that Hetzner cloud volumes are too slow for serious production databases; high IOWAIT and low IOPS are common.
  • Suggested mitigations:
    • Use bare‑metal nodes with local NVMe (often RAID10).
    • Run DBs outside K8s on metal, or use K8s with local NVMe and node pinning.
  • Ceph (rook‑ceph) is seen as powerful but complex and often poor value at small scale; some prefer simpler NFS or block‑replication setups.

Cluster Provisioning & Tooling

  • Popular tooling mentioned: terraform‑hcloud‑kube‑hetzner, Cluster‑API + Hetzner provider, Talos + Omni, k3s, and various operators (DB, MinIO, load balancer).
  • Some vendors offer “managed Kubernetes on Hetzner” layers to provide self‑healing and one‑click upgrades while still benefiting from low prices.

Hybrid / Multi‑Environment Clusters & Networking

  • Several people explore clusters spanning on‑prem + cloud or multiple providers.
  • Techniques: WireGuard overlays, Tailscale operator, Cilium, Nebula, Netmaker, BGP on Hetzner vSwitch, etc.
  • Skeptics warn that extra hops, asymmetric routing, and “internet‑quality” links can wreck performance during peak load; others counter that with good design (edge caching, peering DCs) it can work.

Reliability, Support, and Abuse Handling

  • Experiences with Hetzner support range from “outstanding, very direct and technical” to “they null‑routed us on launch day and took days to fix.”
  • Reports of vSwitch resets, false‑positive abuse triggers, and fair‑use limits on “unlimited” 1 Gbit traffic.
  • Some see this as acceptable trade‑off for price; others prefer AWS/major clouds for more predictable support and fewer surprise interventions.

Kubernetes Complexity & Alternatives

  • Multiple voices question using Kubernetes at small scale, calling it overkill compared to simpler schedulers (e.g., Nomad) or even basic VMs/compose.
  • Counter‑arguments: even single‑server k3s can pay off where cloud is expensive; K8s APIs (Ingress, Services, PVCs, CRDs) and ecosystem (operators, Helm) solve many hard problems cleanly.
  • General agreement: K8s adds significant complexity; managed control planes or expert help are often worthwhile.

Hetzner vs Other Providers and Environment

  • Hetzner is consistently seen as far cheaper than DigitalOcean, OVH, and orders of magnitude cheaper than AWS egress.
  • Some worry about IP reputation (blacklisting, email deliverability) typical of budget providers.
  • Sustainability briefly discussed: EU Hetzner DCs are said to use certified renewable energy; US locations are unclear. Some argue data‑center emissions are non‑trivial and should be considered; others see transport and other sectors as much higher‑leverage targets.