Moving from AWS to Bare-Metal saved us $230k per year

Original Article ↗ Hacker News Discussion ↗

Claims that moving an uptime-monitoring service from AWS to bare‑metal servers saved about $230,000 per year sparked a broader examination of cloud versus self‑hosted infrastructure economics. Commenters argue over whether such savings persist once migration effort, ongoing ops work, and redundancy are fully costed, noting that the company’s previous AWS setup appeared overprovisioned and poorly optimized. Many conclude that predictable, steady workloads can be far cheaper on colo or dedicated hardware, while cloud platforms still win for fast iteration, managed services, and spiky or uncertain demand.

Cost & Savings Debate

  • Many agree bare metal / colo is inherently cheaper than AWS for steady workloads; some cite cloud often costing ~2–2.5× on-prem.
  • Several argue the reported $230k/year savings is overstated because:
    • They were on on‑demand EC2 with no Reserved Instances or Savings Plans (could save ~35%+).
    • Spot instances and Graviton (m7g) could have cut compute cost far further for suitable workloads.
  • Counterpoint: even after reasonable AWS optimizations, commenters still expect meaningful savings from bare metal, especially on bandwidth and storage.
  • Some note the savings equate roughly to 1 mid-level US engineer per year; outside the US that could fund several engineers.

Hidden & Operational Costs

  • Skeptics emphasize:
    • Migration effort and risk are not priced in.
    • Ongoing hardware work: failures, capacity planning, procurement lead times, remote hands.
    • Complexity of HA storage (Ceph/Longhorn), networking (Cilium/eBPF), Kubernetes control plane, and serious database setups with tested backups and PITR.
  • Others respond that:
    • AWS estates also require specialists (DevOps/SRE, finops, security, YAML/IaC work, vendor management).
    • For small fleets (1–2 racks), ops overhead can be modest, especially with colocation and automation (PXE, Harvester, Proxmox, etc.).

Reliability, HA, and Architecture

  • Strong criticism that a single-rack / single-DC setup is less reliable than AWS multi‑AZ; “uptime monitoring on one rack” is seen as ironic.
  • Supporters note they maintain an AWS failover cluster they can spin up quickly, but others point out DNS, data sync, and failover drills are nontrivial.
  • Debate over whether multi‑region/multi‑cloud in AWS is “Cloud 101” and whether on‑prem can match that resilience realistically.

Cloud vs Bare Metal: When Each Fits

  • Pro‑cloud arguments:
    • Fast experimentation with managed services (Fargate, S3, Aurora, etc.).
    • Avoids dealing with physical hardware, facilities, generators, HVAC.
    • Enables rapid scaling and very high growth without large upfront capex.
  • Pro‑bare‑metal/colo arguments:
    • Huge savings on bandwidth and predictable compute.
    • Good fit for batch/ML, long‑running workloads, and mature stable products.
    • Potentially better performance (direct NVMe, no noisy neighbors) and less vendor lock-in if cloud‑specific services are avoided.

Meta & Skepticism About This Case

  • Some think their original AWS footprint (28‑node K8s for an uptime app) was obviously overbuilt.
  • Others say the blog is light on key details (data volume, HA design, RTO/RPO, colo terms) making it hard to judge the decision fully.
  • Several note that cloud was still useful to validate the business quickly; moving off later is seen as a reasonable “phase two” optimization.