Cutting down AWS cost by $150k per year simply by shutting things off

Original Article ↗ Hacker News Discussion ↗

Cloud cost optimization is emerging as a major concern as engineers report cutting tens or hundreds of thousands of dollars simply by auditing infrastructure and shutting down unused or overprovisioned AWS resources. Contributors describe how lack of cost visibility, misaligned incentives, and management culture lead to years of waste—while tools, FinOps practices, tagging, and automation can systematically reduce spend. The thread also touches on broader trade-offs, from serverless vs. bare metal and vendor lock-in to whether employees who generate large savings should share directly in the financial gains.

Observed Cost Savings & Low-Hanging Fruit

  • Many commenters report huge, “embarrassingly easy” savings by turning off unused resources or right‑sizing:
    • Cutting single accounts from hundreds of thousands per month to a fraction.
    • Finding abandoned S3 pipelines or test databases costing hundreds of thousands to millions per year.
    • Discovering CI artifacts like node_modules being shipped to S3 and downloaded by clients, adding six‑figure yearly transfer costs.
  • Common culprits: forgotten test clusters, logs with infinite retention, oversized dev/test infra, NAT gateways, and old VMs/EBS volumes no one owns.

Incentives, Bonuses, and Perverse Effects

  • Repeated theme: engineers who save big money rarely see proportional bonuses; often they get more meetings or mild recognition at best.
  • Some argue this discourages cost optimization (“why bother?”), others say “it’s just doing your job.”
  • Paying a percentage of savings is attractive but seen as prone to abuse (“cobra farming”: inflate then “optimize”).

Visibility, FinOps, and Billing Access

  • “You can’t optimize what you can’t see”: strong emphasis on dashboards, tagging, weekly cost reports, and formal FinOps practices.
  • Several complain devs are blocked from billing consoles, making it impossible to notice accidental cost spikes.
  • When costs and private discounts are kept secret by management, engineers stop trying to optimize.

Dev/Stage Environments and Automation

  • Non‑production environments can cost more than production if left always‑on or full of test data.
  • Popular strategies: auto‑shutdown schedules, opt‑out policies backed by tags, bots that enforce lifetimes, and tools that nuke unused resources.
  • Some teams go further with “blue‑green” EKS clusters or serverless/Knative setups that scale to zero.

Cloud Platform Choices and Architectures

  • Debate over moving from major clouds to cheaper providers like Hetzner or bare metal:
    • Pro: potential 10x savings if you don’t need managed services or strict SLAs.
    • Con: you re‑implement managed services yourself and pay in engineering time and reliability risk.
  • Serverless can massively cut costs for spiky or dev workloads, but often requires substantial re‑architecture.

Organizational Culture and Priorities

  • Many see waste as a product of culture: rushed feature delivery, lack of production readiness reviews, and cloud commitments that blunt cost‑cutting incentives.
  • For large enterprises, even multi‑million‑dollar savings can be treated as a rounding error; for small companies, it’s existential.