Claude Code wiped our production database with a Terraform command

Original Article ↗ Hacker News Discussion ↗

Responsibility for the outage

  • Most commenters say the human operator is fully responsible: they ran destructive commands, ignored warnings, had poor or no backups, and let an unsupervised tool access prod.
  • Several note this could have happened without AI; blaming the agent is compared to “blaming the intern” or “the dog ate my homework.”
  • Some emphasize that the post itself is framed as a “here’s what I did wrong and fixed” lesson, not an anti-AI rant, and criticize others for culture-war reactions.

Backups and recovery

  • Strong consensus: the real root cause is inadequate backups and poor recovery planning.
  • Recommendations: off-account / separate-account, append-only backups; deletion protection on critical resources; RPO/RTO planning; avoid backups being in the same blast radius as prod.
  • Some note that provider-level backups can sometimes save you, but should not be relied upon.

Terraform and infra practices

  • Many argue Terraform is powerful but a “footgun” when misused.
  • Best practices discussed:
    • Always inspect terraform plan before apply; never let agents (or CI) run apply unsupervised on prod.
    • Avoid terraform destroy on production; forward-evolve infra instead.
    • Use remote state (e.g., S3) and never local state files for prod.
    • Keep snapshots/DB backups defined and managed independently of primary infra state.

AI agents in production

  • Split views:
    • Some say AI agents will inevitably manage prod, potentially faster and better than humans, so guardrails must mature.
    • Others insist AI (and even most devs) should not have direct destructive prod access; only tightly controlled pipelines should.
  • It’s noted that the agent reportedly tried to warn about risks, and the user overrode it; still, critics say a “senior engineer–level” tool should push back harder or refuse.

Security, governance, and culture

  • Recurrent themes: principle of least privilege, no local direct access to prod, and human approval for destructive actions.
  • Several see this as an example of “vibecoding” / “vibeadministration” and influencer-style clout chasing, rather than professional engineering discipline.