Anthropic's proposed AI safety bill only kicks in if 50 deaths or $1B in damages
Anthropic’s proposed U.S. AI safety bill would trigger stricter oversight only if a model is judged capable of causing at least 50 deaths or $1 billion in damage, prompting criticism that this effectively prices human life at around $20 million and leaves smaller-scale harms unaddressed. Commenters argue the thresholds are easily met by AI-enabled cyberattacks yet too high to protect individuals, consumers, and small businesses, and see the framework as tilted toward shielding large “frontier” labs while constraining open-source models. Others counter that some form of pre-release risk auditing is necessary, but worry this approach invites regulatory capture and concentrates power over AI deployment in a few well-capitalized firms and government agencies.
Thresholds: 50 Deaths / $1B Damages
- Many see the thresholds as arbitrary and ethically troubling, implying up to 49 deaths or sub‑$1B harm is “acceptable.”
- Some argue $1B is actually a low bar, especially for AI-assisted cyberattacks or large firms; others call it high enough to exclude harms to small entities.
- Questions arise about whether this applies per incident or cumulatively, and over what time period; the proposal’s mechanics are described as unclear or “nonsensical.”
Implied Value of Human Life
- Posters note the bill implicitly prices a life at ~$20M.
- Some compare this to real-world legal compensation, suggesting it’s not unusually high; others mock the idea of a “minimum quantity” of 50 lives.
Regulatory Capture & Impact on Open Models
- Strong concern that the bill mainly entrenches large labs and gated APIs by:
- Mandating review and insurance that only big players can afford.
- Making “Certified Safe AI” effectively inaccessible to small/open projects.
- Several say this is a deliberate attempt to suppress open-weight models under the banner of “safety.”
- Others counter that the bill is at least aimed at “frontier labs,” not small ones, though opponents read it as de facto banning powerful open models.
Liability, Law, and Enforcement
- Debate over whether new laws are needed versus applying existing tort, criminal, and RICO frameworks.
- Some say companies should be held liable like any software vendor, especially when they deploy known-dangerous systems.
- There’s extensive Mens Rea discussion: negligence vs intent, repeated risky behavior, and whether executives or specific engineers should face charges.
Comparison to Other Technologies
- Analogies to cars, planes, guns, encryption, and hacking tools:
- One view: regulate misuse and defective design, not capabilities per se.
- Another: if AI is framed as more dangerous than guns, significant ex ante regulation is inevitable.
Effectiveness and “Genie Out of the Bottle”
- Skeptics argue that powerful open models and foreign labs make strict control impossible; at best, the bill buys a few months’ delay.
- Others see any slowdown of frontier deployment as still valuable time to develop further safeguards.