Why AI companies want you to be afraid of them
AI labs’ habit of warning that their own products could end humanity is framed as a strategic choice rather than pure altruism. Commenters argue that apocalyptic rhetoric builds hype, attracts investors, steers regulation in favor of large incumbents, and distracts from more immediate harms such as job loss, surveillance, spam, deepfakes, environmental costs, and unsafe deployment in critical systems. Others counter that many researchers genuinely fear long‑term risks, leaving society in a “damned if you do, damned if you don’t” bind over how seriously to take existential claims while still regulating today’s concrete abuses.
Fear-based marketing and hype
- Many see “too dangerous to release” claims (from GPT‑2 to current models) as deliberate fear-mongering to generate hype, funding, and a sense of inevitability.
- Others argue some early “danger” claims (e.g., spam, deepfakes, manipulation) were actually prescient given today’s Internet.
- Several commenters think apocalyptic talk sells FOMO to investors and enterprise buyers more than to end users.
- Some argue leaders genuinely fear x‑risk; others see this as mostly PR and brand-building.
Real vs speculative risks
- Near-term harms discussed: spam, deepfakes, porn, buggy software, surveillance, energy/water use, environmental impact, and degraded information quality.
- Economic insecurity (job loss, worse working conditions, “rust-belt treatment” for knowledge workers) is seen as a central, under-addressed fear.
- Military, policing, and autonomous weapons are alarming but feel more abstract to most people than losing their job.
- Opinions split on existential risk: some treat it as plausible and important to plan for; others see it as a distraction from present harms.
Labor, economics, and social stability
- Concern that mass unemployment or underemployment (e.g., 25% general, 50% youth) could destabilize democracies.
- AI is framed as a tool for cost-cutting and layoffs, exciting investors but terrifying workers.
- Suggestions include stronger safety nets, liability for AI-caused harms, and constraints on AI in high-stakes decisions.
Warfare, security, and “Mythos” zero‑days
- AI in warfare (targeting, drones) predates LLMs but is accelerating.
- Debate over specialized security models finding zero‑day vulnerabilities: some report strong empirical results; others doubt demos and view “too dangerous to release” as marketing.
- Even skeptics worry that powerful vuln-finding tools plus poor operational security could be dangerous.
Regulation, moats, and open source
- Many see fear narratives as a bid for regulatory capture: strict rules that incumbents can meet but which squeeze out open source and small players.
- Geopolitical framing (“China will win if we regulate”) is viewed as another lobbying tool.
Capabilities, limits, and non-determinism
- Repeated emphasis that LLMs are “just software,” yet highly unpredictable and non-deterministic.
- People warn against giving agents unsupervised access to production systems; early incidents (e.g., accidental DB deletion) are seen as previews.
- Proposed research direction: “cheap verifiers” or guardrails to make unreliable agents practically usable.