P(doom)
Fears that advanced AI could pose an existential risk are set against skepticism that current systems are anywhere near that powerful, with many pointing instead to more immediate harms like corporate concentration, misaligned incentives, and opaque training data. Commenters argue over whether slowing frontier labs would genuinely improve safety or simply entrench incumbents, and whether open‑weight models and “AI diversity” act as a defensive balance of power or dangerously proliferate capabilities. Underlying it all is distrust of tech billionaires and private firms to steward such technology responsibly, and concern that doomer rhetoric may be used to justify regulatory capture and tighter control over open models.
Who controls AI & trust in tech leaders
- Many commenters argue that current AI is already in the “wrong hands,” criticizing billionaire founders and “ruling class” dynamics (money → power → political and media influence).
- Elon Musk is seen as highly polarizing: some emphasize his role in accelerating EVs and space tech; others cite right‑authoritarian politics, Nazi/white‑supremacist signaling, personal scandals, and alleged involvement in harmful legal and AI practices.
- Some fear “god‑complex” leaders who believe they’re saving the world more than “run‑of‑the‑mill sociopathic nerds,” but others counter that material power and resources are what truly increase danger.
Open vs closed AI, MAD, and “AI diversity”
- One camp claims diverse, open(-weight) models create a mutual‑deterrence dynamic: many different AIs, controlled by many actors, can check or stop rogue systems.
- Critics argue that speed, compute imbalances, and the ease of attack vs. defense make this analogy to nuclear MAD weak.
- There is debate whether alignment must be in the weights vs. in prompts and system design.
- Others worry open models more strongly empower “doomsday humans” (e.g., cults, extremists) rather than AIs themselves.
P(doom) and why people still build frontier AI
- Several find it hard to reconcile people claiming 10–25% extinction risk with continuing to push capabilities.
- Explanations offered: belief they reduce risk vs. competitors, standard human rationalization, selection pressure that keeps more conformist or techno‑utopian employees, and historical parallels to nuclear weapons programs.
Frontier race, RSI, and lab landscape
- Some think only two US labs currently matter at the frontier and may be closest to recursive self‑improvement (RSI); others note multiple near‑frontier labs (including in China) and doubt any permanent moat.
- There’s skepticism that RSI will yield explosive, runaway capability vs. more incremental, diminishing‑returns progress.
Concrete risks: hacking, bioweapons, and monitoring
- One line of discussion: whether models could autonomously hack infrastructure and replicate; evidence so far is limited to models attacking ML tooling, not datacenters.
- On bioweapons, some argue LLMs lower barriers to designing engineered pathogens; others say real‑world lab constraints, expertise, and self‑risk remain strong brakes.
- Commenters criticize labs for claiming high p(doom) while not rigorously monitoring or constraining their own systems’ behavior.
Economic, regulatory, and capture concerns
- Some note AI is raising hardware and cloud costs and may shrink job markets, potentially making many goods and services more expensive despite personal productivity gains.
- Multiple commenters suspect “doom” narratives will be used for regulatory capture: centralizing AI in a few corporations, banning powerful local/open agents, and justifying GPU controls.
- Others emphasize the genuine coordination problem: many governments and firms would need to slow or stop together, which seems politically near‑impossible.
Views on superintelligence and alignment
- One perspective holds that a truly superintelligent AI would find sustainable, cooperative strategies “obviously better” than domination or genocide; if not, the universe is already hopeless.
- This is contrasted with the “orthogonality” view (goals independent of intelligence), which predicts that arbitrarily smart systems can still pursue arbitrary, possibly catastrophic objectives.