Everyone should slow down AI development except for me
Calls by major AI lab CEOs to “slow down” frontier model development are provoking skepticism and cynicism among technologists. Many see existential-risk rhetoric and proposals for government-aligned “independent evaluators” as a bid for regulatory capture that would entrench a small US oligopoly, restrict open-source and Chinese models, and concentrate power over a general‑purpose technology. Others argue that some form of global regulation or pause is still necessary given rapid capability gains and potential misuse, even if the current safety narratives are self‑serving or poorly targeted at today’s concrete harms.
Overall tone & context
- The linked note is read as satire of recent calls by major AI labs to “pace the frontier,” interpreted as “everyone else should slow down, not us.”
- Many commenters find the parody on-point; others say it misrepresents what the labs actually asked for.
AI safety vs present-day harms
- Some see “AI safety” rhetoric as focused on speculative “vengeful god” scenarios instead of current harms: manipulation, scams, child/education impacts, labor displacement, surveillance, and climate/energy use.
- Others argue existential risk is plausibly real, grounded in agentic behavior theory and historical parallels to early climate science, and legitimately dominates their priorities.
- There’s disagreement over whether focusing on doom makes safety folks sound like “end is near” prophets.
Power, regulation, and regulatory capture
- Strong concern that safety-driven regulation will entrench large US labs, crush open-source and smaller players, and be a classic case of regulatory capture.
- Specific worries: IP funnels into frontier labs, “independent evaluators” packed with ex-lab staff/equity holders, and licensing regimes that lock out competitors and non‑US models.
- Others counter that some regulation (e.g., transparency, limits on frontier training, penalties for harmful agent behavior) is necessary and could be targeted at big labs only.
Open vs closed models and uneven access
- Open-weight and Chinese models are said to be rapidly catching up in capability, especially for practical coding and niche tasks, and are cheaper, faster, and more private.
- Others argue frontier models still lead on absolute capability and efficiency, and open models remain months behind at the very top end.
- Many fear a future where governments and militaries get significantly more powerful, secret models while the public is restricted to weaker ones.
Geopolitics and governance
- Debate over “racing China”: some say slowing public AI just hands victory to China; others note potential for joint monitoring of compute but see politics as the real barrier.
- Deep distrust of both tech CEOs and governments; some would rather see broad proliferation and mutual checking between AIs than concentrated control.
Progress, data, and “moral panic”
- Split between “models plateaued or hitting practical limits” and “nowhere near a ceiling; data/compute bottlenecks are surmountable via synthetic data and hardware advances.”
- Disagreement over whether synthetic data inevitably causes model collapse, or can be used safely with care.
- Several liken current doom rhetoric to a moral panic (GPT‑2, Y2K), while others warn that dismissing concerns could age badly if risks materialize.