After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

Original Article ↗ Hacker News Discussion ↗

Debate over “AI safety” is sharply divided between those who see existential risks from future AGI and those who view current fears as sci‑fi hype and regulatory theater. Commenters distinguish between near-term issues like propaganda, job displacement, deepfakes, and automated cyberattacks, and speculative scenarios where a self-improving AI gains power beyond human control. Many argue that corporate “safety” efforts mostly protect brands and entrench incumbents, while others contend that even low-probability catastrophic risks justify serious research and policy attention now, especially in a global arms-race context.

Scope of “AI safety”

  • Commenters distinguish:
    • “Mundane” safety: content filters, preventing explicit/illegal outputs, self‑driving car safety, brand protection.
    • “Existential” safety: AGI/ASI going rogue, self‑improvement “foom,” human extinction or domination.
  • Many say threads and media constantly conflate these two, confusing the debate.

How dangerous are current systems?

  • Sceptics: LLMs are sophisticated text predictors, sometimes “dumb,” not agents; threats now are overhyped “sci‑fi,” akin to early self‑driving car hype.
  • Others: even non‑AGI systems can already:
    • Mass‑generate propaganda, misinformation, blogspam.
    • Assist hacking and cyberwarfare.
    • Aid terrorism, bioengineering, or detailed planning.
  • Some argue risk is mainly humans misusing tools, not AIs having goals.

AGI timelines and “foom”

  • One camp: AGI is far off or undefined; we can regulate when actual capabilities appear.
  • Another: progress (e.g., transformers, GPT‑4, Go‑playing systems) surprised experts; we may be “years, not centuries” away.
  • Debate over “foom”:
    • Fast takeoff (hours–weeks) is seen by many as thermodynamically or practically unlikely.
    • Slower but still rapid self‑improvement over years is considered more plausible and still dangerous.
  • Disagreement whether it’s sensible to work on alignment before AGI exists.

Control, shutdown, and agency

  • Some say you can always “pull the plug” or air‑gap systems.
  • Others note:
    • Distributed cloud deployments, open‑source, and financial autonomy could make shutdown hard.
    • Incentives (corporate profit, military advantage) mean systems will be networked and given more control.
    • Aligning strong agents so they remain “corrigible” is still an unsolved research problem.

Corporate motives and regulatory capture

  • Strong suspicion that “AI safety” rhetoric is used to:
    • Influence regulators.
    • Slow open‑source/competitors.
    • Reassure the public while racing ahead.
  • Some argue current “safety” work is mostly about brand management and censorship, not existential risk.

Geopolitics (especially China)

  • Several note any unilateral pause is unrealistic because of state competition.
  • Some expect China to:
    • Ignore Western safety norms domestically.
    • Publicly support regulations that slow Western firms.
  • Others counter that all major powers face similar ideological and surveillance constraints around AI.

Guardrails, moderation, and ideology

  • Many complain about overzealous filters (e.g., image models refusing benign prompts, text models refusing ordinary queries).
  • Concern that:
    • Safety tuning is entangled with enforcing particular social or political narratives.
    • Over‑filtering “lobotomizes” models and incentivizes users to jailbreak them.
  • Others insist robust guardrails are needed to avoid obvious harms (self‑harm advice, school‑shooting guides, etc.).

Societal, economic, and political risks

  • Anticipated harms include:
    • Job displacement and weaker worker bargaining power.
    • Concentration of power in corporations and states that can best exploit AI.
    • Cheap mass manipulation, targeted propaganda, and synthetic “mass movements.”
    • Delegating opaque AI systems to critical decisions (finance, military, governance) that humans can’t audit.

Alignment research and feasibility

  • Some deride current AI safety/alignment work as speculative, like bad philosophy or theology, with few concrete results.
  • Others point to emerging interpretability and steering work and argue:
    • Understanding current large models is prerequisite to safely handling more capable ones.
    • Even if much research is low‑value, the potential downside justifies substantial investment.

Ethics, rights, and values

  • A minority raises:
    • Potential future AI rights if systems become sentient or “minded.”
    • The contradiction between worrying about AI welfare while tolerating large‑scale animal suffering.
  • Others emphasize that current systems are unconscious tools; talk of AI resentment or “feelings” is seen as speculative or anthropomorphic.