AI’s big rift is like a religious schism

Original Article ↗ Hacker News Discussion ↗

Debate over AI’s future is increasingly framed as a quasi-religious split between “accelerationists,” who see near-term superintelligent systems as inevitable and desirable, and “doomers,” who warn of existential risk and call for strong controls. Commenters argue that today’s systems are powerful but narrow—closer to sophisticated autocomplete than true minds—and that the most urgent dangers are political and economic: surveillance, concentrated corporate power, mass job displacement, and regulatory capture justified by hype. Several voices call for more humility about long-range predictions of a “singularity,” stressing that past technologies have transformed society in unpredictable, often slower, and more mundane ways than their prophets expected.

Religious / ideological framing

  • Many see the “doomer vs accelerationist” split as essentially eschatological: competing secular religions with prophecies of apocalypse or salvation.
  • Some argue this is psychologically similar to past millenarian or scholastic debates (angels on pins, universal intellect), where elaborate logic is built on shaky priors.
  • Others push back that AI x‑risk is not just a cult; scientists closely involved with modern ML genuinely worry about catastrophic risk.

Singularity concept and history

  • Discussion clarifies the original “technological singularity” idea: not just AI, but a runaway acceleration in innovation so steep it becomes opaque from the “pre‑singularity” side.
  • Several note that real-world growth is usually sigmoid, not literally singular; physical limits and resource constraints likely prevent infinite takeoff.
  • There’s broad skepticism that we’re near such a point, especially on the basis of current LLMs.

AGI feasibility and timelines

  • Opinions range from “AGI in a few years” to “not this century” to “maybe never, given current methods”.
  • Some emphasize huge gaps: no clear definition of “human-level intelligence”, no embodiment, no robust self‑improvement.
  • Others argue that once you accept non-mystical views of mind, human-level machine intelligence is ultimately a matter of scale and engineering.

Near-term vs long-term risks

  • Many say existential “Skynet” scenarios are low-probability compared to mundane harms:
    • Surveillance, behavioral profiling, and sentiment analysis at scale.
    • Floods of low-quality or malicious content.
    • Automated decision systems with bias and opaque errors.
    • Military uses: drone swarms, autonomous weapons, escalation risks.
  • Some contend that misaligned superintelligence is still worth thinking about, because we routinely underestimate tail risks of new technologies (nukes, biotech analogies).

Regulation, power, and incumbents

  • Strong concern that “AI safety” discourse will be used to entrench large incumbents via heavy regulation (e.g., parameter-count thresholds).
  • Others argue policy work so far is mostly exploratory and not yet clearly capture-driven.
  • Several note the real alignment problem today may be aligning AI with the public interest vs. corporate or state power, not just with “human values” in the abstract.

Societal and labor impacts

  • Widespread expectation that current systems already threaten many “junior” or routine white‑collar roles.
  • Some predict accelerated enshitification: cost-cutting, mass automation, and gig-ified cleanup work rather than broad shared prosperity.
  • A minority stress potential for AI to empower small firms and individuals against large organizations, depending on access and governance.

Nature and limits of current LLMs

  • One camp insists LLMs are “just” next-token predictors / glorified autocomplete, lacking agency, understanding, or consciousness; thus talk of godlike AGI is premature.
  • Others counter that:
    • Statistical modeling of language implicitly encodes concepts and some reasoning.
    • Even if “only prediction”, emergent capabilities at scale are nontrivial and already superhuman in narrow domains.
  • Debate continues over anthropomorphism: some fear we over-interpret chatty systems; others say dismissing them as mere parlor tricks ignores real capabilities.

Data, copyright, and training

  • Ongoing argument over whether training on copyrighted works is analogous to human learning, and whether outputs that remix styles/content should trigger licensing.
  • Key question: should legality hinge on who (human vs model) did the remixing, or on the similarity and market impact of the result?

Cults, rationalism, and safety movements

  • Several threads criticize parts of the rationalist / effective altruist ecosystem as doomsday cults with quasi-religious narratives (AI gods, future trillions of lives, moralized calls for extreme action).
  • Others defend them as unusually serious about risk analysis and calibration, albeit with strong priors and social incentives.
  • There is significant unease about fringe actors potentially justifying violence (e.g., talk of striking datacenters) in the name of “saving the world”.