Where did the old web go? We followed 657,607 links to find out

Original Article ↗ Hacker News Discussion ↗

An analysis of 657,607 URLs from a resurrected 2009–2014 link shortener finds that roughly three-quarters of the pages once shared through the service no longer load, underscoring how fragile the web’s memory is. Commenters debate what era truly counts as the “old web,” contrasting early personal sites, forums, and blogs with today’s centralized, paywalled, JavaScript-heavy platforms and AI-generated content. Many see link rot, corporate walled gardens, and shifting incentives as key forces eroding the open, human-curated web, even as projects like archive.org and new tools such as AI-driven maintenance try to keep pieces of it alive.

Link rot and ephemerality of the web

  • Many see the high share of dead links as evidence that the web’s impermanence is a serious design flaw.
  • Others argue all media are impermanent (books burn, stone erodes); the web is not uniquely fragile.
  • Several note the irony that a link shortener that went offline for a decade is now analyzing others’ outages.
  • A key nuance: “still serving HTTP” ≠ “original content still exists” (parked domains, paywalls, login walls, removed-content notices).

Archiving, storage, and technical fixes

  • Archive.org is widely praised but also criticized as incomplete for complex sites.
  • People discuss bit-rot, ECC, content-addressed storage, and torrents as ways to make data loss “astronomically” unlikely, but note consumer systems rarely use robust protections.
  • Content-addressable, peer-to-peer systems (IPFS, Freenet/Hyphanet) are cited as promising but still limited and dependent on someone continuing to host the data.

Defining the “old web”

  • Strong disagreement on what “old web” means:
    • Ranges from pre-Google / pre-SEO / pre-social media (early–mid 2000s)
    • To “Cretaceous web” (gopher, Usenet, pre-WWW)
    • Some consider 2009–2014 “middle” or “late middle” web, not “old.”
  • Nostalgia centers on: personal homepages, webrings, guestbooks, tilde directories, phpBB forums, Geocities/Angelfire, early blogs, less monetization, and fewer algorithmic feeds.
  • Critics say the old web was also cluttered, ugly, and full of junk; they argue similar creativity still exists but now lives on large platforms.

Centralization, walled gardens, and culture

  • Many blame social media and mega-platforms for sucking content out of the open web into ad-driven, rule-heavy silos.
  • Loss of small forums and local communities (e.g., gaming boards, LiveJournal-like spaces) is mourned; some see sites like HN, mailing lists, gopher/gemini as “old web” holdouts.
  • There’s concern that corporate moderation, advertiser pressure, and platform rules force self-censorship and narrow acceptable culture.

AI, spam, and the future

  • The reboot of 0.mk as an AI-assisted service prompts questions about costs and risks (e.g., prompt injection leading to destructive actions).
  • Some hope AI-based search will kill SEO spam; others fear a new wave of AI-generated spam optimized for LLMs instead of humans.
  • One view: the mass audience may retreat into even more enclosed platforms, leaving a smaller, more “nerdy” open web reminiscent of earlier eras.