If you love a website, back it up

Original Article ↗ Hacker News Discussion ↗

Personal website archiving is emerging as a response to accelerating link rot, corporate “enshittification,” and increasing AI-driven scraping barriers that threaten the longevity and accessibility of online content. Commenters trade tools and workflows—from SingleFile, ArchiveBox, Linkding, Linkwarden, Hister, HTTrack and wget to Obsidian clippers and custom scripts—while debating what’s worth preserving: entire sites, just the readable text, or even transformed formats like ebooks and PDFs. Many emphasize backing up what you truly value (including videos and niche forums), doing it politely to avoid overloading sites, and not relying solely on centralized services like the Internet Archive or commercial platforms to safeguard the digital record.

Link rot & motivation

  • Many commenters describe extensive link rot in decades of bookmarks, image collections, niche forums, university course pages, and personal sites.
  • People regret assuming beloved sites or platforms (fanfic archives, university video portals, Angelfire/Blogspot-era sites, niche technical wikis, indie music blogs) would “always be there”.
  • Internet Archive coverage is praised but often incomplete, blocked by robots.txt, or missing images/files.
  • Several note painful losses of unique technical info, art, music, and personal achievements that are now unrecoverable.

Tools and workflows for archiving

  • Popular combo: browser-based SingleFile + a self‑hosted bookmark manager (Linkding, Linkwarden, Karakeep) to store HTML snapshots; some also save PDFs and screenshots.
  • Heavier tools: ArchiveBox (recently rewritten, supports many formats and large-scale archiving), Hister (personal web-history search with HTML capture), Zotero (using SingleFile), Monolith, HTTrack, grab-site, custom crawlers.
  • Simpler approaches: print-to-PDF, screenshots, markdown clippers, Obsidian Web Clipper, ebooks via “save-as-ebook”, wget for static course sites.
  • Some build custom tools or bookmarklets to push pages to Internet Archive and personal repos.

Technical challenges

  • Modern web complexity (SPAs, JS rendering, anti-scraping, Cloudflare, CAPTCHAs) breaks many server-side archivers.
  • Client-side capture (browser extensions that upload rendered pages) is seen as the most reliable way around these defenses.
  • HTML vs content: several argue for focusing on extracting core text/knowledge (reader modes, Readability.js, markdown) instead of whole-site mirroring.
  • Scalability and maintenance of self-hosted systems are recurring pain points.

Ethics, legality, and ecosystem effects

  • Advice to crawl politely and slowly to avoid overloading small sites.
  • Concern that aggressive AI/SEO crawlers are already pushing sites to lock down, making personal archiving harder.
  • Private backups for personal use are widely assumed to be legally fine; redistribution is flagged as a different question.
  • Some highlight fragility of personal storage (bitrot on CDs/DVDs, failed drives) and stress multiple backups.

Philosophical splits & future worries

  • Tension between “archive everything you love” and “accept that some things should fade; avoid hoarding”.
  • Some emphasize printing and buying books as the ultimate long-term archive.
  • Several worry that as LLMs and closed platforms displace the open web and search, centralized actors could subtly distort or “rewrite” accessible history, strengthening the case for personal archives.