Twitter shut off API access; users volunteering their own data for an open API

Original Article ↗ Hacker News Discussion ↗

Motivation for a Community Twitter Archive

  • Goal: rebuild useful API-like access by letting users donate their own Twitter data exports.
  • Seen as a way to escape Twitter’s lockdown, preserve conversation history, and enable new tools (e.g., question-answering over someone’s tweet history).
  • Some frame it as helping people migrate off Twitter while keeping their “living wiki” of posts and threads.

Data Storage, Cost, and Infrastructure

  • Skepticism about “just put everything in S3” as “cheap.”
  • Suggestions to avoid big clouds due to bandwidth costs; proposals for dedicated servers (e.g., unmetered storage boxes) or S3‑compatible object storage / self-hosted systems.

API Lockdowns, Scraping, and the Changing Web

  • Widespread frustration with Twitter/Reddit-style API restrictions and high pricing; many foresee a return to heavy web scraping.
  • Others argue operators understandably don’t want AI companies and scrapers to extract huge value from their data for free.
  • Historical context: early Twitter had RSS, SMS posting/alerts, and a more open API; shutdowns are tied to ad/engagement models.

Privacy, Abuse Risks, and Cognitive Security

  • Concern that making identifiable archives queryable enables targeted phishing and other manipulation.
  • Calls for explicit consent and clear warnings that data may be permanently mirrored by others.
  • Some suggest making datasets private or invite‑only, or smaller community‑scoped collections.
  • Worries about data poisoning (fake tweet archives); ideas include requiring multiple independent corroborating uploads and web‑of‑trust mechanisms.

Decentralized Alternatives and Their Trade-offs

  • Some advocate moving to Mastodon, Bluesky, or Nostr; others report Mastodon’s culture, admin drama, defederation, and lack of post migration as major drawbacks.
  • Debate over whether a federated, server‑oriented design is inherently flawed or simply the only non‑corporate option.

Crowdsourced Scraping via Browser Extensions

  • Interest in extensions/userscripts that passively and anonymously upload what users already view (likened to RECAP for PACER or other crowdsourced tools).
  • Recognized as TOS‑gray but potentially hard to distinguish from normal browsing; privacy and site‑specific tailoring are unsolved challenges.

Broader Social Media Reflections

  • Many describe Twitter as increasingly toxic and engagement‑optimized; a minority say it still works well with careful curation.
  • Lock‑ins are attributed to network effects, income from engagement programs, and behavioral inertia despite perceived harm.