Please don't share our links on Mastodon

Original Article ↗ Hacker News Discussion ↗

A Linux news site asked people not to share its articles on Mastodon after link previews from many federated servers overwhelmed its small, poorly cached backend, despite being behind Cloudflare. Commenters argue that while Mastodon’s current preview mechanism can create a “thundering herd” of duplicate requests, the immediate failure stems from the site’s own misconfiguration and lack of caching for largely static content. The exchange highlights broader tensions between decentralized social platforms’ design choices, trust in shared previews, and the responsibility of publishers to harden sites against sudden traffic spikes.

Traffic spike and Cloudflare configuration

  • Many argue the site’s outage is primarily due to misconfiguration, not Mastodon or Cloudflare.
  • The HTML responses are marked with headers like max-age=0, effectively disabling CDN caching.
  • Nginx appears not to cache dynamic resources; responses are extremely slow even without load.
  • The page pulls many third‑party assets (payments, external images), further increasing latency.
  • Commenters note Cloudflare works reasonably well with default settings; you must actively sabotage caching to get this fragile.

How Mastodon link previews behave

  • When a link is posted, it’s federated to many Mastodon instances; each instance independently fetches the page and preview image.
  • This can create a “Mastodon stampede” / thundering herd: thousands of nearly simultaneous GETs.
  • Unlike centralized platforms (Twitter, Facebook, Reddit), there’s no shared preview cache across instances.
  • Some see this as a form of unintentional DDoS or “traffic amplification,” even though the absolute volume (e.g., ~100MB) is modest.

Responsibility and “victim blaming”

  • One side: any news site in 2024 should be robust to small viral spikes; failing to cache and then blaming Mastodon is a “skill issue.”
  • Other side: even poorly configured sites shouldn’t be hammered by thousands of automated preview fetches; Mastodon’s design is inherently abusive to small sites.
  • There’s disagreement whether criticizing the site’s setup is reasonable or unfair “victim blaming.”

Proposed fixes on the website side

  • Enable proper caching (CDN and/or application-level).
  • Use static site generation or at least CMS caching plugins.
  • Cache all pages and purge via API on updates.
  • Optionally use CAPTCHAs or stricter WAF/bot rules, though some see that as degrading the web.

Proposed fixes on Mastodon/fediverse side

  • Include full preview metadata (and possibly image) in the post itself so instances don’t re-scrape.
  • Federate and cache previews, or delay/stagger preview fetching.
  • Respect robots.txt for non-user-triggered crawls, or only generate previews on explicit user actions.
  • Some object that sharing previews requires trust between instances; others argue trust should be anchored to the poster, not the preview generator.