Apparently Google hates us now

Original Article ↗ Hacker News Discussion ↗

Context: Pokémon Central Wiki Deindexing

  • Italian Pokémon wiki reports going from ~500k indexed pages to 11, with >100k URLs marked “crawled but not indexed.”
  • Other small wikis and long‑running blogs report similar recent drops and “crawled but not indexed” with no clear reason in Search Console.

Speculated Technical Causes

  • Possible bugs or “jank” on Google’s side; large systems can accidentally exclude small fractions of sites.
  • Hypotheses: Cloudflare anti‑bot rules blocking Googlebot in some paths; wiki spam or malware; misconfigured robots.txt (though OP says this was checked); use of anti‑LLM training flags (e.g., TDMRep) coinciding in time.
  • One theory: because much content is translated from an English wiki, internal systems may treat it as easily derivable and deprioritize crawling/indexing.
  • Another angle: brand confusion with “Pokémon Trainer Central” rebrand affecting rankings for “Pokemon Central,” though OP stresses the bigger issue is loss of indexing for specific topic pages in Italian.

Wikis, Spam, and Anti‑Abuse

  • Multiple wiki maintainers describe severe modern spam: bots, “sleeper” accounts, LLM‑assisted sign‑ups solving CAPTCHAs.
  • Mitigations mentioned: Cloudflare rules, limited permissions for new accounts, manual patrolling, Anubis (anti‑scraper), invite‑only systems, custom knowledge‑based CAPTCHAs, domain blacklists.
  • Debate over tree‑based invite/reputation systems: some see them as powerful; others point to abuse, account hacking, and raising barriers for genuine newcomers.

Broader Indexing Trends

  • Several participants note widespread “crawled but not indexed” reports and argue Google is drastically shrinking its index to a smaller set of “primary authorities.”
  • Some think wikis are inherently high‑risk SEO targets and require meticulous hygiene (sitemaps, metadata, spam control).

Economic and Power Concerns

  • Sites relying on Google Search + AdSense see sharp traffic and revenue hits; some move to in‑house ads, acknowledging significant extra work.
  • Strong criticism of opaque, one‑sided decisions: platforms can effectively “disappear” sites or accounts without clear explanation, functioning as unaccountable gatekeepers.

Alternatives and Changing Search Behavior

  • Kagi, DuckDuckGo, Brave, Startpage, Ecosia, Yandex, Marginalia cited as alternatives; some report better results, others mixed.
  • Several argue classic SEO and search traffic are fading as users increasingly ask LLMs directly; others note LLMs still depend on SEO‑shaped web data.

Views on Google’s Motives

  • Split between “bug/latent side effect” vs. “deliberate strategy” interpretations.
  • Many describe Google as profit‑maximizing, indifferent to publishers, pushing zero‑click results, AI overviews, and more ads.
  • Some call for antitrust or EU action; others see this as the predictable evolution of an ad‑driven monopoly.