Rate limits on GitLab.com are changing

Original Article ↗ Hacker News Discussion ↗

GitLab.com’s new API rate limits — 60 requests per hour for anonymous users and 5,000 per hour on the free authenticated tier — are prompting concern about how easily humans, CI pipelines, and open source contributors will be able to interact with public projects. Many see the change as a response to heavy automated scraping by LLMs and bots, and as part of a broader trend toward paywalls, authentication requirements, and a less open web. Others argue the limits are a reasonable cost-control measure that encourage sign‑in or self‑hosting, while worrying this may still push open source hosting and collaboration into more commercialized or fragmented models.

Perceived Reason: LLM Scraping & Bot Traffic

  • Many assume the tighter limits are driven by LLM/agent scraping and automated traffic.
  • Some argue this is an early step in a broader shift: the open web cannot handle AI-scale scraping, so it will become more closed, authenticated, and paywalled.
  • Others say AI scraping could be mitigated technically and is being used as a convenient justification for monetization.

Impact of New Rate Limits

  • Unauthenticated users get 60 requests/hour per IP; several note this is “one request per minute” on average but likely implemented as a token bucket.
  • People worry this will:
    • Hit humans behind CGNAT or shared school/office networks.
    • Make casual browsing of issues/PRs or multi-API-page loads frustrating.
  • Logged-in free users reportedly get 5,000 requests/hour, which many see as reasonable.
  • Some predict a walk-back; others think limits may still be too generous given bot load.

Broader Internet & Commercialization Debate

  • Thread veers into calls for a “non-commercial internet” without ads or bots, paid directly by users.
  • Counterpoints:
    • Anything valuable attracts commercialization and abuse.
    • Paywalls hurt lower-income users; ISPs already gate access.
    • Proof-of-identity to block bots would undermine privacy and anonymity.
  • Some suggest “indie web” approaches (personal sites, webrings) but note that anything popular will attract predators and spam.

Open Source Funding and Paywalls

  • Idea: charge scrapers and share revenue with repos, like streaming royalties.
  • Others warn of “Cobra effect” incentives: fake repos and self-scraping to farm payouts.
  • Concern that generic limits plus “upgrade to Premium/Ultimate” guidance effectively push OSS into paywalled or private setups.

GitHub & Other Platforms

  • Users observe rising friction on GitHub (bot detection, strict logged-out limits).
  • Comparisons suggest unauthenticated 60/hour is becoming a de facto norm.
  • Some believe generous unauthenticated access is unsustainable at AI scale.

GraphQL, APIs, and Agents

  • Several recommend using GraphQL for LLM agents to reduce request counts and payload sizes.
  • Others caution that complex GraphQL queries can be server-expensive, hit hidden limits, and strain backends like GitHub/GitLab.
  • General view: great for agents when carefully scoped; dangerous for servers if not.

Self‑Hosting & Alternatives

  • Some move to self-hosted Git platforms (e.g., Forgejo, self-hosted GitLab) to avoid external rate limits and future paywalls.
  • Suggestion that if you need large unauthenticated access, you should run your own mirror or infrastructure.