Incident with Actions and Pages

Original Article ↗ Hacker News Discussion ↗

Current Incident & Reliability Concerns

  • Actions and Pages were down; many saw errors like “account suspended” and initially thought their own accounts or YAML were broken.
  • github-actions[bot] apparently became a “ghost” user / was suspended, breaking workflows; some PR checks showed green despite pipelines not running.
  • Commenters note multiple similar outages in recent weeks; some now assume “if something feels off, GitHub is probably down.”

Status Page, Monitoring, and Uptime Metrics

  • GitHub’s official status and uptime figures are widely distrusted; people cite third‑party trackers showing significantly worse availability.
  • Complaints that status updates lag user experience, likely due to thresholds, manual approvals, and SLA/marketing considerations.
  • Debate over good SRE practice: some argue you should page on every 500 (or synthetic failure); others say that’s impractical at scale and error budgets are standard.

Root Causes: Load, AI, Architecture, and Microsoft

  • One camp blames massive growth and AI/agent traffic: LLM agents cloning repos thousands of times and GitHub reporting 10–14× usage growth.
  • Others counter that competitors haven’t seen similar instability and that GitHub outages predate “agentic AI,” correlating more with the Microsoft acquisition and Azure migration.
  • Additional theories: architectural complexity (“Frankenstein” Azure/control plane), Copilot/AI-driven internal development, and under‑investment in reliability.

Operational Impact on Teams

  • Many teams’ deployments and CI are fully blocked, including those with self‑hosted runners or external runners, because GitHub’s control plane and job queue are the single point of failure.
  • Some now create explicit contingency plans for long GitHub outages; others are reconsidering org‑wide moves to Actions that were pushed after acquisitions.

Alternatives to GitHub and Actions

  • Suggested source hosts: self‑hosted GitLab, Forgejo/Gitea, SourceHut, Codeberg, custom bare repos, and various hosted Forgejo instances.
  • Suggested CI/CD alternatives: GitLab CI, Buildkite, Woodpecker CI, TeamCity, Jenkins, CircleCI, Azure DevOps, plus newer “Actions‑compatible” or local‑runner products.
  • Several new projects/startups are being built specifically as “modern GitHub/GHA replacements,” reflecting perceived market opportunity.

Self‑Hosting, Decentralization, and CI Philosophy

  • Strong current toward self‑hosting: “stop relying on GitHub,” run your own forge and runners, keep local clones and offline backups.
  • Others warn about the maintenance burden (patching, monitoring, backups) and argue that some third‑party dependency is inevitable.
  • Individual developers report abandoning CI entirely or relying on simple local scripts/Makefiles, arguing risk is acceptable for small teams, while others insist automated, independent CI remains essential.