AI handles incidents, engineers lose touch with their systems
As AI systems increasingly diagnose and remediate production incidents, many engineers fear they’re losing the hands‑on intuition needed to understand and safely operate complex software. Commenters draw parallels to aviation’s “ironies of automation,” noting that when machines handle routine work, humans get fewer reps and may be unprepared for rare, high‑stakes failures—especially as new generations start their careers in an AI‑mediated world. Others counter that, used well, AI can raise the operational “floor,” accelerate troubleshooting, and free experts to focus on architecture and guardrails, but only if organizations invest deliberately in training, simulation, and clear boundaries for automated actions.
Role of AI in Incidents and Operations
- Many see LLMs already outperforming humans at triage in observable, standardized systems (e.g., Kubernetes, cloud infra), rapidly narrowing root causes.
- Others argue that any AI-produced change is still an information system humans can inspect; truly hard incidents will still require competent operators.
- There’s concern that “AI clears the easy tickets,” leaving humans only rare, severe incidents without enough real practice, echoing aviation’s automation issues.
Skill Atrophy and Loss of System Intuition
- Repeated theme: using AI feels like “quicksand” — the more it writes/debugs, the less intuitive grasp engineers have of their own systems and code.
- Senior engineers report colleagues regressing: jumping straight to AI, brute‑forcing error messages, losing debugging discipline and architectural thinking.
- Worry is highest for juniors who may never build fundamentals, especially under time pressure and “use AI or you’re slow” culture.
Quality of AI-Generated Code and Debugging
- Complaints: over‑engineered, verbose, duplicated, and inconsistent code; unstable “libraries” that change with each generation; tech debt explosion.
- Others counter that in many existing codebases, LLM output is already better than average human code and excellent for refactors/ports when guided.
- Benchmark results and anecdotes show models are “spiky”: brilliant on some bugs, terrible or hallucination‑prone on others. Trust remains a core problem.
Organizational Incentives and Culture
- Several comments blame management: OKRs explicitly incentivize heavy AI use, low‑code tooling, and shipping speed over understanding or reliability.
- Some argue this mirrors long‑standing industry patterns (outsourcing, RPA, “ops via scripts”) rather than something uniquely new.
- There’s tension between viewing developers as craftspeople vs interchangeable delivery labor; AI amplifies that conflict.
Mitigations and Proposed Practices
- Suggestions: drills/chaos engineering, tabletop incident simulations, explicit guardrails (AGENTS.md), RPI/SDD loops, and “AI chef” roles mediating tool use.
- Emphasis on keeping humans in charge of architecture, boundaries, and high‑risk areas, with AI as a chisel, not an autopilot.
- Some advocate professionalization/licensing (analogous to pilots) to force incident training and prevent corner‑cutting.
Longer‑Term Trajectory and Open Questions
- Split views:
- One side predicts equilibrium where abstraction levels stabilize and AI is “just another tool.”
- Others foresee deep dependency: humans no longer understanding underlying systems, with AI both building and babysitting opaque stacks.
- How to train experts in an AI‑first world remains unresolved.