Drift towards danger and the normalization of deviance (2017)
Safety failures rarely stem from a single bad decision; they emerge from a gradual “normalization of deviance,” where shortcuts and workarounds become accepted as standard practice because nothing has gone wrong yet. Commenters connect this pattern to incidents in aviation, spaceflight, nuclear power, healthcare, industrial work, and software engineering, and to broader shifts like financialized “Welchian” management, weak antitrust enforcement, and duopoly markets that blunt competitive pressure on quality. Proposed mitigations include treating procedural violations as signals of flawed processes, maintaining fast formal mechanisms to update rules, and recognizing that genuine safety and reliability must be an organization-wide responsibility rather than something left to frontline workers alone.
Concept of Drift and Normalization of Deviance
- People describe how small, local optimizations under pressure (faster, cheaper, less hassle) gradually push systems toward failure boundaries.
- Over time, rule-breaking becomes routine and no longer feels deviant, even when it clearly violates original safety assumptions.
- This is seen as a general pattern across domains, not just in physical safety.
Corporate Culture, Boeing, and Managerialism
- Several commenters argue Boeing is no outlier; it has converged toward a broader US corporate culture prioritizing short‑term profit over engineering judgment.
- Cited drivers: finance/MBAs displacing domain experts, weakened antitrust, “shareholder value” ideology, and a managerial “caste” that believes in perfect plans executible by interchangeable workers.
- Others emphasize duopolies and weak competition as enabling bad products to survive.
Safety-Critical Organizations and Officially Sanctioned Deviance
- NASA’s shuttle program is discussed as a case where deviations were deliberately cataloged and repeatedly judged “acceptable,” making risk routine rather than exceptional.
- Nuclear power and Chernobyl are used to illustrate how planned tests and low‑probability edge cases can expose catastrophic failure modes.
- Healthcare is described via the “Swiss cheese” model: many overlapping safeguards, but rare alignments of small failures still injure patients.
Everyday Tools and Personal Risk Drift
- Multiple anecdotes (angle grinders, chainsaws, table saws, climbing) show how initial caution erodes with familiarity until a near miss or injury.
- Removing guards, using the wrong tool because it’s nearby, and accepting “pro” norms that look risky from the outside are recurring themes.
Software Engineering and Testing
- Parallels are drawn between physical safety and software practices: test coverage starts high then is eroded under delivery pressure.
- Debate over test costs: some see heavy testing as essential risk mitigation; others note long, expensive tests and incidents where test code itself caused failures.
- Frontend/UI testing is seen as particularly hard; visual regression tools are mentioned as a partial answer.
Mitigation Strategies and Limits
- Suggestions: zero tolerance for undocumented procedure deviations plus fast, formal mechanisms to update procedures; defense in depth; treating noncompliance as a design problem, not individual blame.
- Skepticism that individuals can change culture without authority; true safety culture requires coordinated leadership across departments.
- Heuristics for spotting trouble: frequent “if they had just…” narratives, pressure to remove safeguards, and long‑standing deviance that has merely “worked so far.”