Drift towards danger and the normalization of deviance (2017)

Concept of Drift and Normalization of Deviance

• People describe how small, local optimizations under pressure (faster, cheaper, less hassle) gradually push systems toward failure boundaries.
• Over time, rule-breaking becomes routine and no longer feels deviant, even when it clearly violates original safety assumptions.
• This is seen as a general pattern across domains, not just in physical safety.

Corporate Culture, Boeing, and Managerialism

• Several commenters argue Boeing is no outlier; it has converged toward a broader US corporate culture prioritizing short‑term profit over engineering judgment.
• Cited drivers: finance/MBAs displacing domain experts, weakened antitrust, “shareholder value” ideology, and a managerial “caste” that believes in perfect plans executible by interchangeable workers.
• Others emphasize duopolies and weak competition as enabling bad products to survive.

Safety-Critical Organizations and Officially Sanctioned Deviance

• NASA’s shuttle program is discussed as a case where deviations were deliberately cataloged and repeatedly judged “acceptable,” making risk routine rather than exceptional.
• Nuclear power and Chernobyl are used to illustrate how planned tests and low‑probability edge cases can expose catastrophic failure modes.
• Healthcare is described via the “Swiss cheese” model: many overlapping safeguards, but rare alignments of small failures still injure patients.

Everyday Tools and Personal Risk Drift

• Multiple anecdotes (angle grinders, chainsaws, table saws, climbing) show how initial caution erodes with familiarity until a near miss or injury.
• Removing guards, using the wrong tool because it’s nearby, and accepting “pro” norms that look risky from the outside are recurring themes.

Software Engineering and Testing

• Parallels are drawn between physical safety and software practices: test coverage starts high then is eroded under delivery pressure.
• Debate over test costs: some see heavy testing as essential risk mitigation; others note long, expensive tests and incidents where test code itself caused failures.
• Frontend/UI testing is seen as particularly hard; visual regression tools are mentioned as a partial answer.

Mitigation Strategies and Limits

• Suggestions: zero tolerance for undocumented procedure deviations plus fast, formal mechanisms to update procedures; defense in depth; treating noncompliance as a design problem, not individual blame.
• Skepticism that individuals can change culture without authority; true safety culture requires coordinated leadership across departments.
• Heuristics for spotting trouble: frequent “if they had just…” narratives, pressure to remove safeguards, and long‑standing deviance that has merely “worked so far.”