"Any sufficiently bad software update is indistinguishable from a cyberattack"
A recent mass outage caused by a faulty CrowdStrike security update is prompting comparisons between catastrophic software updates and cyberattacks, since both can rapidly disable millions of systems with no practical difference to the victim. Commenters debate whether giving third-party security tools kernel‑level or rootkit-like access creates more risk than it mitigates, touching on supply‑chain threats, the role of government and compliance mandates, and whether only first‑party or open‑source components should have such deep privileges. Others argue that safer programming languages, staged rollouts, and treating vendor updates as part of the threat model are essential to reducing the damage from inevitable internal failures as well as malicious attacks.
Meaning of “sufficiently bad update”
- Some see the phrase as trivial or circular: a bad update that looks like an attack… looks like an attack.
- Others argue it’s a useful reminder that vendor mistakes belong in the same threat model as supply‑chain attacks.
- Several note it’s a play on Clarke’s “indistinguishable from magic,” not meant as deep theory.
- Others link it to Hanlon’s Razor: don’t assume malice where incompetence explains events, but note that harm is similar either way.
Security Products vs Malware
- Many comments argue endpoint security tools with kernel/root access, remote command, and data exfiltration are technically indistinguishable from malware.
- There’s concern that government and compliance regimes push such tools, centralizing massive power and creating single points of failure.
- Some stress that in practice these tools are chosen by corporate IT/infosec, not end users, and thus come with “owner consent,” though others contest how informed that consent is.
- A recurring criticism: relying on rootkit‑like agents is “security to check boxes,” not layered, holistic security.
Languages, Memory Safety, and Reliability
- Strong debate over C/C++ vs Rust vs Ada.
- One side: memory‑safe languages significantly reduce whole classes of bugs; Rust in particular gives long‑term confidence in correctness.
- Counterpoint: highly reliable C exists; language choice isn’t a magic bullet; bad programmers can misuse Rust (
unsafe) too. - Rust panics are debated: safe in user space but unacceptable in kernels/real‑time systems where robustness beats crashing.
- Memory safety is seen as important for security, but not sufficient for resilience to failures like this incident.
Kernel Extensions and System Architecture
- Many advocate minimizing or banning third‑party kernel modules, comparing Windows’ culture of kernel hooks to macOS deprecating kexts and Linux “tainted” kernels.
- Others note practical constraints: hardware drivers, gaming anti‑cheat, and monitoring often still demand kernel‑level components.
- Suggestion: favor user‑space mechanisms (eBPF, tracing APIs, network extensions), strict signing, and strong warnings if deeper access is used.
Update Process, Testing, and Rollout
- Multiple commenters are baffled that a staged rollout wasn’t used or wasn’t effective.
- Speculation that “content” updates (e.g., rules, configs) may be tested less rigorously than code.
- Hearsay claims (flagged as such in the thread) criticize weak engineering practices and ignored post‑mortems; others note the financial impact may force improvement.
Trust, Consent, and Open Source
- Some argue only open‑source software can really be trusted, since it’s inspectable and modifiable, and you can run exactly what you audit.
- Others reply that open source still breaks, and that “open vs closed” distracts from more actionable reforms (rollout safety, least privilege, better design).
- There’s broad agreement that granting any vendor remote, high‑privilege control is a major, often underappreciated, risk.