Going Dark, and the era of law enforcement hacking
AI-boosted tools for finding software vulnerabilities could sharply reduce the number of exploitable bugs in widely used systems, potentially cutting off a major avenue for law enforcement and intelligence agencies to hack phones and computers. Commenters fear this will intensify pressure on tech companies and legislators to mandate backdoors, client-side scanning, or other “front door” access mechanisms, reshaping both global software markets and civil liberties. Others are skeptical that vulnerabilities will ever truly run out, pointing instead to AI-generated “slop” code, human error, and social engineering as enduring weak points.
AI, Software Bugs, and the “Going Dark” Thesis
- Some agree that AI vulnerability scanners will drastically reduce remotely exploitable bugs in major platforms, shrinking the available 0‑day pool.
- Others find this implausible: they see more buggy “AI slop,” rushed features, and believe new attack surface is growing faster than AI can secure it.
- Several argue AI is currently far from capable of generating the multi-stage, high-end mobile 0‑days sold on the gray market.
- A common synthesis: AI is a multiplier. Competent teams can become far more secure; under-skilled teams will ship even worse systems.
Backdoors vs Exploits and Likely Government Responses
- Many expect that as traditional exploits dry up, governments will increase pressure for legal/technical backdoors or “front doors” (e.g., client-side scanning, identity-verified access, anti-E2E measures).
- Concern that mandated backdoors would weaken national infrastructure and be exploitable by foreign adversaries.
- Others think fully preventing strong encryption in democracies is unrealistic; attempts may just drive more people to “dark” networks.
Self‑Hosting, Open Models, and Model Poisoning
- Strong current flowing toward self‑hosting open‑weight models and agents on user-controlled hardware to avoid legal pressure on big US-based AI providers.
- Counterpoint: open models can be poisoned too; hidden backdoors in training data could trigger on rare token sequences.
- Suggested mitigations: review generated code manually, use multiple independent vendors/countries, inspect reasoning traces for anomalies.
Security Practice and Industry Gaps
- Observations of a “two-worlds” split: elite offensive/defensive teams vs everyday orgs that still ignore basic security hygiene.
- Game industry cited as particularly lax, with old titles still shipping with serious RCEs.
- Some fear organizations already letting LLMs both write and review code with minimal human oversight.
Civil Liberties, Surveillance, and Democratic Drift
- Many commenters welcome law enforcement “going dark” as a net win for civil rights, given pervasive metadata, cloud logs, and cameras already.
- Others worry that loss of cheap exploits will accelerate authoritarian legislation and corporate coercion, creating de facto nationalized, backdoored stacks.
- Debate over how much citizens can realistically counter this via politics vs the risk of sliding into soft or overt tyranny.
Historical and Architectural Context
- Discussion of past wiretapping constraints (physical taps, pen registers, CALEA) vs today’s “surveillance too cheap to meter.”
- Some note that even with secure endpoints, attack focus may shift to infrastructure, supply chains, automatic update channels, or humans in the loop.