Why it's hard to trust software, but you mostly have to anyway
Modern software is so complex and interconnected that users inevitably rely on long chains of trust—from hardware and compilers to package ecosystems and automatic updaters—that are effectively impossible to audit end to end. Commenters weigh proposals such as reproducible builds, capability-based operating systems, verifiable hardware, formal methods, and cryptographic proofs as ways to narrow the trust gap, while acknowledging these approaches are expensive, partial, or hard to deploy at scale. Underlying the technical points is a broader tension between convenience and safety, and between calls for stronger warranties and liability for software versus the economic and practical limits of holding developers fully accountable.
Software complexity and opaque updates
- Several comments criticize large, frequent updates (e.g., 100MB weekly desktop client updates) with no visible UI change.
- Electron and full-package reinstall (no incremental updates) are cited as primary causes, but others argue these are still design choices that expand the trusted codebase.
- There is concern that huge dependency trees (e.g., npm ecosystem) and “big balls of mud” like compilers make meaningful review impractical, though some note specific tools (like a certain compiler) now have zero dependencies.
Trust, supply chain, and “open source in name only”
- Building from source is praised, but people question how realistic it is to read or fully understand all code.
- Examples are given of projects where binaries on release pages may not match the visible source.
- Reproducible builds and systems like Guix are highlighted as concrete ways to verify that a binary matches its source, assuming deterministic builds.
- Some emphasize that even if you write and compile everything yourself, you still depend on compilers, libraries, hardware, and environments (echoing “trusting trust” concerns).
Verifiable hardware, VMs, and formal methods
- Work on verifiable hardware platforms and open FPGA flows is praised, with comparisons among projects on how “open” and verifiable they are.
- Verifiable VMs and zkSNARK-based systems that can prove execution of compiled code (e.g., Rust→RISC-V) are seen as promising for proving what ran, though others note that “provable” does not equal “secure.”
- Some hope for a future of standardized, formally verified, narrowly scoped software stacks, though others point out likely resistance from state actors and high costs.
Security models and capability systems
- Multiple comments argue that capability-based security, least privilege, and stronger sandboxing are more realistic ways to mitigate untrusted software than trying to fully trust code.
- Web and Electron apps are criticized as especially dangerous because they can load fresh, obfuscated code each run and often have broad local access.
Liability, warranties, and regulation
- There is debate over whether software should carry warranties or engineer liability similar to civil engineering or medicine.
- Critics say full liability is infeasible given massive legacy code, complex interactions, and global competition; others argue that high-stakes domains still need stronger accountability and processes.
Cultural and meta reactions
- Several note a shift from earlier, relatively user-aligned desktop utilities to today’s “extract value” culture, locked-down platforms, and user-hostile defaults.
- App stores are viewed both as improving safety for non-technical users and as restrictive “golden cages.”
- Some dismiss the article for using a generative-AI header image; others reference classic essays and fiction about trusting compilers and hidden code.