Banning open weight models would be a disaster
Proposed U.S. rules that could restrict or ban the release of open‑weight AI models are drawing sharp criticism from technologists and civil libertarians. Many see the move as a bid for regulatory capture by large AI companies, comparing it to past attempts to control encryption and warning it would entrench corporate and state power while doing little to stop bad actors abroad. Others, more worried about AI’s potential for disinformation, economic disruption, and concentrated power, argue that some limits on frontier models may be justified, but acknowledge enforcement would be technically and geopolitically fraught.
Awareness and Process
- Several commenters only learned of the NTIA/DoC open‑weights RFC after the comment deadline, and felt it had little media or HN visibility.
- Some attribute this to dense legal language, news-cycle overload, and public apathy; others suspect that “boring” framing hid a major policy shift.
Free Speech, Law, and Constitutionality
- One camp argues model weights are expressive like source code and so bans would violate the First Amendment, citing 1990s encryption precedents.
- Others counter that weights are machine‑generated numbers, not human-authored “speech,” so protection is less clear.
- Legal discussion notes that even if weights are speech, content‑neutral restrictions (e.g., on parameter size) might survive “intermediate scrutiny,” whereas content-based “safety standards” (e.g., blocking hate/disinformation) likely would not.
- There is skepticism that the current Supreme Court will reliably protect rights when money and power are at stake.
Safety, Misuse, and Comparisons (Nukes vs. Encryption)
- Supporters of restrictions liken frontier models to nuclear tech: huge “blast radius,” potential for disinformation, deepfakes, and scalable scams.
- Opponents say this is overstated: current LLMs are more like lossy Wikipedia and autocomplete; harms are real but not existential.
- Encryption analogy: many see open weights as the new crypto wars; others argue AI uniquely undermines trust and thus is not comparable.
Regulatory Capture and Power Centralization
- Strong concern that bans would entrench large US labs and cloud providers, giving them monopolies and surveillance leverage.
- “Please regulate us” statements from big labs are widely viewed as self‑serving, aiming at regulatory capture rather than genuine caution.
- Some warn closed models enable unaccountable manipulation, since users cannot inspect or control the systems shaping information.
Practicality and Geopolitics
- Many argue bans are unenforceable: weights fit on hard drives, can be torrented and mirrored abroad; US would need China-style controls or even extreme measures (e.g., tracking GPUs, attacking “rogue” data centers).
- Others note US/EU control talent, TSMC/Nvidia pipelines, data, and institutions, so restrictions could still significantly slow open research.
- There is debate whether global coordination is realistic; some expect non‑Western countries to ignore bans and gain advantage.
Copyright, Training Data, and Fair Use
- Disagreement over whether model weights are derivative works of training data or even copyrightable at all.
- Some argue open models built on copyrighted corpora are likely fair use; others emphasize that copyright and derivative-work doctrine are already stretched and contentious.