Call-to-Action on SB 1047 – Frontier Artificial Intelligence Models Act

Scope and Intent of SB 1047

• Targets “frontier” models: those trained above a 10^26 FLOPs threshold and with “hazardous capabilities” (e.g., enabling WMDs, massive cyber damage, or large-scale criminal harm).
• Requires developers to:
  • Assess hazardous capabilities before/after training.
  • Maintain shutdown/“kill switch” capability.
  • Annually certify safety compliance and report incidents to a new “Frontier Model Division.”
• Regulates large compute clusters, requiring policies to evaluate AI deployment use, and creates a state cloud cluster (“CalCompute”) for safer, equitable AI work.
• Mandates transparent, uniform pricing for commercial access to covered models.

Impact on Open Source, Small Developers, and Academia

• Many argue the bill effectively kills open-source frontier models in California by:
  • Making base model creators liable for harmful fine-tunes they do not control.
  • Imposing compliance burdens only large firms or states can meet, pushing centralization and regulatory capture.
• Others say most current projects are just consumers of existing models and would be unaffected; only very large, hazardous-capable models are in scope.
• Disagreement over whether derivative/academic models are meaningfully exempt; some see academics as disproportionately burdened.

Definitions and Predictability

• “Artificial intelligence model” definition is viewed by several as so broad it could cover most software, raising fears of spillover.
• “Hazardous capability” is tied to immense damage, but posters disagree:
  • Some say today’s models already qualify (phishing, code analysis, deepfakes).
  • Others argue nothing yet represents a genuine step-change in harm.
• Multiple commenters doubt developers can realistically foresee or quantify future damage as the law seems to require.

Liability, Free Speech, and Existing Law

• Core criticism: the bill criminalizes creating a general-purpose model that others later misuse, even via extreme fine-tuning.
• Comparisons to other tools (guns, chemistry books, Google, Photoshop): some argue tools shouldn’t be blamed, only users.
• Debate over whether regulating model code/weights is regulating speech:
  • Some invoke precedent that code is speech.
  • Others counter that product safety rules (like for cars) are not First Amendment issues.

Regulation Philosophy and AI Risk

• One camp: regulation should focus on harmful actions (WMD construction, cybercrime) rather than model training; many such acts are already illegal.
• Another camp: AI is uniquely high-stakes (potential superintelligence, existential risk); waiting for global consensus is unrealistic, so jurisdictions must act even if work moves elsewhere.
• Concern about collective-action dynamics: stricter rules may just push development to laxer places, reducing local benefits without improving global safety.

Current vs Speculative Harms

• Several focus on present harms: algorithmic decision-making without appeal (account terminations, insurance, surveillance, self-checkout prosecutions), and deepfakes used in personal vendettas.
• Others emphasize speculative extinction-level risks; some respondents see coexistence with superintelligence as at least plausible and argue against panic-driven law.

Politics, Rhetoric, and Process

• Alliance for the Future is identified as a lobby group opposed to the bill and aligned with effective accelerationist views; some participants distrust its funding and framing.
• Many find the article’s “thoughtcrime”/“EA police” rhetoric exaggerated and seek less biased analyses (bill text, civil-liberties and industry critiques).
• Some want the bill revised rather than killed; others want it stopped outright and replaced with better-crafted, harm-focused laws.
• Several commenters report submitting formal feedback and urge Californians to contact their representatives with specific sections to keep, change, or remove.