AI Art Generators Can Be Fooled into Making NSFW Images

Original Article ↗ Hacker News Discussion ↗

Researchers have shown that AI image generators with built‑in safety filters can still be prompted into producing NSFW or otherwise restricted content, including via nonsense “secret” words that bypass both input and output checks. Commenters debate whether such controls are desirable or merely corporate and regulatory “security theater,” raising broader issues around censorship, child safety, legal risk, advertiser and app‑store pressures, and the limits of current alignment techniques. Many note that these vulnerabilities matter beyond porn: if systems can’t reliably block nudity, they likely also can’t be fully trusted in higher‑stakes applications.

What “safety” and NSFW mean

  • Several comments clarify that “safety” here means NSFW, “trust and safety,” and “brand safety,” not physical danger.
  • Some argue nudity isn’t inherently unsafe; “unsafe” mainly refers to getting fired or violating social norms.
  • Others push back that people can reasonably find public nudity offensive and want boundaries.

Filtering architectures and their flaws

  • Many assume major image generators use both prompt filters and output filters.
  • Users report:
    • Prompts pass input filters but outputs get blocked, including false positives (e.g., beige clothes on younger people, the word “breast pocket”).
    • Different platforms (Bing, DALL·E, OpenAI APIs) behave similarly, sometimes charging even when the result is blocked.
  • Smaller/indie APIs often have no or minimal filtering.

Adversarial prompts and “nonsense” tokens

  • The paper’s contribution is seen as algorithmically generating adversarial prompts that bypass both prompt and image-based NSFW classifiers, including via “nonsense” tokens.
  • Some are interested in how these token embeddings emerge and why they map to NSFW concepts.
  • Others note that this undermines broader alignment claims: if you can’t reliably block porn, you likely can’t reliably block more harmful behavior.

Censorship vs capability debate

  • One camp: limit models (80% functionality) to avoid misuse and satisfy platforms, advertisers, app stores, and regulators.
  • Another camp: prefer full capability and place responsibility on users; see restrictions as paternalistic, puritanical, or “security theater.”
  • Local / self-hosted models are framed as the “Linux vs Windows” route to bypass corporate filters.

Children, products, and liability

  • Some emphasize protecting kids and schools: a “safe” generator is needed so educational apps don’t unexpectedly show porn.
  • Others argue kids will always find a way to see NSFW material, and over-sanitizing is mainly about corporate PR, not real harm.

Legal and ethical edge cases

  • Distinction raised between NSFW in general and “NSFLE” (illegal content like synthetic child abuse), which can have serious legal consequences even if AI-generated.
  • Debate on whether such purely artificial illegal content should be criminalized is marked as contentious and unresolved.

Broader reactions to the research and coverage

  • Some see the study as obvious or trivial (“of course you can break constraints if you try hard”).
  • Others compare it to hacking or coercing self-driving cars: necessary stress-testing, not pointless.
  • A few criticize the IEEE article’s explanation of LLMs as oversimplified, and the focus on nudity as somewhat juvenile given more serious alignment issues.