I'm sorry but I cannot fulfill this request it goes against OpenAI use policy

Original Article ↗ Hacker News Discussion ↗

AI-generated product listings on Amazon are starting to surface with ChatGPT refusal messages left verbatim as titles and descriptions, revealing that some sellers are bulk-generating copy and publishing it without any human review. Commenters link this to a wider flood of low-quality, often drop-shipped goods, fake brands, and spammy text that make Amazon feel increasingly like AliExpress and undermine trust in search results, reviews, and even product authenticity. The episode is used to illustrate how unattended use of large language models can fail in unpredictable ways at scale, and how weak marketplace oversight incentivizes volume over quality.

AI-generated Amazon listings

  • The linked product was an Amazon listing whose title was a ChatGPT-style refusal about OpenAI’s use policy, likely pasted verbatim as the product name.
  • Commenters quickly found many similar listings: titles beginning with apologies, references to “OpenAI use policy,” or generic boilerplate like “[product] is crafted with the highest quality materials…”.
  • A “FOPEAS” brand and other nonsense brands had entire catalogs of AI-written, largely content-free descriptions, and mismatched images (e.g., a cat mirror described like an SSD, furniture with absurd dimensions).

Why this is happening

  • Consensus: these are mostly drop-shipping or arbitrage operations, often from China, auto‑creating thousands of listings from scraped images and competitor titles.
  • LLMs are used to:
    • Translate into English.
    • Rewrite competitor/trademarked titles.
    • Generate generic “marketing” copy.
  • When ChatGPT hits guardrails (trademarks, religion, sexual wording, etc.), the refusal text sometimes gets blindly pasted as the title or description.

LLM behavior and safety

  • People note that LLM refusals can be triggered by seemingly innocent prompts (e.g., religious calendars, colors like “black/brown dresser”).
  • Debate over comparing LLMs to human employees:
    • One side: both make mistakes; this is just a new flavor of copy‑paste error.
    • Other side: LLM failures are less predictable and can scale orders of magnitude faster (“FUPS” – fuckups per second), with no understanding or accountability.

Mitigation ideas

  • Heuristics: flag titles/descriptions containing “sorry”/“apologize” or known refusal patterns for human review.
  • Stronger integration: require JSON or function-call outputs with explicit error fields; use OpenAI’s moderation API; run a second (cheap) model to classify whether a text is an apology vs. a valid description; embedding-based similarity to known refusal messages.
  • Skepticism: such checks cost money, can be gamed, and low-effort spammers plus indifferent marketplaces may not bother.

Amazon marketplace quality and search

  • Many describe Amazon as increasingly unusable:
    • Flood of near-identical white‑label products under random brands.
    • Misleading titles/images, counterfeit risks, and obviously AI-written reviews.
    • Search dominated by ads and “sponsored” junk rather than the exact item queried.
  • Others report largely smooth experiences when:
    • They already know the exact brand/model.
    • They stick to certain categories (e.g., books) or “sold by Amazon” items.
  • Broader worry: AI‑generated spam on Amazon, social media, and review sites accelerates a general “enshittification” of the internet and erodes trust in online information and marketplaces.