OpenAI threatens to revoke o1 access for asking it about its chain of thought

Original Article ↗ Hacker News Discussion ↗

OpenAI is warning users it may revoke access to its new o1 reasoning model if they probe for its hidden “chain-of-thought,” prompting concerns about secrecy and control over how the system works. Commenters debate whether the real motivation is safety (avoiding unfiltered, potentially harmful or politically sensitive intermediate reasoning), preserving competitive advantage, or preventing others from training rival models on these traces. Many see a broader pattern: a company that benefited from scraping public data now tightly locking down its own outputs, raising worries about transparency, auditability, and dependence on a closed AI black box.

Why OpenAI hides chain-of-thought (CoT)

  • Many see the main motive as competitive: CoT traces would be prime training data for rivals, as happened when others trained on GPT‑4 outputs.
  • Others take OpenAI’s stated reasons at face value: hidden CoT can be “unaligned,” improving reasoning because it isn’t RLHF‑lobotomized, and only the final answer is sanitized.
  • Several comments stress PR/safety: internal steps may include racist, violent, or “thoughtcrime”-like content (e.g., explicit bomb instructions or offensive intermediate hypotheses) that would be a PR disaster if surfaced.
  • Some argue this shows alignment harms reasoning, so they must keep the unaligned reasoning hidden.

AI safety, transparency, and user trust

  • Many note the contradiction: OpenAI touts CoT as crucial for accuracy but prohibits users from inspecting it and threatens bans for trying.
  • Critics say this undermines AI safety: humans can’t check logic, must treat o1 as a black box, and error detection gets harder.
  • A few defend hiding CoT as analogous to not exposing people’s intrusive thoughts; only outcomes, not internal reasoning, should be judged.

Technical speculation about o1 and CoT

  • Hypotheses include:
    • Self‑prompting / agentic loops over a base model (e.g., GPT‑4) tuned for this workflow.
    • Integration with formal methods, interpreters, or proof checkers, especially for math/code.
    • Hidden RAG over external code/text, possibly with dubious licensing.
  • Some think CoT is just structured prompting and search over reasoning traces; others argue it reflects real “world models,” not just token parroting.
  • Multiple comments note the visible “thought for N seconds” summary in the UI is not the real CoT, just an LLM‑generated digest.

Training data, RLHF, and human labor

  • Widespread belief that CoT‑style supervision largely comes from humans: chat logs, expert contractors, and curated datasets, plus heavy reinforcement learning.
  • Some see this as “just machine learning”; others emphasize the scale of hidden human labor and manual review.

Business model, moat, and competition

  • Many frame secrecy as basic IP protection, not legally “anti‑competitive,” though it clashes with the original “open” charter.
  • Others argue OpenAI leans into “dangerous, powerful, secret” rhetoric to attract funding and slow open‑source competitors.
  • Several note competitors (Anthropic, Meta, open models) are close behind, so any revealed CoT could quickly erode OpenAI’s lead.

Billing and hidden computation

  • Some are uneasy that users pay for CoT tokens they can’t see or audit, calling it a “hidden token money printer.”
  • Others reply that usage‑based pricing is disclosed; if you dislike it, you can use local/open models instead.