OpenAI threatens to revoke o1 access for asking it about its chain of thought

Why OpenAI hides chain-of-thought (CoT)

• Many see the main motive as competitive: CoT traces would be prime training data for rivals, as happened when others trained on GPT‑4 outputs.
• Others take OpenAI’s stated reasons at face value: hidden CoT can be “unaligned,” improving reasoning because it isn’t RLHF‑lobotomized, and only the final answer is sanitized.
• Several comments stress PR/safety: internal steps may include racist, violent, or “thoughtcrime”-like content (e.g., explicit bomb instructions or offensive intermediate hypotheses) that would be a PR disaster if surfaced.
• Some argue this shows alignment harms reasoning, so they must keep the unaligned reasoning hidden.

AI safety, transparency, and user trust

• Many note the contradiction: OpenAI touts CoT as crucial for accuracy but prohibits users from inspecting it and threatens bans for trying.
• Critics say this undermines AI safety: humans can’t check logic, must treat o1 as a black box, and error detection gets harder.
• A few defend hiding CoT as analogous to not exposing people’s intrusive thoughts; only outcomes, not internal reasoning, should be judged.

Technical speculation about o1 and CoT

• Hypotheses include:
  • Self‑prompting / agentic loops over a base model (e.g., GPT‑4) tuned for this workflow.
  • Integration with formal methods, interpreters, or proof checkers, especially for math/code.
  • Hidden RAG over external code/text, possibly with dubious licensing.
• Some think CoT is just structured prompting and search over reasoning traces; others argue it reflects real “world models,” not just token parroting.
• Multiple comments note the visible “thought for N seconds” summary in the UI is not the real CoT, just an LLM‑generated digest.

Training data, RLHF, and human labor

• Widespread belief that CoT‑style supervision largely comes from humans: chat logs, expert contractors, and curated datasets, plus heavy reinforcement learning.
• Some see this as “just machine learning”; others emphasize the scale of hidden human labor and manual review.

Business model, moat, and competition

• Many frame secrecy as basic IP protection, not legally “anti‑competitive,” though it clashes with the original “open” charter.
• Others argue OpenAI leans into “dangerous, powerful, secret” rhetoric to attract funding and slow open‑source competitors.
• Several note competitors (Anthropic, Meta, open models) are close behind, so any revealed CoT could quickly erode OpenAI’s lead.

Billing and hidden computation

• Some are uneasy that users pay for CoT tokens they can’t see or audit, calling it a “hidden token money printer.”
• Others reply that usage‑based pricing is disclosed; if you dislike it, you can use local/open models instead.