The text in Claude Code’s “Extended Thinking” output
Major AI vendors are increasingly hiding their models’ raw “chain-of-thought” reasoning, instead exposing only short, summarized traces or encrypted blobs, as in Anthropic’s Claude Code “extended thinking” feature. Commenters argue this is primarily to prevent competitors from distilling proprietary reasoning processes, but it also reduces liability, masks potentially unsettling intermediate thoughts, and reinforces a black-box dynamic. Critics say the opacity makes it harder to debug, measure model drift, and trust agentic behavior, while others counter that chain-of-thought text is lossy and not a faithful window into how these systems actually compute their answers.
What “extended thinking” actually is
- Many comments say the visible “reasoning_summary” is itself a summary of hidden CoT tokens, not the raw token-by-token trace.
- Some argue these hidden traces are just more text generation, not a transparent window into internal activations or “true thought.”
- Others note research suggesting even raw CoT often doesn’t faithfully match the underlying computation.
Motivations for summarization & hiding CoT
- Widely believed primary motive: anti-distillation and IP protection. Full CoT makes it easier for competitors to copy “how” frontier models solve problems.
- Also cited:
- Reducing liability and PR risk from revealing misaligned or “deranged” internal text.
- Safety: preventing users from editing CoT, injecting instructions, or gaming feedback signals.
- UX: raw CoT can be long, weird, or illegible; summaries are faster to scan.
Usefulness and limitations of visible reasoning
- Some users find full CoT very useful for debugging, catching wrong assumptions mid-stream, and measuring model drift over time.
- Others say they rarely read full chains; a short rationale or no CoT is fine for everyday queries.
- Several point out that LLM “thinking” is often post-hoc-looking, noisy, or in quasi-jargon (“neuralese”), so its interpretive value is limited.
Security, safety, and threat models
- Concern: hidden reasoning plus tool calls could enable prompt-injection exfiltration or “secret” actions users can’t audit.
- Counterpoint: most major vendors reportedly strip or drop reasoning tokens from context in some cases, and tool calls themselves must still surface somehow.
- Unclear from thread exactly which vendors keep which thinking tokens, when, and how encrypted blobs are re-used.
Business models, moats, and openness
- Many see hidden CoT as part of constructing an economic moat around proprietary models with no real “moat” otherwise.
- Others argue it’s standard IP protection, analogous to closed-source software or trade secrets.
- Some strongly prefer open or Chinese models that expose raw reasoning, calling opaque “black box” assistants socially harmful and bad for serious work.