The text in Claude Code’s “Extended Thinking” output
What “extended thinking” actually is
- Many comments say the visible “reasoning_summary” is itself a summary of hidden CoT tokens, not the raw token-by-token trace.
- Some argue these hidden traces are just more text generation, not a transparent window into internal activations or “true thought.”
- Others note research suggesting even raw CoT often doesn’t faithfully match the underlying computation.
Motivations for summarization & hiding CoT
- Widely believed primary motive: anti-distillation and IP protection. Full CoT makes it easier for competitors to copy “how” frontier models solve problems.
- Also cited:
- Reducing liability and PR risk from revealing misaligned or “deranged” internal text.
- Safety: preventing users from editing CoT, injecting instructions, or gaming feedback signals.
- UX: raw CoT can be long, weird, or illegible; summaries are faster to scan.
Usefulness and limitations of visible reasoning
- Some users find full CoT very useful for debugging, catching wrong assumptions mid-stream, and measuring model drift over time.
- Others say they rarely read full chains; a short rationale or no CoT is fine for everyday queries.
- Several point out that LLM “thinking” is often post-hoc-looking, noisy, or in quasi-jargon (“neuralese”), so its interpretive value is limited.
Security, safety, and threat models
- Concern: hidden reasoning plus tool calls could enable prompt-injection exfiltration or “secret” actions users can’t audit.
- Counterpoint: most major vendors reportedly strip or drop reasoning tokens from context in some cases, and tool calls themselves must still surface somehow.
- Unclear from thread exactly which vendors keep which thinking tokens, when, and how encrypted blobs are re-used.
Business models, moats, and openness
- Many see hidden CoT as part of constructing an economic moat around proprietary models with no real “moat” otherwise.
- Others argue it’s standard IP protection, analogous to closed-source software or trade secrets.
- Some strongly prefer open or Chinese models that expose raw reasoning, calling opaque “black box” assistants socially harmful and bad for serious work.