The session you cannot take with you
AI platforms are increasingly hiding key parts of model interactions—such as “thinking” tokens, tool calls, and server-side compaction—inside opaque or encrypted blobs that users cannot inspect, move, or replay. Commenters worry this erodes auditability, makes it hard to switch models or providers mid-project, and sets up classic vendor lock-in even as models themselves become commoditized. Many see open-weight models, provider-agnostic harnesses, and local storage of full session context as the main counterweights to this trend.
Concerns about Hidden State & Session Lock-In
- Many comments endorse the article’s core worry: providers increasingly hide parts of a session (reasoning tokens, subagent messages, server-side tools, compaction) and tie them to a single vendor.
- This breaks inspection, export, replay, and audit, and makes it hard to move a “live” session to another model or provider.
- Several see this as deliberate moat-building and a setup for future “enshittification” once users are sufficiently locked in.
Technical & Cost Considerations
- Discussion of KV caches: long contexts are expensive; cache misses force replaying history. Some argue users shouldn’t have to resend full transcripts; others point out that cache management is inherently costly.
- Complaints about being charged for “thinking” tokens that are neither visible nor verifiable.
- Some note that OpenAI’s newer APIs (Responses, reasoning effort levels, server-side compaction) likely embed proprietary orchestration logic that’s hard to reproduce client-side.
Security, Safety, and Reasoning Visibility
- One camp cites “role confusion” / prompt-injection research and distillation risks as real reasons to hide reasoning.
- Another camp argues hiding traces only weakly mitigates these attacks; signing or validating traces and better training would be more honest than encrypting them away.
Open Models, Local Hosting, and Harnesses
- Strong current toward open-weight and local models: seen as persistent, inspectable, and not subject to rug-pulls.
- Several users already route between multiple models (including Chinese models) and value being able to switch mid-session, even at some cache cost.
- Tools and harnesses (e.g., terminal UIs, MCP-based systems, custom routers) are being built specifically to keep sessions portable and logs client-side.
User Workarounds & Alternative Patterns
- Some store their own transcripts, build search over past sessions, or use git-like archives for context.
- Others push important state into separate artifacts: notes, “basis” documents (intent, design, blueprints), tickets, or task databases, so new sessions or models can pick up work without needing opaque provider logs.
Skepticism and Counterpoints
- A minority says session portability is overvalued: conversations are noisy, context windows are finite, and the real asset is the resulting code or documents, which can be moved freely.
- Some emphasize that sophisticated account systems and hosted tools are hard to build; delegating to providers (including “Sign in with Google” and server-side tools) is often practical, even if it increases dependency.
Standards, Middleware & Market Dynamics
- Multiple comments call for open, provider-neutral session formats or APIs (possibly via MCP or neutral routers) to slow fragmentation and preserve portability, even if hidden reasoning cannot be recovered.
- Others predict that as APIs diverge and hidden state grows, full session portability will be increasingly infeasible, making careful harness design or treating provider switches as explicit “migrations” essential.
Meta: AI-Writing Accusations
- Some tools flagged the article as AI-generated; others dispute this and criticize detector reliability.
- The author (per thread) describes a workflow where an LLM helps with structuring and copy-editing but denies having it write the piece outright, highlighting how blurred “AI-assisted” vs “AI-generated” has become.