The agent harness belongs outside the sandbox

Original Article ↗ Hacker News Discussion ↗

Architects of AI “agents” are rethinking where the agent harness — the control loop that calls models, tools, and sandboxes — should run for both safety and practicality. Many argue it should live outside the code-execution sandbox, treating the harness like any other backend service while routing risky operations (e.g., bash on user code) into tightly isolated environments and keeping secrets out of the model’s reach. Others counter that rapidly evolving, semi‑trusted harnesses and models need multiple layers of sandboxing and fine-grained auth, highlighting unresolved challenges around access control, shared memory, concurrency, and emerging best practices for secure agent design.

Scope of “Harness” and “Agent”

  • Multiple definitions: harness as “everything around the model” (tools, memory, loop, sandboxing, steering), or more specifically the program that calls the LLM, routes tool calls, and manages conversation state.
  • “Agent” is also loosely used: anything from “LLM + tools” to an autonomous loop that can trigger its own further actions.
  • Some see “harness engineering” as just a new label for older ideas like orchestration/runtimes; others see it as clarifying that this layer is non-magical infrastructure.

Harness Inside vs Outside Sandbox

  • Core article claim: run the agent harness outside the code-execution sandbox; only forward risky operations (bash, user code) into the sandbox.
  • Supporters:
    • Harness is “just a loop” plus API calls; no need to sandbox a process that doesn’t execute untrusted code.
    • Easier to treat LLM as another API client reusing existing authN/authZ and multi-tenant scoping.
    • Keeps credentials and persistent state outside ephemeral workspaces; lets you share memories/skills across sessions.
  • Critics:
    • Many don’t trust harnesses much more than LLMs, given rapid change and complexity; they prefer harness-in-sandbox or multi-layer sandboxing.
    • Some argue the article’s title is misleading; you still want the harness sandboxed from the host, even if it’s “outside” the code sandbox.
    • Others say the model + harness together form a “lethal trifecta” and must be constrained by an external security layer.

Security, Sandboxes, and Secrets

  • Strong emphasis on keeping API keys and sensitive data out of environments where LLM-controlled code can read them.
  • Proposals for multiple sandboxes with different privilege levels; external hypervisor-like layer mediating access; short-lived, scoped credentials; MITM proxies for observability and policy.
  • Debate on whether API calls themselves need sandboxing vs being guarded by strict tool implementations and permission checks.

State, Memory, and Concurrency

  • One design: expose everything to the model as a filesystem-like interface, but back skills/memories by a database and workspace files by a sandbox FS.
  • Path-based routing (/workspace vs /memories) is combined with standard API-style auth and scoping.
  • Concurrent writes to shared memory/skills are unresolved; current patterns include blocking or rejecting conflicting writes instead of complex merging.

Alternative Architectures and Concerns

  • Some prefer giving agents full VMs or durable “computers” isolated from sensitive resources, rather than ephemeral sandboxes.
  • Others run both harness and tools inside k8s pods or containers for simplicity.
  • Skeptics see a buzzword race: harness designs change rapidly as models evolve, and no consensus architecture has emerged yet.