The agent harness belongs outside the sandbox

Original Article ↗ Hacker News Discussion ↗

Scope of “Harness” and “Agent”

  • Multiple definitions: harness as “everything around the model” (tools, memory, loop, sandboxing, steering), or more specifically the program that calls the LLM, routes tool calls, and manages conversation state.
  • “Agent” is also loosely used: anything from “LLM + tools” to an autonomous loop that can trigger its own further actions.
  • Some see “harness engineering” as just a new label for older ideas like orchestration/runtimes; others see it as clarifying that this layer is non-magical infrastructure.

Harness Inside vs Outside Sandbox

  • Core article claim: run the agent harness outside the code-execution sandbox; only forward risky operations (bash, user code) into the sandbox.
  • Supporters:
    • Harness is “just a loop” plus API calls; no need to sandbox a process that doesn’t execute untrusted code.
    • Easier to treat LLM as another API client reusing existing authN/authZ and multi-tenant scoping.
    • Keeps credentials and persistent state outside ephemeral workspaces; lets you share memories/skills across sessions.
  • Critics:
    • Many don’t trust harnesses much more than LLMs, given rapid change and complexity; they prefer harness-in-sandbox or multi-layer sandboxing.
    • Some argue the article’s title is misleading; you still want the harness sandboxed from the host, even if it’s “outside” the code sandbox.
    • Others say the model + harness together form a “lethal trifecta” and must be constrained by an external security layer.

Security, Sandboxes, and Secrets

  • Strong emphasis on keeping API keys and sensitive data out of environments where LLM-controlled code can read them.
  • Proposals for multiple sandboxes with different privilege levels; external hypervisor-like layer mediating access; short-lived, scoped credentials; MITM proxies for observability and policy.
  • Debate on whether API calls themselves need sandboxing vs being guarded by strict tool implementations and permission checks.

State, Memory, and Concurrency

  • One design: expose everything to the model as a filesystem-like interface, but back skills/memories by a database and workspace files by a sandbox FS.
  • Path-based routing (/workspace vs /memories) is combined with standard API-style auth and scoping.
  • Concurrent writes to shared memory/skills are unresolved; current patterns include blocking or rejecting conflicting writes instead of complex merging.

Alternative Architectures and Concerns

  • Some prefer giving agents full VMs or durable “computers” isolated from sensitive resources, rather than ephemeral sandboxes.
  • Others run both harness and tools inside k8s pods or containers for simplicity.
  • Skeptics see a buzzword race: harness designs change rapidly as models evolve, and no consensus architecture has emerged yet.