Agents don't need memory, they need documentation

Original Article ↗ Hacker News Discussion ↗

AI engineers are rethinking how software agents should “remember” past work, arguing that opaque memory stores and vector databases often create stale, unstructured cruft, while explicit, versioned documentation, ADRs, and repo-local markdown can give agents clearer, auditable guidance. Many favor treating agents like junior developers: constrain them with AGENTS.md, lint rules, tests, and well-structured folders, then have the agents themselves help keep docs in sync with code. Others counter that documentation also rots, so practical systems will likely need a mix of curated docs, lightweight memory, strong context management, and deterministic feedback rather than a single magic memory layer.

Role of “Memory” vs Documentation

  • Many agree naive memory plugins (append-only snippets, opaque stores) quickly become stale, noisy, and hard to curate.
  • Several argue that well-structured documentation (AGENTS.md, ADRs, specs, decisions) is a better “memory” layer: queryable, auditable, and updateable.
  • Others counter that docs also rot, can be fragmented, and rarely reflect reality unless someone (often the agent) is explicitly tasked with keeping them synced.
  • Some see the distinction as mostly semantic: documentation is just a structured, externalized memory system.

Concrete Practices & Repo Structures

  • Popular setups:
    • Minimal AGENTS.md pointing to focused docs.
    • Feature/issue-based markdown folders; decisions/, plans/, knowledge/, workbench/ trees.
    • Index/catalog files that describe each doc and when to read it (“read_if” rules).
  • Several tools and patterns mentioned: markdown “brains” in repos, CLI tools to manage docs, S3- or git-backed doc stores, internal skill marketplaces, and custom memory MCPs.

Agent Behavior, Control, and Drift

  • Repeated complaint: agents ignore sophisticated memory/ADR/principle systems over time and fall back to grep + quick Python scripts.
  • Strong desire for enforceable rules (e.g., “use jq, not ad‑hoc Python”) via hooks, lint rules with prescriptive messages, and deterministic runtime monitors.
  • Some use interview-like tools or principle/version tokens in comments to steer agents and trigger re-evaluation when principles change.

Testing, Verification, and Feedback

  • Mixed experiences with TDD loops for agents: tests ensure regression protection and communicate intent, but can degenerate into box-ticking.
  • Alternative: custom “verify” skills and lint-based feedback tailored to project-specific code smells and architectural principles.
  • Deterministic feedback (similar to RL-style scoring) is viewed as powerful but hard to operationalize.

Context Management & Evaluation

  • Strong focus on explicit context management: dynamic pruning, per-folder sessions, monorepos with clear structure.
  • Skepticism about giant AI-generated doc piles; concern they become unreadable “AI slop” and don’t measurably help.
  • Calls for real A/B evals of documentation-heavy vs minimal setups; current evidence in the thread is anecdotal.

Safety & Multi‑Agent Concerns

  • Some run many agents 24/7 with inter-agent communication and report good results; others express fear about easy large-scale deployment, misalignment, and financial-system risk.
  • Overall, participants agree that agent systems need transparent, inspectable state more than opaque long-term “memories.”