The short leash AI coding method for beating Fable
Overall reaction to the “short leash” method
- Many see the workflow (close supervision, stepwise permissions, human-in-the-loop) as basically how careful developers already use AI for important code.
- Others think it’s overcautious “micromanagement,” slowing things down vs letting agents work more autonomously in a sandbox and just reviewing diffs/PRs.
Autonomy vs oversight in AI coding
- Strong camp for tight control:
- Argues models still hallucinate, ignore instructions, and introduce subtle bugs and security issues (e.g., IDORs, bad concurrency, poor DB access patterns).
- Claims it’s “humanly impossible” to fully trust autonomous agents, especially in regulated or high‑risk systems; humans remain accountable.
- Emphasizes that frequent review keeps a workable mental model of the codebase.
- Strong camp for longer leashes:
- Runs agents in VMs/containers or separate users with restricted keys, then lets them “yolo” within that boundary.
- Treats AI like a very fast junior–mid engineer: let it build features, then do human review and massage the result.
- Some report good results on large production codebases with mid‑sized tasks handled autonomously.
Harnesses, tooling, and safety patterns
- Common patterns:
- Sandboxed environments (containers, VMs, dedicated users, Nix shells).
- Tool allowlists / MCPs with only safe commands, or bash command allowlists plus hooks.
- Auto‑mode / semi‑auto workflows to reduce constant permission prompts.
- Disagreement:
- Some say “never skip permissions” outside a sandbox; others say strict sandbox + full autonomy is better than endless approvals.
Code quality, tests, and review
- Broad agreement that:
- AI-generated code must be reviewed; tests and CI are crucial.
- Multiple AI passes (planning, implementation, review, security scan) plus human review work better than single-shot generation.
- Using one model to review another’s code can catch more issues than self‑review.
Model capabilities and “intelligence”
- Split views on capability:
- Some say current frontier models rival or exceed many staff engineers on many tasks; others find them weak on hard problems (complex concurrency, large multi-service systems).
- Long philosophical subthread on:
- “Next‑token predictor” vs real intelligence; human vs model reasoning; post‑hoc justifications and “lying.”
- No consensus; several note that practical reliability, not labels, is what matters.
Impact on software practice
- Some fear a future where humans no longer understand codebases and AI “drives” them; others see that as inevitable for many domains but not for safety‑critical systems.
- Several stress that AI is most effective when humans focus on architecture, specs, and tests, and let models do more of the rote implementation, with reviews kept tight enough to maintain understanding.