We ran AI agent swarms in Minecraft and Mindustry
Experiments from Y Combinator Labs that unleashed swarms of LLM-based agents in Minecraft and Mindustry highlight both the promise and current limits of AI in complex, real-time game environments. Commenters note that while the models can reason, they struggle with latency, coordination, and basic execution, often devolving into comical bureaucracy rather than effective play. The thread uses these results to probe broader questions about whether large reasoning models, agent swarms, and code-writing harnesses are the right architecture for interactive worlds, and what hybrid designs (macro LLM + micro deterministic control) might work better.
Overall Reaction to the Experiments
- Many found the demos funny, charming, and technically impressive, but also highlighted serious limitations.
- Several commenters felt the agents “played” Minecraft in a turn-based, sluggish way, making the civilization-building feel bureaucratic and ineffective.
- Some were more impressed by the embedded Minecraft viewer than the AI itself.
Latency, Control, and Architecture
- A recurring theme: big LLMs are a poor fit for real‑time action games due to high latency and token-by-token computation.
- Commenters argue these tasks are more like robotics: need tight, continuous feedback loops for movement and combat.
- Suggested pattern: use deterministic scripts or small fast models for micro actions (movement, fighting, “click spam loops”), and LLMs only for higher-level planning.
- Existing setups that have a mod/bot for sub-second actions, a separate macro decision server, and LLMs just for policy/design were cited as more practical.
Behavior, Bureaucracy, and “Middle Manager” Vibes
- Multi-agent runs often devolved into elaborate coordination, committees, and process for trivial issues (e.g., roles for single bone meal, detailed postmortems for minor mishaps).
- Commenters linked this to genre (“build a civilization” Minecraft content), human training data, and general multi-agent tendencies toward bikeshedding and bureaucracy.
- Logs were widely read as comedy gold: agents acting like anxious senior engineers or panicked middle managers.
Model Differences and Reasoning Limits
- Some felt the need to write JavaScript/Python action scripts exposes LLMs’ difficulty with continuous logical control over the environment.
- Others countered that having the model generate deterministic helper code is actually a strength and a natural way to reduce error and cost.
- Observations from the experiments:
- GPT-based agents appeared more aggressive, exploratory, and better at diagnosing tool bugs and sharing workarounds.
- Claude-based agents tended to be more cautious, defensive, and “Slack-like” in communication, sometimes concluding tasks were impossible when tools were buggy.
Swarms vs. Simpler Systems
- Several questioned the value of 50-agent “swarms” when all agents share the same underlying model.
- Critiques: redundant roles, token waste, emergent office politics, minimal real gain over a single well-instructed agent.
- Defenders of multi-agent systems argued they can help when agents have genuinely different tools, knowledge, and scopes, or when problems are highly parallelizable.