Cultural Evolution of Cooperation Among LLM Agents

LLM–LLM Conversations and “Cooperation”

• People report running models against each other (e.g., via local tools) and observing endless polite “goodbye” exchanges.
• Many argue this is just next-token prediction over human-style data, not genuine cooperation or cultural evolution.
• Some note that models are usually forced by the surrounding code to always reply; they lack a true “end of conversation” state.

End-of-Conversation and Agent Framing

• Suggestions: introduce explicit “[silence]” or “[end-conversation]” tokens, or treat “end chat” as a tool the model can call.
• Others counter that models are optimized to always respond and don’t “decide to obey”; they just continue the script.
• A popular framing: LLM interactions are like movie scripts where a writer extends text containing fictional agents; “cooperation” is a property of the story, not the underlying program.

Mimicry, Reasoning, and Consciousness

• One side: LLMs just mimic patterns; there’s no new culture unless they create their own slang or styles, not found in training data.
• Counterpoint: humans also learn from examples; cultural evolution only appears in larger interacting populations.
• Debate extends to whether LLMs “reason” or could be conscious, with comparisons to ordinary software and even rocks or chemical reactions. No consensus is reached.

Security, Prompt Injection, and Token Trust

• Prompt-injection is framed as hard because the model sees one undifferentiated text stream.
• Proposed fix: “colored” tokens (trusted vs untrusted) or stronger treatment of system messages.
• Critics note annotation and conceptual complexity, arguing we should mostly reuse standard security principles (least privilege, user prompts, defense in depth).

Paper’s Methodology and Claims

• Several see the work as interesting for game-theoretic / evolutionary analysis (e.g., Donor Game and indirect reciprocity).
• Skeptics say parameter choices are arbitrary; results may be artifacts of the setup or output style (more detailed vs vague strategies) rather than deep training biases.
• Concerns about weak ablation/sensitivity analysis and overreaching language around “cultural evolution” and “falsifying” broad claims about LLM cooperation.

LLMs as Experimental Agents

• Some are excited about using LLM agents for large-scale social or game-theoretic simulations, potentially aiding sociology and theory-of-mind research.
• Others caution that cultural effects in LLMs are transient (lost once context is gone), so “evolution” may be very different from human culture.