Cultural Evolution of Cooperation among LLM Agents

Large language models (LLMs) provide a compelling foundation for building\ngenerally-capable AI agents. These agents may soon be deployed at scale in the\nreal world, representing the interests of individual humans (e.g., AI\nassistants) or groups of humans (e.g., AI-accelerated corporations). At\npresent, relatively little is known about the dynamics of multiple LLM agents\ninteracting over many generations of iterative deployment. In this paper, we\nexamine whether a "society" of LLM agents can learn mutually beneficial social\nnorms in the face of incentives to defect, a distinctive feature of human\nsociality that is arguably crucial to the success of civilization. In\nparticular, we study the evolution of indirect reciprocity across generations\nof LLM agents playing a classic iterated Donor Game in which agents can observe\nthe recent behavior of their peers. We find that the evolution of cooperation\ndiffers markedly across base models, with societies of Claude 3.5 Sonnet agents\nachieving significantly higher average scores than Gemini 1.5 Flash, which, in\nturn, outperforms GPT-4o. Further, Claude 3.5 Sonnet can make use of an\nadditional mechanism for costly punishment to achieve yet higher scores, while\nGemini 1.5 Flash and GPT-4o fail to do so. For each model class, we also\nobserve variation in emergent behavior across random seeds, suggesting an\nunderstudied sensitive dependence on initial conditions. We suggest that our\nevaluation regime could inspire an inexpensive and informative new class of LLM\nbenchmarks, focussed on the implications of LLM agent deployment for the\ncooperative infrastructure of society.\n