Reinforcement learning and collective cooperation on higher-order networks

Abstract

Collective cooperation is essential for the survival and advancement of groups. However, current studies on evolutionary dynamics within higher-order networks often focus on learning and imitation rules, neglecting the potential impact of dynamic environments on individual strategic choices. To address this gap, we propose an approach that combines evolutionary game theory with reinforcement learning, presenting a Q-learning framework tailored for higher-order networks to investigate the influence of dynamic environments on group cooperation. More precisely, we iteratively update the Q-table and enable agents to autonomously determine whether to engage in the game, whereby active agents utilize social learning to adapt their strategies over time. By introducing varying rewards for inactive agents, our research reveals that moderate rewards prompt more defectors to exit the game, fostering the emergence and persistence of cooperation. Additionally, adjusting intrinsic parameters of reinforcement learning, such as employing a higher learning rate and a lower discount factor, can further promote the evolution of cooperation. We also examine the impact of group size and find that medium-sized groups provide a more favorable environment for collective cooperation on higher-order networks.