Multi-Agent Reinforcement Learning for Connected and Automated Vehicles Control: Recent Advancements and Future Prospects

Connected and automated vehicles (CAVs) have emerged as a potential solution to the future challenges of developing safe, efficient, and eco-friendly transportation systems. However, CAV control presents significant challenges significant challenges due to the complexity of interconnectivity and coordination required among vehicles. Multi-agent reinforcement learning (MARL), which has shown notable advancements in addressing complex problems in autonomous driving, robotics, and human-vehicle interaction, emerges as a promising tool to enhance CAV capabilities. Despite its potential, there is a notable absence of current reviews on mainstream MARL algorithms for CAVs. To fill this gap, this paper offers a comprehensive review of MARL’s application in CAV control. The paper begins with an introduction to MARL, explaining its unique advantages in handling complex and multi-agent scenarios. It then presents a detailed survey of MARL applications across various control dimensions for CAVs, including critical scenarios such as platooning control, lane-changing, and unsignalized intersections. Additionally, the paper reviews prominent simulation platforms essential for developing and testing MARL algorithms for CAVs. Lastly, it examines the current challenges in deploying MARL for CAV control, including safety, communication, mixed traffic, and sim-to-real challenges. Potential solutions discussed include hierarchical MARL, decentralized MARL, adaptive interactions, and offline MARL. The work has been summarized in MARL_in_CAV_Control_Repository. Note to Practitioners—This paper explores the application of MARL for controlling CAVs in complex traffic scenarios such as platooning, lane-changing, and intersections. MARL provides an adaptive and decentralized control approach, offering advantages over traditional rule-based and optimization-based methods in handling dynamic interactions, improving traffic flow, enhancing safety, and optimizing fuel efficiency. The paper reviews state-of-the-art MARL algorithms and simulation platforms, serving as a resource for practitioners looking to implement these advanced control strategies. However, real-world deployment remains challenging due to communication reliability, real-time decision-making constraints, and the complexities of mixed traffic environments involving both automated and human-driven vehicles. Future research should focus on ensuring robust inter-agent communication, developing safety-aware MARL frameworks, and addressing the sim-to-real transfer gap. This paper provides insights to help practitioners bridge the gap between research and deployment, facilitating the development of more scalable, adaptive, and reliable CAV control.