Efficient Policy Transfer in Large-Scale Traffic Light Control via Multi-Agent Hierarchical Reinforcement Learning

Abstract

Multi-agent reinforcement learning (MARL) is increasingly being used for traffic light control in traffic networks. However, applying MARL to large-scale traffic light control faces challenges due to the time-consuming and resource-intensive nature of learning in such environments. This paper addresses this challenge by leveraging transfer learning, utilizing policies trained in small-scale environments to effectively control large-scale traffic light systems. To enhance the zero-shot transferability of trained policies, we introduce communication channels and sub-task decomposition, while ensuring transferability by sharing all neural network parameters. In our experiments, we assess the zero-shot transferability of policies trained in different small-scale traffic networks, both with and without sub-task decomposition. The results demonstrate the significance of establishing a hierarchical structure through sub-task decomposition for zero-shot transfer.