Robust Traffic Signal Timing Control using Multiagent Twin Delayed Deep Deterministic Policy Gradients

Abstract

Traffic congestion is an omnipresent and serious problem that impacts people around the world on a daily basis. It requires solutions that can adapt to the changing traffic environments and reduce traffic congestion not only across local intersections but also across the global road network. Traditional traffic control strategies suffer from being too simplistic and moreover, they cannot scale to real-world dynamics. Multiagent reinforcement learning is being widely researched to develop intelligent transportation systems where the different intersections on a road network co-operate to ease vehicle delay and traffic congestion. Most of the literature on using Multiagent reinforcement learning methods for traffic signal control is focussed on applying multi-agent Q learning and discrete-action based control methods. In this paper, we propose traffic signal control using Multiagent Twin Delayed Deep Deterministic Policy Gradients (MATD3). The proposed control strategy is evaluated by exposing it to different time-varying traffic flows on simulation of road networks created on the traffic simulation platform SUMO. We observe that our method is robust to the different kinds of traffic flows and consistently outperforms the state-of-the-art counterparts by significantly reducing the average vehicle delay and queue length.