Optimizing Urban Route Choice for Autonomous Vehicles using Multi-Agent Reinforcement Learning

Autonomous vehicles (AVs) have already been introduced in some cities worldwide, and understanding how they could effectively learn optimal routing strategies, that is, routes to travel from an origin to a destination point in a traffic network, is essential. In my Ph.D., I use Multi-Agent Reinforcement Learning (MARL) to model AV routing decisions in a microscopic setting, shared with human drivers, where AVs learn to select routes that minimize their costs (e.g., travel time) given the currently observed state of the traffic network. This paper provides a brief overview of my work and focuses on two main contributions. First, we show that when multiple independent learning AVs simultaneously learn routing strategies in a traffic network, they may destabilize it by increasing human and AV travel times, as the state-of-the-art MARL algorithms used to train their routing decisions require long training iterations to converge to the optimal solution. Second, we study a solution to this problem by introducing a social component into the AV's reward functions, building on prior work on socially aware incentives in multi-agent systems. We show that this can accelerate convergence to the system-optimal solution and benefit individual agents in this routing game.