Hierarchical Dynamic Routing in Complex Networks via Topologically-decoupled and Cooperative Reinforcement Learning Agents

The transport capacity of a communication network can be characterized by the transition from a free-flow state to a congested state. Here, we propose a dynamic routing strategy in complex networks based on hierarchical bypass selections. The routing decisions are made by the reinforcement learning agents implemented at selected nodes with high betweenness centrality. The learning processes of the agents are decoupled from each other due to the degeneracy of their bypasses. Through interactions mediated by the underlying traffic dynamics, the agents act cooperatively, and coherent actions arise spontaneously. With only a small number of agents, the transport capacities are significantly improved, including in real-world Internet networks at the router level and the autonomous system level. Our strategy is also resilient to link removals.