Graph Neural Networks and Reinforcement Learning for Behavior Generation in Semantic Environments

Most reinforcement learning approaches used in behavior generation utilize\nvectorial information as input. However, this requires the network to have a\npre-defined input-size -- in semantic environments this means assuming the\nmaximum number of vehicles. Additionally, this vectorial representation is not\ninvariant to the order and number of vehicles. To mitigate the above-stated\ndisadvantages, we propose combining graph neural networks with actor-critic\nreinforcement learning. As graph neural networks apply the same network to\nevery vehicle and aggregate incoming edge information, they are invariant to\nthe number and order of vehicles. This makes them ideal candidates to be used\nas networks in semantic environments -- environments consisting of objects\nlists. Graph neural networks exhibit some other advantages that make them\nfavorable to be used in semantic environments. The relational information is\nexplicitly given and does not have to be inferred. Moreover, graph neural\nnetworks propagate information through the network and can gather higher-degree\ninformation. We demonstrate our approach using a highway lane-change scenario\nand compare the performance of graph neural networks to conventional ones. We\nshow that graph neural networks are capable of handling scenarios with a\nvarying number and order of vehicles during training and application.\n