ROAD: Responsibility-Oriented Reward Design for Reinforcement Learning in Autonomous Driving

Reinforcement learning (RL) in autonomous driving employs a trial-and-error mechanism, enhancing robustness in unpredictable environments. However, crafting effective reward functions remains challenging, as conventional approaches rely heavily on manual design and demonstrate limited efficacy in complex scenarios. To address this issue, this study introduces a responsibility-oriented reward function that explicitly incorporates traffic regulations into the RL framework. Specifically, we first use a vision language model (VLM) to determine the liability in traffic collisions and propose reward signals. To mitigate VLM hallucination and ensure regulatory grounding, we propose a Traffic Regulation Knowledge Graph (TRKG) that structures unstructured traffic laws into a queryable ontology of driving scenarios and liability standards. This mechanism retrieves the precise regulatory context required to quantify accident responsibility (primary/shared/secondary), which is then used to modulate the crash penalty in the reward function. Building on these liability-informed signals, we construct a reward function and employ it to train the agent, thereby encouraging behavior that more faithfully adheres to traffic regulations. Experimental validations show that our approach achieves task success rates of 73.2% (intersection) and 54.0% (roundabout). Compared with the original policy, the success rate improves by +8.2 pp/+11.2 pp, while the ego vehicle's primary-liability share decreases by 13.5 pp/5.7 pp.