Evaluating Robustness over High Level Driving Instruction for Autonomous Driving

In recent years, we have witnessed increasingly high performance in the field\nof autonomous end-to-end driving. In particular, more and more research is\nbeing done on driving in urban environments, where the car has to follow high\nlevel commands to navigate. However, few evaluations are made on the ability of\nthese agents to react in an unexpected situation. Specifically, no evaluations\nare conducted on the robustness of driving agents in the event of a bad\nhigh-level command. We propose here an evaluation method, namely a benchmark\nthat allows to assess the robustness of an agent, and to appreciate its\nunderstanding of the environment through its ability to keep a safe behavior,\nregardless of the instruction.\n