Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles

Extracting interesting scenarios from real-world data as well as generating\nfailure cases is important for the development and testing of autonomous\nsystems. We propose efficient mechanisms to both characterize and generate\ntesting scenarios using a state-of-the-art driving simulator. For any scenario,\nour method generates a set of possible driving paths and identifies all the\npossible safe driving trajectories that can be taken starting at different\ntimes, to compute metrics that quantify the complexity of the scenario. We use\nour method to characterize real driving data from the Next Generation\nSimulation (NGSIM) project, as well as adversarial scenarios generated in\nsimulation. We rank the scenarios by defining metrics based on the complexity\nof avoiding accidents and provide insights into how the AV could have minimized\nthe probability of incurring an accident. We demonstrate a strong correlation\nbetween the proposed metrics and human intuition.\n