SAFETY ANALYSIS FRAMEWORK

Techniques for determining a safety metric associated with a vehicle controller are discussed herein. To determine whether a complex system (which may be uninspectable) is able to operate safely, various operating regimes (scenarios) can be identified based on operating data and associated with a scenario parameter to be adjusted. To validate safe operation of such a system, a scenario may be identified for inspection. Error metrics of a subsystem of the system can be quantified. The error metrics, in addition to stochastic errors of other systems/subsystems can be introduced to the scenario. The scenario parameter may also be perturbed. Any multitude of such perturbations can be instantiated in a simulation to test, for example, a vehicle controller. A safety metric associated with the vehicle controller can be determined based on the simulation, as well as causes for any failures.