Validation of Image-Based Neural Network Controllers through Adaptive Stress Testing

Neural networks have become state-of-the-art for computer vision problems\nbecause of their ability to efficiently model complex functions from large\namounts of data. While neural networks can be shown to perform well empirically\nfor a variety of tasks, their performance is difficult to guarantee. Neural\nnetwork verification tools have been developed that can certify robustness with\nrespect to a given input image; however, for neural network systems used in\nclosed-loop controllers, robustness with respect to individual images does not\naddress multi-step properties of the neural network controller and its\nenvironment. Furthermore, neural network systems interacting in the physical\nworld and using natural images are operating in a black-box environment, making\nformal verification intractable. This work combines the adaptive stress testing\n(AST) framework with neural network verification tools to search for the most\nlikely sequence of image disturbances that cause the neural network controlled\nsystem to reach a failure. An autonomous aircraft taxi application is\npresented, and results show that the AST method finds failures with more likely\nimage disturbances than baseline methods. Further analysis of AST results\nrevealed an explainable cause of the failure, giving insight into the\nproblematic scenarios that should be addressed.\n