Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach

The robustness of neural networks to adversarial examples has received great\nattention due to security implications. Despite various attack approaches to\ncrafting visually imperceptible adversarial examples, little has been developed\ntowards a comprehensive measure of robustness. In this paper, we provide a\ntheoretical justification for converting robustness analysis into a local\nLipschitz constant estimation problem, and propose to use the Extreme Value\nTheory for efficient evaluation. Our analysis yields a novel robustness metric\ncalled CLEVER, which is short for Cross Lipschitz Extreme Value for nEtwork\nRobustness. The proposed CLEVER score is attack-agnostic and computationally\nfeasible for large neural networks. Experimental results on various networks,\nincluding ResNet, Inception-v3 and MobileNet, show that (i) CLEVER is aligned\nwith the robustness indication measured by the $\\ell_2$ and $\\ell_\\infty$ norms\nof adversarial examples from powerful attacks, and (ii) defended networks using\ndefensive distillation or bounded ReLU indeed achieve better CLEVER scores. To\nthe best of our knowledge, CLEVER is the first attack-independent robustness\nmetric that can be applied to any neural network classifier.\n