Stochastic Security: Adversarial Defense Using Long-Run Dynamics of Energy-Based Models

The vulnerability of deep networks to adversarial attacks is a central\nproblem for deep learning from the perspective of both cognition and security.\nThe current most successful defense method is to train a classifier using\nadversarial images created during learning. Another defense approach involves\ntransformation or purification of the original input to remove adversarial\nsignals before the image is classified. We focus on defending naturally-trained\nclassifiers using Markov Chain Monte Carlo (MCMC) sampling with an Energy-Based\nModel (EBM) for adversarial purification. In contrast to adversarial training,\nour approach is intended to secure pre-existing and highly vulnerable\nclassifiers.\n The memoryless behavior of long-run MCMC sampling will eventually remove\nadversarial signals, while metastable behavior preserves consistent appearance\nof MCMC samples after many steps to allow accurate long-run prediction.\nBalancing these factors can lead to effective purification and robust\nclassification. We evaluate adversarial defense with an EBM using the strongest\nknown attacks against purification. Our contributions are 1) an improved method\nfor training EBM's with realistic long-run MCMC samples, 2) an\nExpectation-Over-Transformation (EOT) defense that resolves theoretical\nambiguities for stochastic defenses and from which the EOT attack naturally\nfollows, and 3) state-of-the-art adversarial defense for naturally-trained\nclassifiers and competitive defense compared to adversarially-trained\nclassifiers on Cifar-10, SVHN, and Cifar-100. Code and pre-trained models are\navailable at https://github.com/point0bar1/ebm-defense.\n