When Should You Defend Your Classifier -- A Game-theoretical Analysis of Countermeasures against Adversarial Examples

Adversarial machine learning, i.e., increasing the robustness of machine\nlearning algorithms against so-called adversarial examples, is now an\nestablished field. Yet, newly proposed methods are evaluated and compared under\nunrealistic scenarios where costs for adversary and defender are not considered\nand either all samples or no samples are adversarially perturbed. We scrutinize\nthese assumptions and propose the advanced adversarial classification game,\nwhich incorporates all relevant parameters of an adversary and a defender.\nEspecially, we take into account economic factors on both sides and the fact\nthat all so far proposed countermeasures against adversarial examples reduce\naccuracy on benign samples. Analyzing the scenario in detail, where both\nplayers have two pure strategies, we identify all best responses and conclude\nthat in practical settings, the most influential factor might be the maximum\namount of adversarial examples.\n