Black-box Detection of Backdoor Attacks with Limited Information and Data

Although deep neural networks (DNNs) have made rapid progress in recent\nyears, they are vulnerable in adversarial environments. A malicious backdoor\ncould be embedded in a model by poisoning the training dataset, whose intention\nis to make the infected model give wrong predictions during inference when the\nspecific trigger appears. To mitigate the potential threats of backdoor\nattacks, various backdoor detection and defense methods have been proposed.\nHowever, the existing techniques usually require the poisoned training data or\naccess to the white-box model, which is commonly unavailable in practice. In\nthis paper, we propose a black-box backdoor detection (B3D) method to identify\nbackdoor attacks with only query access to the model. We introduce a\ngradient-free optimization algorithm to reverse-engineer the potential trigger\nfor each class, which helps to reveal the existence of backdoor attacks. In\naddition to backdoor detection, we also propose a simple strategy for reliable\npredictions using the identified backdoored models. Extensive experiments on\nhundreds of DNN models trained on several datasets corroborate the\neffectiveness of our method under the black-box setting against various\nbackdoor attacks.\n