A Target-Agnostic Attack on Deep Models: Exploiting Security Vulnerabilities of Transfer Learning

Due to insufficient training data and the high computational cost to train a\ndeep neural network from scratch, transfer learning has been extensively used\nin many deep-neural-network-based applications. A commonly used transfer\nlearning approach involves taking a part of a pre-trained model, adding a few\nlayers at the end, and re-training the new layers with a small dataset. This\napproach, while efficient and widely used, imposes a security vulnerability\nbecause the pre-trained model used in transfer learning is usually publicly\navailable, including to potential attackers. In this paper, we show that\nwithout any additional knowledge other than the pre-trained model, an attacker\ncan launch an effective and efficient brute force attack that can craft\ninstances of input to trigger each target class with high confidence. We assume\nthat the attacker has no access to any target-specific information, including\nsamples from target classes, re-trained model, and probabilities assigned by\nSoftmax to each class, and thus making the attack target-agnostic. These\nassumptions render all previous attack models inapplicable, to the best of our\nknowledge. To evaluate the proposed attack, we perform a set of experiments on\nface recognition and speech recognition tasks and show the effectiveness of the\nattack. Our work reveals a fundamental security weakness of the Softmax layer\nwhen used in transfer learning settings.\n