To Transfer or Not to Transfer: Misclassification Attacks Against Transfer Learned Text Classifiers

Transfer learning --- transferring learned knowledge --- has brought a\nparadigm shift in the way models are trained. The lucrative benefits of\nimproved accuracy and reduced training time have shown promise in training\nmodels with constrained computational resources and fewer training samples.\nSpecifically, publicly available text-based models such as GloVe and BERT that\nare trained on large corpus of datasets have seen ubiquitous adoption in\npractice. In this paper, we ask, "can transfer learning in text prediction\nmodels be exploited to perform misclassification attacks?" As our main\ncontribution, we present novel attack techniques that utilize unintended\nfeatures learnt in the teacher (public) model to generate adversarial examples\nfor student (downstream) models. To the best of our knowledge, ours is the\nfirst work to show that transfer learning from state-of-the-art word-based and\nsentence-based teacher models increase the susceptibility of student models to\nmisclassification attacks. First, we propose a novel word-score based attack\nalgorithm for generating adversarial examples against student models trained\nusing context-free word-level embedding model. On binary classification tasks\ntrained using the GloVe teacher model, we achieve an average attack accuracy of\n97% for the IMDB Movie Reviews and 80% for the Fake News Detection. For\nmulti-class tasks, we divide the Newsgroup dataset into 6 and 20 classes and\nachieve an average attack accuracy of 75% and 41% respectively. Next, we\npresent length-based and sentence-based misclassification attacks for the Fake\nNews Detection task trained using a context-aware BERT model and achieve 78%\nand 39% attack accuracy respectively. Thus, our results motivate the need for\ndesigning training techniques that are robust to unintended feature learning,\nspecifically for transfer learned models.\n