Model Agnostic Answer Reranking System for Adversarial Question Answering

While numerous methods have been proposed as defenses against adversarial\nexamples in question answering (QA), these techniques are often model specific,\nrequire retraining of the model, and give only marginal improvements in\nperformance over vanilla models. In this work, we present a simple\nmodel-agnostic approach to this problem that can be applied directly to any QA\nmodel without any retraining. Our method employs an explicit answer candidate\nreranking mechanism that scores candidate answers on the basis of their content\noverlap with the question before making the final prediction. Combined with a\nstrong base QAmodel, our method outperforms state-of-the-art defense\ntechniques, calling into question how well these techniques are actually doing\nand strong these adversarial testbeds are.\n