"Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classification

Feature attribution a.k.a. input salience methods which assign an importance\nscore to a feature are abundant but may produce surprisingly different results\nfor the same model on the same input. While differences are expected if\ndisparate definitions of importance are assumed, most methods claim to provide\nfaithful attributions and point at the features most relevant for a model's\nprediction. Existing work on faithfulness evaluation is not conclusive and does\nnot provide a clear answer as to how different methods are to be compared.\nFocusing on text classification and the model debugging scenario, our main\ncontribution is a protocol for faithfulness evaluation that makes use of\npartially synthetic data to obtain ground truth for feature importance ranking.\nFollowing the protocol, we do an in-depth analysis of four standard salience\nmethod classes on a range of datasets and shortcuts for BERT and LSTM models\nand demonstrate that some of the most popular method configurations provide\npoor results even for simplest shortcuts. We recommend following the protocol\nfor each new task and model combination to find the best method for identifying\nshortcuts.\n