Considering Likelihood in NLP Classification Explanations with Occlusion and Language Modeling

Recently, state-of-the-art NLP models gained an increasing syntactic and\nsemantic understanding of language, and explanation methods are crucial to\nunderstand their decisions. Occlusion is a well established method that\nprovides explanations on discrete language data, e.g. by removing a language\nunit from an input and measuring the impact on a model's decision. We argue\nthat current occlusion-based methods often produce invalid or syntactically\nincorrect language data, neglecting the improved abilities of recent NLP\nmodels. Furthermore, gradient-based explanation methods disregard the discrete\ndistribution of data in NLP. Thus, we propose OLM: a novel explanation method\nthat combines occlusion and language models to sample valid and syntactically\ncorrect replacements with high likelihood, given the context of the original\ninput. We lay out a theoretical foundation that alleviates these weaknesses of\nother explanation methods in NLP and provide results that underline the\nimportance of considering data likelihood in occlusion-based explanation.\n