Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport

Selecting input features of top relevance has become a popular method for\nbuilding self-explaining models. In this work, we extend this selective\nrationalization approach to text matching, where the goal is to jointly select\nand align text pieces, such as tokens or sentences, as a justification for the\ndownstream prediction. Our approach employs optimal transport (OT) to find a\nminimal cost alignment between the inputs. However, directly applying OT often\nproduces dense and therefore uninterpretable alignments. To overcome this\nlimitation, we introduce novel constrained variants of the OT problem that\nresult in highly sparse alignments with controllable sparsity. Our model is\nend-to-end differentiable using the Sinkhorn algorithm for OT and can be\ntrained without any alignment annotations. We evaluate our model on the\nStackExchange, MultiNews, e-SNLI, and MultiRC datasets. Our model achieves very\nsparse rationale selections with high fidelity while preserving prediction\naccuracy compared to strong attention baseline models.\n