Analyzing the Source and Target Contributions to Predictions in Neural Machine Translation

In Neural Machine Translation (and, more generally, conditional language\nmodeling), the generation of a target token is influenced by two types of\ncontext: the source and the prefix of the target sequence. While many attempts\nto understand the internal workings of NMT models have been made, none of them\nexplicitly evaluates relative source and target contributions to a generation\ndecision. We argue that this relative contribution can be evaluated by adopting\na variant of Layerwise Relevance Propagation (LRP). Its underlying\n'conservation principle' makes relevance propagation unique: differently from\nother methods, it evaluates not an abstract quantity reflecting token\nimportance, but the proportion of each token's influence. We extend LRP to the\nTransformer and conduct an analysis of NMT models which explicitly evaluates\nthe source and target relative contributions to the generation process. We\nanalyze changes in these contributions when conditioning on different types of\nprefixes, when varying the training objective or the amount of training data,\nand during the training process. We find that models trained with more data\ntend to rely on source information more and to have more sharp token\ncontributions; the training process is non-monotonic with several stages of\ndifferent nature.\n