Contextual Neural Machine Translation Improves Translation of Cataphoric Pronouns

The advent of context-aware NMT has resulted in promising improvements in the\noverall translation quality and specifically in the translation of discourse\nphenomena such as pronouns. Previous works have mainly focused on the use of\npast sentences as context with a focus on anaphora translation. In this work,\nwe investigate the effect of future sentences as context by comparing the\nperformance of a contextual NMT model trained with the future context to the\none trained with the past context. Our experiments and evaluation, using\ngeneric and pronoun-focused automatic metrics, show that the use of future\ncontext not only achieves significant improvements over the context-agnostic\nTransformer, but also demonstrates comparable and in some cases improved\nperformance over its counterpart trained on past context. We also perform an\nevaluation on a targeted cataphora test suite and report significant gains over\nthe context-agnostic Transformer in terms of BLEU.\n