Beyond Noise: Mitigating the Impact of Fine-grained Semantic Divergences on Neural Machine Translation

While it has been shown that Neural Machine Translation (NMT) is highly\nsensitive to noisy parallel training samples, prior work treats all types of\nmismatches between source and target as noise. As a result, it remains unclear\nhow samples that are mostly equivalent but contain a small number of\nsemantically divergent tokens impact NMT training. To close this gap, we\nanalyze the impact of different types of fine-grained semantic divergences on\nTransformer models. We show that models trained on synthetic divergences output\ndegenerated text more frequently and are less confident in their predictions.\nBased on these findings, we introduce a divergent-aware NMT framework that uses\nfactors to help NMT recover from the degradation caused by naturally occurring\ndivergences, improving both translation quality and model calibration on EN-FR\ntasks.\n