Robust Neural Machine Translation: Modeling Orthographic and Interpunctual Variation

Neural machine translation systems typically are trained on curated corpora\nand break when faced with non-standard orthography or punctuation. Resilience\nto spelling mistakes and typos, however, is crucial as machine translation\nsystems are used to translate texts of informal origins, such as chat\nconversations, social media posts and web pages. We propose a simple generative\nnoise model to generate adversarial examples of ten different types. We use\nthese to augment machine translation systems' training data and show that, when\ntested on noisy data, systems trained using adversarial examples perform almost\nas well as when translating clean data, while baseline systems' performance\ndrops by 2-3 BLEU points. To measure the robustness and noise invariance of\nmachine translation systems' outputs, we use the average translation edit rate\nbetween the translation of the original sentence and its noised variants. Using\nthis measure, we show that systems trained on adversarial examples on average\nyield 50% consistency improvements when compared to baselines trained on clean\ndata.\n