Integrating Unsupervised Data Generation into Self-Supervised Neural Machine Translation for Low-Resource Languages

For most language combinations, parallel data is either scarce or simply\nunavailable. To address this, unsupervised machine translation (UMT) exploits\nlarge amounts of monolingual data by using synthetic data generation techniques\nsuch as back-translation and noising, while self-supervised NMT (SSNMT)\nidentifies parallel sentences in smaller comparable data and trains on them. To\ndate, the inclusion of UMT data generation techniques in SSNMT has not been\ninvestigated. We show that including UMT techniques into SSNMT significantly\noutperforms SSNMT and UMT on all tested language pairs, with improvements of up\nto +4.3 BLEU, +50.8 BLEU, +51.5 over SSNMT, statistical UMT and hybrid UMT,\nrespectively, on Afrikaans to English. We further show that the combination of\nmultilingual denoising autoencoding, SSNMT with backtranslation and bilingual\nfinetuning enables us to learn machine translation even for distant language\npairs for which only small amounts of monolingual data are available, e.g.\nyielding BLEU scores of 11.6 (English to Swahili).\n