Multilingual Unsupervised Neural Machine Translation with Denoising Adapters

We consider the problem of multilingual unsupervised machine translation,\ntranslating to and from languages that only have monolingual data by using\nauxiliary parallel language pairs. For this problem the standard procedure so\nfar to leverage the monolingual data is back-translation, which is\ncomputationally costly and hard to tune.\n In this paper we propose instead to use denoising adapters, adapter layers\nwith a denoising objective, on top of pre-trained mBART-50. In addition to the\nmodularity and flexibility of such an approach we show that the resulting\ntranslations are on-par with back-translating as measured by BLEU, and\nfurthermore it allows adding unseen languages incrementally.\n