Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation

We present an easy and efficient method to extend existing sentence embedding\nmodels to new languages. This allows to create multilingual versions from\npreviously monolingual models. The training is based on the idea that a\ntranslated sentence should be mapped to the same location in the vector space\nas the original sentence. We use the original (monolingual) model to generate\nsentence embeddings for the source language and then train a new system on\ntranslated sentences to mimic the original model. Compared to other methods for\ntraining multilingual sentence embeddings, this approach has several\nadvantages: It is easy to extend existing models with relatively few samples to\nnew languages, it is easier to ensure desired properties for the vector space,\nand the hardware requirements for training is lower. We demonstrate the\neffectiveness of our approach for 50+ languages from various language families.\nCode to extend sentence embeddings models to more than 400 languages is\npublicly available.\n