SpeakerStew: Scaling to Many Languages with a Triaged Multilingual Text-Dependent and Text-Independent Speaker Verification System

In this paper, we describe SpeakerStew - a hybrid system to perform speaker\nverification on 46 languages. Two core ideas were explored in this system: (1)\nPooling training data of different languages together for multilingual\ngeneralization and reducing development cycles; (2) A novel triage mechanism\nbetween text-dependent and text-independent models to reduce runtime cost and\nexpected latency. To the best of our knowledge, this is the first study of\nspeaker verification systems at the scale of 46 languages. The problem is\nframed from the perspective of using a smart speaker device with interactions\nconsisting of a wake-up keyword (text-dependent) followed by a speech query\n(text-independent). Experimental evidence suggests that training on multiple\nlanguages can generalize to unseen varieties while maintaining performance on\nseen varieties. We also found that it can reduce computational requirements for\ntraining models by an order of magnitude. Furthermore, during model inference\non English data, we observe that leveraging a triage framework can reduce the\nnumber of calls to the more computationally expensive text-independent system\nby 73% (and reduce latency by 59%) while maintaining an EER no worse than the\ntext-independent setup.\n