Weakly-supervised word-level pronunciation error detection in non-native English speech

We propose a weakly-supervised model for word-level mispronunciation\ndetection in non-native (L2) English speech. To train this model, phonetically\ntranscribed L2 speech is not required and we only need to mark mispronounced\nwords. The lack of phonetic transcriptions for L2 speech means that the model\nhas to learn only from a weak signal of word-level mispronunciations. Because\nof that and due to the limited amount of mispronounced L2 speech, the model is\nmore likely to overfit. To limit this risk, we train it in a multi-task setup.\nIn the first task, we estimate the probabilities of word-level\nmispronunciation. For the second task, we use a phoneme recognizer trained on\nphonetically transcribed L1 speech that is easily accessible and can be\nautomatically annotated. Compared to state-of-the-art approaches, we improve\nthe accuracy of detecting word-level pronunciation errors in AUC metric by 30%\non the GUT Isle Corpus of L2 Polish speakers, and by 21.5% on the Isle Corpus\nof L2 German and Italian speakers.\n