Data augmentation using prosody and false starts to recognize non-native children's speech

This paper describes AaltoASR's speech recognition system for the INTERSPEECH\n2020 shared task on Automatic Speech Recognition (ASR) for non-native\nchildren's speech. The task is to recognize non-native speech from children of\nvarious age groups given a limited amount of speech. Moreover, the speech being\nspontaneous has false starts transcribed as partial words, which in the test\ntranscriptions leads to unseen partial words. To cope with these two\nchallenges, we investigate a data augmentation-based approach. Firstly, we\napply the prosody-based data augmentation to supplement the audio data.\nSecondly, we simulate false starts by introducing partial-word noise in the\nlanguage modeling corpora creating new words. Acoustic models trained on\nprosody-based augmented data outperform the models using the baseline recipe or\nthe SpecAugment-based augmentation. The partial-word noise also helps to\nimprove the baseline language model. Our ASR system, a combination of these\nschemes, is placed third in the evaluation period and achieves the word error\nrate of 18.71%. Post-evaluation period, we observe that increasing the amounts\nof prosody-based augmented data leads to better performance. Furthermore,\nremoving low-confidence-score words from hypotheses can lead to further gains.\nThese two improvements lower the ASR error rate to 17.99%.\n