Comparison of Speech Representations for Automatic Quality Estimation in Multi-Speaker Text-to-Speech Synthesis

We aim to characterize how different speakers contribute to the perceived\noutput quality of multi-speaker Text-to-Speech (TTS) synthesis. We\nautomatically rate the quality of TTS using a neural network (NN) trained on\nhuman mean opinion score (MOS) ratings. First, we train and evaluate our NN\nmodel on 13 different TTS and voice conversion (VC) systems from the ASVSpoof\n2019 Logical Access (LA) Dataset. Since it is not known how best to represent\nspeech for this task, we compare 8 different representations alongside MOSNet\nframe-based features. Our representations include image-based spectrogram\nfeatures and x-vector embeddings that explicitly model different types of noise\nsuch as T60 reverberation time. Our NN predicts MOS with a high correlation to\nhuman judgments. We report prediction correlation and error. A key finding is\nthe quality achieved for certain speakers seems consistent, regardless of the\nTTS or VC system. It is widely accepted that some speakers give higher quality\nthan others for building a TTS system: our method provides an automatic way to\nidentify such speakers. Finally, to see if our quality prediction models\ngeneralize, we predict quality scores for synthetic speech using a separate\nmulti-speaker TTS system that was trained on LibriTTS data, and conduct our own\nMOS listening test to compare human ratings with our NN predictions.\n