Do Acoustic Word Embeddings Capture Phonological Similarity? An Empirical Study

Several variants of deep neural networks have been successfully employed for\nbuilding parametric models that project variable-duration spoken word segments\nonto fixed-size vector representations, or acoustic word embeddings (AWEs).\nHowever, it remains unclear to what degree we can rely on the distance in the\nemerging AWE space as an estimate of word-form similarity. In this paper, we\nask: does the distance in the acoustic embedding space correlate with\nphonological dissimilarity? To answer this question, we empirically investigate\nthe performance of supervised approaches for AWEs with different neural\narchitectures and learning objectives. We train AWE models in controlled\nsettings for two languages (German and Czech) and evaluate the embeddings on\ntwo tasks: word discrimination and phonological similarity. Our experiments\nshow that (1) the distance in the embedding space in the best cases only\nmoderately correlates with phonological distance, and (2) improving the\nperformance on the word discrimination task does not necessarily yield models\nthat better reflect word phonological similarity. Our findings highlight the\nnecessity to rethink the current intrinsic evaluations for AWEs.\n