N-Best ASR Transformer: Enhancing SLU Performance using Multiple ASR Hypotheses

Spoken Language Understanding (SLU) systems parse speech into semantic\nstructures like dialog acts and slots. This involves the use of an Automatic\nSpeech Recognizer (ASR) to transcribe speech into multiple text alternatives\n(hypotheses). Transcription errors, common in ASRs, impact downstream SLU\nperformance negatively. Approaches to mitigate such errors involve using richer\ninformation from the ASR, either in form of N-best hypotheses or word-lattices.\nWe hypothesize that transformer models learn better with a simpler utterance\nrepresentation using the concatenation of the N-best ASR alternatives, where\neach alternative is separated by a special delimiter [SEP]. In our work, we\ntest our hypothesis by using concatenated N-best ASR alternatives as the input\nto transformer encoder models, namely BERT and XLM-RoBERTa, and achieve\nperformance equivalent to the prior state-of-the-art model on DSTC2 dataset. We\nalso show that our approach significantly outperforms the prior\nstate-of-the-art when subjected to the low data regime. Additionally, this\nmethodology is accessible to users of third-party ASR APIs which do not provide\nword-lattice information.\n