Librispeech Transducer Model with Internal Language Model Prior Correction

We present our transducer model on Librispeech. We study variants to include\nan external language model (LM) with shallow fusion and subtract an estimated\ninternal LM. This is justified by a Bayesian interpretation where the\ntransducer model prior is given by the estimated internal LM. The subtraction\nof the internal LM gives us over 14% relative improvement over normal shallow\nfusion. Our transducer has a separate probability distribution for the\nnon-blank labels which allows for easier combination with the external LM, and\neasier estimation of the internal LM. We additionally take care of including\nthe end-of-sentence (EOS) probability of the external LM in the last blank\nprobability which further improves the performance. All our code and setups are\npublished.\n