StableEmit: Selection Probability Discount for Reducing Emission Latency of Streaming Monotonic Attention ASR

While attention-based encoder-decoder (AED) models have been successfully\nextended to the online variants for streaming automatic speech recognition\n(ASR), such as monotonic chunkwise attention (MoChA), the models still have a\nlarge label emission latency because of the unconstrained end-to-end training\nobjective. Previous works tackled this problem by leveraging alignment\ninformation to control the timing to emit tokens during training. In this work,\nwe propose a simple alignment-free regularization method, StableEmit, to\nencourage MoChA to emit tokens earlier. StableEmit discounts the selection\nprobabilities in hard monotonic attention for token boundary detection by a\nconstant factor and regularizes them to recover the total attention mass during\ntraining. As a result, the scale of the selection probabilities is increased,\nand the values can reach a threshold for token emission earlier, leading to a\nreduction of emission latency and deletion errors. Moreover, StableEmit can be\ncombined with methods that constraint alignments to further improve the\naccuracy and latency. Experimental evaluations with LSTM and Conformer encoders\ndemonstrate that StableEmit significantly reduces the recognition errors and\nthe emission latency simultaneously. We also show that the use of alignment\ninformation is complementary in both metrics.\n