Reducing Position Bias in Simultaneous Machine Translation with Length-Aware Framework

Simultaneous machine translation (SiMT) starts translating while receiving\nthe streaming source inputs, and hence the source sentence is always incomplete\nduring translating. Different from the full-sentence MT using the conventional\nseq-to-seq architecture, SiMT often applies prefix-to-prefix architecture,\nwhich forces each target word to only align with a partial source prefix to\nadapt to the incomplete source in streaming inputs. However, the source words\nin the front positions are always illusoryly considered more important since\nthey appear in more prefixes, resulting in position bias, which makes the model\npay more attention on the front source positions in testing. In this paper, we\nfirst analyze the phenomenon of position bias in SiMT, and develop a\nLength-Aware Framework to reduce the position bias by bridging the structural\ngap between SiMT and full-sentence MT. Specifically, given the streaming\ninputs, we first predict the full-sentence length and then fill the future\nsource position with positional encoding, thereby turning the streaming inputs\ninto a pseudo full-sentence. The proposed framework can be integrated into most\nexisting SiMT methods to further improve performance. Experiments on two\nrepresentative SiMT methods, including the state-of-the-art adaptive policy,\nshow that our method successfully reduces the position bias and thereby\nachieves better SiMT performance.\n

Paper

Similar papers

© 2026 NYSGPT2525 LLC