Nowadays, Neural Machine Translation (NMT) has become the mainstream approach for machine translation, and has got considerable improvement after attention mechanism is introduced. However, usage of attention is insufficient, some defects such as rare word problem still remain in end-to-end NMT approach, which is limited by the training targets and priori knowledge. Traditionally, words out of vocabulary (OOV words) are simply represented with a signal ‘UNK’ or cut into sub-words. In this paper, we consider some optimizations on the alignment feature of attention mechanism, which is appropriate not only to guide every output to find the best match in input sentence, but also to help overall quality of the translation. We build English-Chinese and Chinese-English NMT systems based on our algorithm, and experiment on casia2015 corpus from WMT17. The result shows that the translation of our model gets a considerable improvement respectively.
Paper
Full text
Optimizing Attention Mechanism for Neural Machine Transltion
Semantic Scholar · Computer Science · 2018
Abstract
Nowadays, Neural Machine Translation (NMT) has become the mainstream approach for machine translation, and has got considerable improvement after attention mechanism is introduced. However, usage of attention is insufficient, some defects such as rare word problem still remain in end-to-end NMT approach, which is limited by the training targets and priori knowledge. Traditionally, words out of vocabulary (OOV words) are simply represented with a signal ‘UNK’ or cut into sub-words. In this paper, we consider some optimizations on the alignment feature of attention mechanism, which is appropriate not only to guide every output to find the best match in input sentence, but also to help overall quality of the translation. We build English-Chinese and Chinese-English NMT systems based on our algorithm, and experiment on casia2015 corpus from WMT17. The result shows that the translation of our model gets a considerable improvement respectively.