Masked Hard Coverage Mechanism on Pointer-generator Network for Natural Language Generation

Abstract

Natural Language Generation (NLG) task is to generate natural language utterances from structured data. Seq2seq-based systems show great potentiality and have been widely explored for NLG. While they achieve good generation performance, over-generation and under-generation issues still arise in the generated results. We propose maintaining a masked hard coverage mechanism in the pointer-generator network, a seq2seqbased architecture that trains a switch policy to produce output sequences by partially copying from input structured data. The proposed mechanism can be regarded as the inner controlling module to keep track of the copying history and force the network to generate sentences accurately covering all information provided in structured data. Experimental results show that our coverage mechanism alleviates the over-generation and under-generation issues and achieves decent performance on the E2E NLG dataset.