Straight to the Gradient: Learning to Use Novel Tokens for Neural Text Generation

Advanced large-scale neural language models have led to significant success\nin many language generation tasks. However, the most commonly used training\nobjective, Maximum Likelihood Estimation (MLE), has been shown problematic,\nwhere the trained model prefers using dull and repetitive phrases. In this\nwork, we introduce ScaleGrad, a modification straight to the gradient of the\nloss function, to remedy the degeneration issue of the standard MLE objective.\nBy directly maneuvering the gradient information, ScaleGrad makes the model\nlearn to use novel tokens. Empirical results show the effectiveness of our\nmethod not only in open-ended generation, but also in directed generation\ntasks. With the simplicity in architecture, our method can serve as a general\ntraining objective that is applicable to most of the neural text generation\ntasks.\n