Improve Diverse Text Generation by Self Labeling Conditional Variational Auto Encoder

Diversity plays a vital role in many text generating applications. In recent years, Conditional Variational Auto Encoders (CVAE) have shown promising performances for this task. However, they often encounter the so called KL-Vanishing problem. Pervious works use heuristic methods to avoid KL-vanishing, but it is hard to find an appropriate degree to which these methods should be applied. In this paper, we propose an explicit optimizing objective function to guide the encoder towards the "best encoder" and directly pull the CVAE away from KL-vanishing. A labeling network is introduced to estimate the "best encoder It provides a continuous label in the latent space of CVAE to help build a close connection between latent variables and targets. The whole proposed method is named Self Labeling CVAE (SLCVAE). To boost the research of diverse text generation, we also propose a large native one-to-many dataset. Extensive experiments are conducted on two tasks, which show that our method largely improves the generating diversity while achieving comparable accuracy compared with state-of-the-art algorithms.