Variational Autoencoders for Semi-supervised Text Classification

Although semi-supervised learning method based on variational autoencoder (\emph{SemiVAE}) works well in image classification tasks, it fails in text classification tasks if using vanilla LSTM as its conditional generative model. We find that the model with this setting is unable to utilize the positive feedback mechanism of \emph{SemiVAE} and hence fails to boost the performance. To tackle this problem, a conditional Long Short-Term Memory network (conditional LSTM) is presented, which receives the conditional information all the time-steps. In addtion, auxiliary variable is also found to be useful in our method in terms of both training speed and prediction accuracy. Experimental results on Large Movie Review Dataset (IMDB) show that the proposed approach significantly improves the classification accuracy compared with pure-supervised classifier and achieves competitive performance against previous pre-training based methods. Additional improvement can be obtained by integrating pre-training based methods.