Adversarial Training Methods for Boltzmann Machines

Abstract

A Restricted Boltzmann Machines (RBM) is a generative Neural Net that is typically trained to minimize <italic>KL</italic> divergence between data distribution <inline-formula> <tex-math notation="LaTeX">${P} _{data}$ </tex-math></inline-formula> and its model distribution <inline-formula> <tex-math notation="LaTeX">${P} _{RBM}$ </tex-math></inline-formula>. However, minimizing this <italic>KL</italic> divergence does not sufficiently penalize an RBM that place a high probability in regions where the data distribution has a low density, and therefore, RBMs always generate blurry images. In order to solve this problem, this paper extends the loss function of RBMs from <italic>KL</italic> divergence to adversarial loss and proposes an Adversarial Restricted Boltzmann Machine (ARBM) and an Adversarial Deep Boltzmann Machine (ADBM). Different from the other RBMs, an ARBM minimizes its adversarial loss between the data distribution and its model distribution without explicit gradients. Different from traditional DBMs, an ADBM minimizes its adversarial loss without a layer-by-layer pre-training. In order to generate high-quality color images, this paper proposes an Adversarial Hybrid Deep Generative Net (AHDGN) based on an ADBM. The experiments verify that the adversarial loss can be minimized in our proposed models, and the generated images are comparable with the current state-of-the-art results.