Variational (Gradient) Estimate of the Score Function in Energy-based Latent Variable Models

The learning and evaluation of energy-based latent variable models (EBLVMs)\nwithout any structural assumptions are highly challenging, because the true\nposteriors and the partition functions in such models are generally\nintractable. This paper presents variational estimates of the score function\nand its gradient with respect to the model parameters in a general EBLVM,\nreferred to as VaES and VaGES respectively. The variational posterior is\ntrained to minimize a certain divergence to the true model posterior and the\nbias in both estimates can be bounded by the divergence theoretically. With a\nminimal model assumption, VaES and VaGES can be applied to the kernelized Stein\ndiscrepancy (KSD) and score matching (SM)-based methods to learn EBLVMs.\nBesides, VaES can also be used to estimate the exact Fisher divergence between\nthe data and general EBLVMs.\n