Local Disentanglement in Variational Auto-Encoders Using Jacobian $L_1$ Regularization

There have been many recent advances in representation learning; however,\nunsupervised representation learning can still struggle with model\nidentification issues related to rotations of the latent space. Variational\nAuto-Encoders (VAEs) and their extensions such as $\\beta$-VAEs have been shown\nto improve local alignment of latent variables with PCA directions, which can\nhelp to improve model disentanglement under some conditions. Borrowing\ninspiration from Independent Component Analysis (ICA) and sparse coding, we\npropose applying an $L_1$ loss to the VAE's generative Jacobian during training\nto encourage local latent variable alignment with independent factors of\nvariation in images of multiple objects or images with multiple parts. We\ndemonstrate our results on a variety of datasets, giving qualitative and\nquantitative results using information theoretic and modularity measures that\nshow our added $L_1$ cost encourages local axis alignment of the latent\nrepresentation with individual factors of variation.\n