From abstract items to latent spaces to observed data and back: Compositional Variational Auto-Encoder

Conditional Generative Models are now acknowledged an essential tool in\nMachine Learning. This paper focuses on their control. While many approaches\naim at disentangling the data through the coordinate-wise control of their\nlatent representations, another direction is explored in this paper. The\nproposed CompVAE handles data with a natural multi-ensemblist structure (i.e.\nthat can naturally be decomposed into elements). Derived from Bayesian\nvariational principles, CompVAE learns a latent representation leveraging both\nobservational and symbolic information. A first contribution of the approach is\nthat this latent representation supports a compositional generative model,\namenable to multi-ensemblist operations (addition or subtraction of elements in\nthe composition). This compositional ability is enabled by the invariance and\ngenerality of the whole framework w.r.t. respectively, the order and number of\nthe elements. The second contribution of the paper is a proof of concept on\nsynthetic 1D and 2D problems, demonstrating the efficiency of the proposed\napproach.\n