Learning finite-dimensional coding schemes with nonlinear reconstruction maps

This paper generalizes the Maurer--Pontil framework of finite-dimensional\nlossy coding schemes to the setting where a high-dimensional random vector is\nmapped to an element of a compact set of latent representations in a\nlower-dimensional Euclidean space, and the reconstruction map belongs to a\ngiven class of nonlinear maps. Under this setup, which encompasses a broad\nclass of unsupervised representation learning problems, we establish a\nconnection to approximate generative modeling under structural constraints\nusing the tools from the theory of optimal transportation. Next, we consider\nproblem of learning a coding scheme on the basis of a finite collection of\ntraining samples and present generalization bounds that hold with high\nprobability. We then illustrate the general theory in the setting where the\nreconstruction maps are implemented by deep neural nets.\n