A Latent Variable Model for Two-Dimensional Canonical Correlation Analysis and its Variational Inference

The probabilistic dimension reduction has been and is a major concern. Probabilistic models provide a better interpretability of the dimension reduction methods and present a framework for their further extensions. In pattern recognition problems, data that have a matrix or tensor structure is initially transformed into a vector format. This eliminates the internal structure of the data. The available perspective is to maintain the internal structure of each data while reducing the dimensionality, which can reduce the small sample size problem. Canonical correlation analysis is one of the most important techniques in dimension reduction in multi-view data. A two-dimensional canonical correlation analysis as an extension of canonical correlation analysis has been proposed to preserve the matrix structure of the data. Here, a new probabilistic framework for two-dimensional canonical correlation analysis is proposed, where the matrix-variate distributions are applied to model the relation between the latent matrix and the two-view matrix-variate observed data. These distributions, specific to the matrix data, can provide better understanding of two-dimensional canonical correlation analysis and pave the way for further extensions. In general, there does not exist any analytical maximum likelihood solution for this model; therefore, here the two approaches, one based on the expectation maximization and other on variational expected maximization, are proposed for learning the model parameters. The synthetic data are applied to evaluate the convergence and quality of the mapping of these algorithms. The functionalities of these methods and their counterparts are compared on the real face datasets.