Minimum message length estimation of mixtures of multivariate Gaussian and von Mises-Fisher distributions

Mixture modelling involves explaining some observed evidence using a\ncombination of probability distributions. The crux of the problem is the\ninference of an optimal number of mixture components and their corresponding\nparameters. This paper discusses unsupervised learning of mixture models using\nthe Bayesian Minimum Message Length (MML) criterion. To demonstrate the\neffectiveness of search and inference of mixture parameters using the proposed\napproach, we select two key probability distributions, each handling\nfundamentally different types of data: the multivariate Gaussian distribution\nto address mixture modelling of data distributed in Euclidean space, and the\nmultivariate von Mises-Fisher (vMF) distribution to address mixture modelling\nof directional data distributed on a unit hypersphere. The key contributions of\nthis paper, in addition to the general search and inference methodology,\ninclude the derivation of MML expressions for encoding the data using\nmultivariate Gaussian and von Mises-Fisher distributions, and the analytical\nderivation of the MML estimates of the parameters of the two distributions. Our\napproach is tested on simulated and real world data sets. For instance, we\ninfer vMF mixtures that concisely explain experimentally determined\nthree-dimensional protein conformations, providing an effective null model\ndescription of protein structures that is central to many inference problems in\nstructural bioinformatics. The experimental results demonstrate that the\nperformance of our proposed search and inference method along with the encoding\nschemes improve on the state of the art mixture modelling techniques.\n