A unified framework for closed-form nonparametric regression, classification, preference and mixed problems with Skew Gaussian Processes

Skew-Gaussian processes (SkewGPs) extend the multivariate Unified Skew-Normal\ndistributions over finite dimensional vectors to distribution over functions.\nSkewGPs are more general and flexible than Gaussian processes, as SkewGPs may\nalso represent asymmetric distributions. In a recent contribution we showed\nthat SkewGP and probit likelihood are conjugate, which allows us to compute the\nexact posterior for non-parametric binary classification and preference\nlearning. In this paper, we generalize previous results and we prove that\nSkewGP is conjugate with both the normal and affine probit likelihood, and more\nin general, with their product. This allows us to (i) handle classification,\npreference, numeric and ordinal regression, and mixed problems in a unified\nframework; (ii) derive closed-form expression for the corresponding posterior\ndistributions. We show empirically that the proposed framework based on SkewGP\nprovides better performance than Gaussian processes in active learning and\nBayesian (constrained) optimization. These two tasks are fundamental for design\nof experiments and in Data Science.\n