AbstractMetrics specifying distances between data points can be learned in a discriminative manner or fromgenerative models. In this paper, we show how to unify generative and discriminative learning of met-rics via a kernel learning framework. Specifically, we learn local metrics optimized from parametricgenerative models. These are then used as base kernels to construct a global kernel that minimizes adiscriminative training criterion. We consider both linear and nonlinear combinations of local metrickernels. Our empirical results show that these combinations significantly improve performance on clas-sification tasks. The proposed learning algorithm is also ve ry efficient, achieving order of magnitudespeedup in training time compared to previous discriminative baseline methods. 1 Introduction Metric learning – learning how to specify distances between data points – has been a topic of much interestin machine learning recently. For example, discriminative techniques for metric learning aim to improvethe performance of a classifier, such as the k-nearest neighbor classifier, on a training set. As a generalstrategy, these techniques try to reduce the distances between data points belonging to the same class whileincreasing the distances between data points from different classes [1, 2, 3, 4, 5, 6, 7, 8]. In this framework,1