Knot Selection in Sparse Gaussian Processes with a Variational Objective Function

Sparse, knot-based Gaussian processes have enjoyed considerable success as\nscalable approximations to full Gaussian processes. Certain sparse models can\nbe derived through specific variational approximations to the true posterior,\nand knots can be selected to minimize the Kullback-Leibler divergence between\nthe approximate and true posterior. While this has been a successful approach,\nsimultaneous optimization of knots can be slow due to the number of parameters\nbeing optimized. Furthermore, there have been few proposed methods for\nselecting the number of knots, and no experimental results exist in the\nliterature. We propose using a one-at-a-time knot selection algorithm based on\nBayesian optimization to select the number and locations of knots. We showcase\nthe competitive performance of this method relative to simultaneous\noptimization of knots on three benchmark data sets, but at a fraction of the\ncomputational cost.\n