Leveraging Locality and Robustness to Achieve Massively Scalable Gaussian Process Regression

The accurate predictions and principled uncertainty measures provided by GP regression incur O(n^3) cost which is prohibitive for modern-day large-scale applications. This has motivated extensive work on computationally efficient approximations. We introduce a new perspective by exploring robustness properties and limiting behaviour of GP nearest-neighbour (GPnn) prediction. We demonstrate through theory and simulation that as the data-size n increases, accuracy of estimated parameters and GP model assumptions become increasingly irrelevant to GPnn predictive accuracy. Consequently, it is sufficient to spend small amounts of work on parameter estimation in order to achieve high MSE accuracy, even in the presence of gross misspecification. In contrast, as n tends to infinity, uncertainty calibration and NLL are shown to remain sensitive to just one parameter, the additive noise-variance; but we show that this source of inaccuracy can be corrected for, thereby achieving both well-calibrated uncertainty measures and accurate predictions at remarkably low computational cost. We exhibit a very simple GPnn regression algorithm with stand-out performance compared to other state-of-the-art GP approximations as measured on large UCI datasets. It operates at a small fraction of those other methods' training costs, for example on a basic laptop taking about 30 seconds to train on a dataset of size n = 1.6 x 10^6.

Paper

References (38)

Scroll for more · 26 remaining

Similar papers

Peer review

Reviewer 3iYQ7/10 · confidence 3/52023-07-05

Summary

The paper proposes a Gaussian process regression technique that only uses nearest neighbors for prediction. The question of choosing hyperparameters is addressed and asymptotic behavior is analyzed. The method shows substantial speed up on UCI datasets while also providing improved predictive performance.

Strengths

The empirical evaluation shows very good results. Even so, the method is theoretically analyzed and discussed with interesting insights and hypotheses. The paper is well-written.

Weaknesses

A comparison with other nearest neighbor based techniques might have been helpful (e.g. ones cited in the paper). Also, it might make sense to investigate more into the geostatistical work on the subject and cite something from the area. I know that nearest neighbor based predictions are quite often considered to be standard in the area.

Questions

Due to time constraints and NeurIPS not accommodating my request for a reduced reviewing workload, I did not have the opportunity to thoroughly review the proofs or examine all the details.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The limitations are addressed.

Reviewer N8D33/10 · confidence 3/52023-07-06

Summary

The paper proposes a change in perspective in how Gaussian process regression can be leveraged by conditioning predictive distribution on only the neighboring data points. The paper argues against the common practice of using one singular set of data points for model hyperparameter estimation and prediction, and presents a framework for running scalable approximation of GP regression using locality information.

Strengths

I believe that the overall idea of leveraging locality in Gaussian process models has a lot of potential: while I am skeptical as the arguments in the paper seem unconvincing, I think there is merit in good empirical performance for such algorithms. However, I believe that a comparison with existing methods should be done in a more proper manner.

Weaknesses

While using KNN in combination of GP regression might seem like a straightforward idea to massively speed up the inference of GP models, I believe that many key aspects of GPs are overlooked, constituting a major weakness from this paper. I list some of the specific examples below. - Combining KNN with GP is overall not a novel idea, the practice is not exactly new. A cursory search reveals that many research papers outside the machine learning community are already using it as a proxy for the less scalable fullscale GP regression. Some examples relevant to machine learning include: 1. Wu L, Pleiss G, Cunningham JP. Variational nearest neighbor Gaussian process. In: Proceedings of the 39th International Conference on Machine Learning, PMLR; 2022; 2. Chen H, Zheng L, Kontar RA, Raskutti G. Gaussian Process Parameter Estimation Using Mini-batch Stochastic Gradient Descent: Convergence Guarantees and Empirical Benefits. Journal of Machine Learning Research. 2022;23(227):1–59. - The paper focuses on _pointwise_ uncertainty prediction, while GP posteriors provide not only a pointwise uncertainty, but _pairwise_ covariance as well. The covariance kernel is such a crucial element that defines the hypothesis space of GP models that given a configuration of pointwise uncertainty, there exists countless GPs that takes identical pointwise variance. - While KNN is guaranteed to speed up massively in lower dimensions, it suffers from curse of dimensionality itself as it becomes harder to find nearest neighbors. - While it is not mandatory for parameter estimation and prediction to be conditioned on the same set of data, SVGP does not incur much computational expense at prediction level. Conventional GP regression and SVGP only need one matrix inversion operation to predict the uncertainty level on an arbitrary number of test points, while GPnn needs a matrix inversion operation for every separate data point -- essentially it becomes a balancing act of whether optimizing an SVGP or apply KNN to every test point is more expensive. - Theorem 1 discusses the asymptotic behavior as $n\rightarrow\infty$. While the result of insensitivity w.r.t. kernel parameter carries some value, the _rate_ of convergence matters in a finite data regime. For example, Ackermann function and inverse Ackermann function both converge to infinity but at drastically different rates. - While the common squared exponential kernel is designed to look at neighborhood information, many more expressive kernels can have longer-range connections. I think the paper tacitly admits that GPnn can only be used for interpolation tasks, but it is somewhat unfair to state that kernel hyperparameter does not matter as it is always a misspecified model. - The ease at obtaining easy uncertainty information given neighborhood information is conditioned on conjugate likelihood, with a Gaussian observational model. As some degree of variational inference is required for non-conjugate likelihood, GPnn struggles to apply beyond regression tasks. - Simulating synthetic data using Algorithm 11 seems flawed. Algorithm 1 generates test points as if every 2 test points are un-correlated, which naturally gives GPnn an advantage.

Questions

None.

Rating

3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

I have listed the limitations in the "weakness" section.

Reviewer MqBs8/10 · confidence 4/52023-07-09

Summary

This paper starts by proposing use of a simple nearest neighbors scheme for prediction, where instead of using the training set, one uses the $m$ nearest neighbors of the training set to the test set. They then, in Theorem 1, analyze the asymptotic expected MSE, calibration and negative log-likelihood of this prediction approach under a given estimator of the parameters. They find that there is robustness to poor estimation except in noise variance. They show empirically that their simulations match theory in algorithm 1. They then propose a scalable GP regression algorithm: noting that when using their nearest neighbor prediction approach, the MSE, calibration and negative log-likelihood are *somewhat* robust to wrong parameters (except noise variance), they use multiple approximation steps, including using a subset of the training data and using a structured covariance matrix. In order to improve noise variance estimation, they add a calibration step. They show that it outperforms several baselines on real world datasets.

Strengths

This is a very interesting paper: the resulting technique is very simple but pops out in a very non-obvious way. Further, it leads to a powerful consequence: you don't need the greatest parameter estimates for training if you do nearest neighbors prediction, as long as you add a calibration step to improve the estimate of noise variance. Despite my slight reservations about the baselines and the strange paper structure, I strongly recommend acceptance.

Weaknesses

This paper is written in a very non-standard way and completely ignores the standard ML paper format. It completely ignores the standard introduction, replacing it by what is often the 2nd or third section, which is some technical background. It then has the 2nd section, which is *similar* to what often comes at the end of an intro, but is somehow different. The related work section is very short. In general everything is somewhat terse. I don't completely hate it, but it's jarring to read this, and I'm not sure what the justification for doing it this way is, particularly since the authors seem aware of literature and the techniques used in the literature so presumably are aware of how standard ML papers are written to flow a certain way. The main weakness is that I'm not sure how close to state of the art the baselines are.

Questions

Why did you use this particular, somewhat strange, paper structure? Why is your calibration definition what it is? I'm a little confused: you said you replace $X$, but eqn. 5 still has it. Is this a typo? Is SVGP really state of the art? What about SKI [1] and its extensions? [1] Wilson, Andrew, and Hannes Nickisch. "Kernel interpolation for scalable structured Gaussian processes (KISS-GP)." International conference on machine learning. PMLR, 2015.

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

4 excellent

Limitations

Yes.

Reviewer jFQi7/10 · confidence 4/52023-07-12

Summary

This work at one level is about proposing a scalable GP approach called GPnn where the prediction step uses only the nearest neighbours of a test point in order to form the predictive distribution. This approach is well-motivated as the authors argue that there isn't a strong mathematical reason to couple the training step / estimation of hyperparameters and the prediction / generalisation over unseen inputs. At the end of the day, we care about the latter (prediction performance) and not really the former. At another level, there is a theoretical component where they show that predictive distributions obtained through GPnn are robust to a wide array of model misspecification, for instance, wrong kernel choice or Gaussian noise in the large data limit.

Strengths

The paper is well-written and clear. There is always a demand for new GP approximations which scale to millions of data points as SVGP has a known issue with over-estimating the aleatoric uncertainty. The algorithms clearly describe the simulation and evaluation step. There is a good summary of other distributed methodologies and clear delineation of how this work is different. Some components from this paper can be applied to other methods in order to refine estimates and calibration (Remark 5).

Weaknesses

Figure 2 is hard to read - there is a lot going on and ideally should have been plotted with N on the x-axis.

Questions

- What about a sensitivity analysis to the choice of M? or are GPnn predictions insensitive to this in the large data limit. - Can M be learnt instead of set? - Perhaps some areas of the input space require more M for good predictive performance than areas where the function isn't changing much. - It would be interesting to see the performance on non-stationary functions. - There isn't sufficient insight on the selection of nearest neighbours exact or approximate - at first blush one would dismiss the idea because of the added compute entailed in finding neighbours per test data point. What is the complexity of this as a function of N and d precisely?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

There is some discussion of limitations.

Reviewer 3iYQ2023-08-14

I have read the rebuttal. My score remains unchanged. I suggest the authors to compare at least against https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5927603/, at least describing the differences between the mentioned paper and their paper in a sentence or two: the cited work is quite well-known and thus I think such a comparison would benefit the readers.

Reviewer jFQi2023-08-14

Post rebuttal

Thank you for responding to the questions and I would encourage you to action point 0 and add details on 5 in the final manuscript. Otherwise, I am happy to persist my score.

Reviewer MqBs2023-08-15

Thank you for your response. You are right that SKI suffers from the curse of dimensionality. The extensions (e.g. [1]) have not yet taken the place of SVGP, although you should probably mention them and (if you have time) add a comparison of one of them in the final version. My score remains unchanged. [1] Yadav, Mohit, Daniel R. Sheldon, and Cameron Musco. "Kernel Interpolation with Sparse Grids." Advances in Neural Information Processing Systems 35 (2022): 22883-22894.

Authorsrebuttal2023-08-15

Thank for your further comments. We will follow up on the suggestions you have made and will endeavour to make comparisons in this paper, and certainly add SKI-based methods to future evaluations.

Reviewer N8D32023-08-15

Post-rebuttal comments

I appreciate the authors for their rebuttal and for other reviewers for their insights, and maintain the same assessment as before with a reduced confidence score. I have gathered more insight into the the interplay between GP models and its scalable kNN variants from the rebuttal and subsequent discussion from other reviewers. The paper “Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasets” mentioned by reviewer 3iYQ is an important precursor work and bridges this paper with the original full GP model: we do not need to see nearest-neighbor GPs as a straightforward approximation to GP models, but as a hierarchical model with a nearest-neighbor element in its own right. I believe that this paper gives us more insight in how to bridge hierarchical NN-GP with the GP model that uses all the training data points. I have, however, found that the theoretical results presented in the paper to have a quite limited impact on how we understand this connection. The paper considers the $n\rightarrow\infty$ limit behavior for virtually all its theoretical findings, but it is the infinite training data regime that yields many confounding results: when we are presented with infinitely many training data, the $m$ nearest neighbors of an arbitrary test point is itself (or a selection of points within an arbitrarily small ball surrounding the test point). In this asymptotic scenario, no kernel hyperparameter would ever matter as the kernel matrix converges to a matrix of all $1$s, hence the result negating the purpose of kernel learning from the asymptotic sense. This also shows why evaluating the variance parameter in the observation model still matters: the nearest neighbor observed value is essentially a collection of observation centered around the ground truth value with variance $\sigma_\xi^2i$. The paper presents a counterintuitive result in Remark 2 that isotropic kernels already suffice for best MSE in the asymptotic sense, and I argue that this line of counterintuitive reasoning could go one step further as we are in the domain of infinite data: kernels with infinitely small length scale is the optimal choice for all regression tasks, as it offers the best flexibility in the prior space and all the rest of Theorem 1 still holds. I unpack this line of reasoning in order to show that the infinite limiting behavior might tell very little about this model in a realistic setting, to an extent bordering on no realistic meaning. I am not, however, using this logic to discount, for example, the empirical convergence result presented in Figure 2. The author demonstrate in the rebuttal that they think evaluating GPnn using Algorithm 1 is fair practice, but I am still unsure whether it is true, and I phrase my question in a more straightforward manner. I assume training and test data should be generated as an entire set at first in one go, and then partitioned, but Algorithm 1 treats training and test data differently. Could you tell me why generating test data conditioned only on its nearest neighbors does not make it easier to predict based on its nearest neighbors? I agree after some re-consideration that applying GP models along with kNN carries some utility in certain applied domains (for example, geostatistics), and this is the main reason for the reduction in my confidence score. I remain unconvinced about this paper’s theoretical results, as I believe that a true leap in understanding this type of models would involve some degree of non-asymptotic analysis.

Authorsrebuttal2023-08-17

Thank you for your additional comments. We take on board and to some extent agree with your view on the importance of non-asymptotic results. We hope to produce such results that explicitly depend on the kernel parameters in the near future. Similarly, we understand your reservations and example regarding "infinitely small lengthscales" but argue that even without explicit convergence rate results, empirical results confirm that this approach (in less extreme circumstances) has very good performance and exhibits convergent behaviour, despite obviously existing in the finite data regime. In addition, we would point out that although rates obviously depend on the choice of kernel, as an example if we consider an isotropic kernel we can infer that the corresponding convergence will depend on the convergence of the distance between neighbouring input points, something which has been studied extensively in the classical kNN literature and shown to display gradual convergence rates (e.g. [1],[2],[3]). As such, we expect similarly gradual convergence for a large number of commonly-used kernels. Thank you for your insightful feedback and clarification regarding Algorithm 1. We now further understand your concerns, but remain confident in the validity of our conclusions pertaining to convergence behaviour. We believe that we can probe your concern by supplementing the current version with a computationally cheap deterministic function. Having applied this enhancement already to Figure 2 using the Oakley and O'Hagan function from Section 7.2, we can report that the plots maintain their core characteristics consistent with the original figure, while directly addressing the issues you raised. We can also run (smaller scale) versions to generate very similar plots to Figure 2 using data sampled directly from a GP and avoiding the use of Algorithm 1, further reinforcing our confidence in our conclusions. We hope that our successful reproduction of the salient features of figure 2 via alternative means will have already satisfied you on the matter of Alg. 1. However, we realise that the above does not fully explain its validity in response to your final question on the topic. Details are given below. If we include algorithm 1, in addition to other evidence, we will include a proof of its validity in the appendix. Define "Algorithm 1" as in the paper. As argued in our original rebuttal, the evaluations ${e_i^*}$ are iid RVs and so our estimator, $\frac{1}{n^*}\sum_{i=1}^{n^*} {e_i^*}$ is valid for the corresponding expectation. Define "Algorithm 1b" to be the procedure whereby we generate an $n$-length $x$ training set that we subsequently hold constant. For each generated test point $x^*$ we then generate an $(n+1)$-length GP sample $y$. We take the $m$ nearest-neighbours of the test point $x^*$ and evaluate the function $e(\cdot)$ to obtain ${e_i^*}'$. We repeat this $n^*$ times and compute a Monte-Carlo estimate of the expectation, $\frac{1}{n^*}\sum_{i=1}^{n^*}{e_i^*}'$. This method is clearly valid, albeit computationally very expensive. Finally, we show that the expectations in both cases are equivalent, where we begin with the case of Algorithm 1b and show that it is equivalent to that of Algorithm 1. We use $y^*$ to refer to the test observation, $y'$ to refer to the nearest-neighbours and $y''$ to the remaining disjoint observations. $y=(y^*,y',y'')$ refers to the full (n+1) length vector and hence $p(y)$ to the full joint distribution: $$ \frac{1}{n^*}\sum_{i=1}^{n^*} {e_i^*}' \approx E_{(y^*,y',y'')}[e(y^*,y',y'')] = E_{(y^*,y',y'')}[e(y^*,y')] = E_{(y^*,y')}[e(y^*,y')] \approx \frac{1}{n^*}\sum_{i=1}^{n^*} {e_i^*} $$ Since $e(\cdot)$ is only a function of $y^*,y'$. [1] L ́aszl ́o Gy ̈orfi et al. A Distribution-Free Theory of Nonparametric Re- gression. 2010., [2] Kohler, Krzyzak, and Walk, ‘Rates of Convergence for Partitioning and Nearest Neighbor Regression Estimates with Unbounded Data’, [3] Kulkarni and Posner, ‘Rates of Convergence of Nearest Neighbor Estimation Under Arbitrary Sampling’.

Authorsrebuttal2023-08-21

Edit

Please note that we have both amended and amalgamated our previous discussion comments via the editing process.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC