Thin and Deep Gaussian Processes

Gaussian processes (GPs) can provide a principled approach to uncertainty quantification with easy-to-interpret kernel hyperparameters, such as the lengthscale, which controls the correlation distance of function values. However, selecting an appropriate kernel can be challenging. Deep GPs avoid manual kernel engineering by successively parameterizing kernels with GP layers, allowing them to learn low-dimensional embeddings of the inputs that explain the output data. Following the architecture of deep neural networks, the most common deep GPs warp the input space layer-by-layer but lose all the interpretability of shallow GPs. An alternative construction is to successively parameterize the lengthscale of a kernel, improving the interpretability but ultimately giving away the notion of learning lower-dimensional embeddings. Unfortunately, both methods are susceptible to particular pathologies which may hinder fitting and limit their interpretability. This work proposes a novel synthesis of both previous approaches: Thin and Deep GP (TDGP). Each TDGP layer defines locally linear transformations of the original input data maintaining the concept of latent embeddings while also retaining the interpretation of lengthscales of a kernel. Moreover, unlike the prior solutions, TDGP induces non-pathological manifolds that admit learning lower-dimensional representations. We show with theoretical and experimental results that i) TDGP is, unlike previous models, tailored to specifically discover lower-dimensional manifolds in the input data, ii) TDGP behaves well when increasing the number of layers, and iii) TDGP performs well in standard benchmark datasets.

Paper

References (23)

Scroll for more · 11 remaining

Similar papers

Peer review

Reviewer DvNb7/10 · confidence 4/52023-07-02

Summary

Techniques such as Deep Kernel Learning and extensions such as Deep GPs have been proposed in order to overcome the flexibility constraints associated with standard shallow GPs. However, although these techniques are better suited to model non-stationary data, they are still susceptible to issues such as pathologies when extended to more than a few layers, as well as overfitting. These techniques also lack the interpretability of the latent space and associated length-scales that make standard GPs so appealing. In order to mitigate these limitations, the authors propose a novel approach that generalises over the more widely-used compositional DGPs via a hierarchical model that incorporates locally linear deformations of stationary kernels. A synthetic example is crafted in order to showcase how TDGP improves over competing methods for problems exhibiting heavy non-stationarity. This is further complemented by an analysis on a real-world dataset, as well as 4 benchmark datasets from the UCI repository.

Strengths

- The paper is very well-written and a pleasure to read. I commend the authors for structuring the paper in a manner that clearly shows connections to related models, while also showcasing the key contributions of this work. - I particularly appreciated Sections 4.1 and 4.2, which capture the various dimensions along which TDGP can lead to improvements over other methods (predictive performance, uncertainty quantification, and interpretability). - The problem statement tackled in this paper is well-motivated, and the limitations of other techniques are clearly pointed out both verbally and visually in the experiments section. One possible improvement to this could be highlighting application domains where TDGP is expected to be most impactful – i.e., which application domains tend to non-stationary data where well-calibrated uncertainty is especially useful. While fairly minor, this could help clarify the significance of this work earlier on the paper.

Weaknesses

- The authors comment in the *Limitations* section that the increase in the number of variational parameters could slow down optimisation – it could be worth supporting this statement by specifying the computational complexity, and maybe an analysis reporting wall clock time for one of the experiments in Section 4. - I would also be interested in whether the increased parametrisation of the model results in models that are less stable or difficult to consistently converge. This has implications on the practical utility of the model. - The paper focuses heavily on using the SE kernel throughout – can this be extended to other kernels or is the value-add of this extension not considered to be worthwhile for the paper?

Questions

I do not have additional questions beyond the points raised in the *Weaknesses* section. I would appreciate if the authors could read through this and indicate whether they can improve upon these aspects in a future revision. The paper is a solid addition to the literature on hierarchical GP models – besides proposing a novel method, it also gather and illustrates potential deficiencies in existing methods that are widely-used in the community. I also believe that the paper is already in a state that is fit for publication, and can only be improved further with additional minor iterations.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

N/A

Reviewer HPwG4/10 · confidence 4/52023-07-03

Summary

This paper extends the CDGP to address two weaknesses in deep Gaussian processes. The solution is similar intuitively to a residual connection. Instead of letting each layer depend only on the last layer, they let each layer depend on the last layer and the input. This allows the model to induce a manifold and use a lengthscale field.

Strengths

This seems like an important problem, and the solution seems principled.

Weaknesses

There should be more discussion of what it means to not induce a manifold, why violating the triangle inequality leads to that, and why not inducing a manifold is a drawback. I think inducing a manifold implies that one can make the latent space plots: why are these valuable? Why do they require the triangle inequality? There should be more discussion of how a lengthscale field leads to interpretability. In both the latent space and the inverse lengthscale plots, there should be some discussion about why we want to see these plots: for instance, in figure 7, we have the two sets of plots: what can we glean from them about the data/problem/etc? Figure 8 has a little bit (the zone of high correlation extends into the mountains), but this is not obvious for a non-expert application reader. You could have a short tutorial perhaps of how to read these plots in the supplement.

Questions

Most of my questions are in the weaknesses.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

They have a good discussion of limitations: mainly that it is computationally expensive and leads to requiring more parameters.

Reviewer CrCc4/10 · confidence 4/52023-07-04

Summary

This paper considers the problem of non-stationary kernel design for Gaussian processes. Two main approaches for this problem are deformation kernels and length-scale mixture kernels. On one hand, deformation kernels, eg deep GPs, have found great success while trading off expressivity and the ability to learn latent manifolds in intermediate layers, for lack of interpretability and being prone to pathologies. On the other, length-scale mixture kernels retain interpretability but typically break the triangle inequality in the induced latent space, precluding a manifold structure. The authors propose thin and deep GPs, which aim to combine the benefits of both these approaches, while mitigating their drawbacks. The approach learns a position-dependent matrix, parameterized by a GP, which learns a pointwise linear transformation of the data. The output of such a layer can then be composed to learn a new matrix acting on the original data, enabling a deep learning structure. Numerical examples are provided which appear competitive with existing techniques.

Strengths

This paper is clearly presented and combines existing ideas in a novel way. The numerical examples seem well selected, with a mixture of synthetic/analytic examples that demonstrate the utility of the method, with standard benchmark datasets allowing for meaningful comparisons on real data. A case study is also provided that is geared towards the strengths of the technique.

Weaknesses

The main weakness of the paper in my view is Theorem 3.1. It appears that the proof that the CDGP is a generalization of deep GPs hinges upon the introduction of a new variable that plays the role of the deep GP, and then turning off their method's parameters by concentrating their priors around zero. This seems highly misleading unless the method is introduced in a way that uses the augmented space as default, or some numerical evidence is given that it should be turned off if appropriate. In this case, presumably the deep GP should also be tested with and without an augmented space for fair comparison. It seems that the augmented space is not used in any of the experiments in any of the models presented, since the local linearity leaving the data unchanged around zero is listed as a limitation.

Questions

Have you tested the method with and without the data being augmented with a bias term? Has this been tested in prior methods as well? Since the matrices are position dependent, the dimension of their range can vary across the domain as a different number of their singular values. How does this influence the latent manifold interpretation that motivated your work?

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Besides those listed above, the computational expense is listed as a major limitation. This is a common drawback of related work, and it may be helpful to list the memory and computational complexity of this method with L layers, as is commonly done in the support literature.

Reviewer 6DJk7/10 · confidence 3/52023-07-13

Summary

The paper presents a new deep Gaussian process (DGP) model, the thin and deep GP (TDGP), which does not suffer from a diminishing signal as the number of layers increases. The crux of the TDGP model is the covariance function that, for each layer $\ell$, acts on a linear combination of a (non-linear) transformation of the outputs of the previous layer and the inputs $\mathbf{x}$, $\mathbf{h}^\ell(\mathbf{x}) = \mathbf{W}^\ell(\mathbf{h}^{\ell-1}(\mathbf{x})) \mathbf{x}$. This hierarchical construction is shown to cover the usual DGP models while not suffering from diminishing signals as the depth increases. To perform inference in this model, the authors adopt a mean-field VI scheme, which is demonstrated to be effective through a number of experiments on both synthetic and real-world data.

Strengths

1. The paper presents an interesting and original approach to DGPs, an important field that should interest the NeurIPS community. 2. The paper is of high overall quality and very well-written. 3. The presented model has the potential to be of considerable significance, given that it generalises previous DGP models while not suffering from the same pathologies.

Weaknesses

1. While I think the paper presents a great idea with lots of potential, my main concern is the lack of a deeper analysis of the model. Given that the proposed model is essentially just a particular transformation of the inputs at each layer and a rather straightforward mean-field VI scheme, I would expect some insights into the behaviour of the model, even if these were just empirical. The current experiments do present some interesting insights, but, say, what happens to the model's performance as we increase the depth of the model? What if we change the width of the layers? What if we use different covariance functions than the squared exponential? How does the induced covariance matrix look at different layers of the model compared to, say, DNSGP? Why does the model encourage low-dimensional latent spaces, and does this happen too for models deeper than just two layers? A Gaussian prior shouldn't encourage sparsity, so this aspect of the model is particularly puzzling. 2. As mentioned, a standard mean-field VI scheme seems very rough given that this is known to perform poorly for many models, including DGPs. To my knowledge, the current state-of-the-art is still the doubly stochastic VI scheme by Salimbeni & Deisenroth (2017), so it would make sense to try something similar (and unless I misunderstand something, it seems fairly straightforward to do this for the proposed model). 3. One clarity issue (perhaps the only one) I find with the paper is that it is unclear to me which DGP model the authors refer to as "CDGP". Deep GPs have evolved dramatically from the original formulation of Damianou & Lawrence (2013), and the mentioned pathologies are not as pronounced anymore (to my knowledge). It is great that the authors compare against DNSGP, but comparing against the doubly stochastic DGP rather than the original formulation makes much more sense. As it is unclear if the authors actually compare to the current state-of-the-art, it is also unclear if the proposed model addresses actual problems. 4. Experimentally, the model seems to work well, but the authors use only two layers for all models. This is particularly strange as a key selling point of the proposed model is that it doesn't degenerate as the number of layers increases. But it is also an odd choice since deeper models should work better (and DNSGP demonstrably does for two of the UCI datasets). It is also strange that MRAE statistics are only reported for the GEBCO dataset; these should also be reported for the UCI experiments. 5. The model is claimed to be "interpretable", but this only seems to be the case for a two-layer model (i.e., one with a single latent space). For deeper models, the distance matrix is still a highly non-linear function of the inputs, which I don't think will be particularly interpretable. References: - Andreas Damianou & Neil Lawrence, "Deep Gaussian processes", AISTATS 2013. - Hugh Salimbeni & Marc Peter Deisenroth. "Doubly Stochastic Variational Inference for Deep Gaussian Processes", NeurIPS 2017.

Questions

1. Which specific variation of the DGP model is referred to as "CDGP"? 2. In lines 138-139, you say that "By placing zero-mean priors on the entries of $\mathbf{W}$, we encode an inductive bias to maximally reduce the latent dimensionality of $\mathbf{h}$." Why is this? Intuitively, since the prior is Gaussian, it shouldn't encourage sparsity. 3. Your contribution seems to be an input transformation, which isn't tied to a specific covariance function, so is there a reason that you chose the squared exponential (or exponentiated quadratic) covariance function? The Matérn class is more often used in practice, I believe. 4. Is a model with more than two layers still interpretable? If so, how would you interpret the latent space? 5. In Figures 4 and 7, how is the inverse length scale computed for TDGP? Also, why can we not compute the inverse length scales for CDGP or DKL? And why can we not visualise the latent space for DNSGP? 6. In lines 51-52, you mention that TDGP is the only deep architecture that induces data embeddings. What do you mean by "data embedding" here, and why do other models not have this property? For instance, even for the original DGP model, we can easily visualise the distribution of a particular sample at a particular layer.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The authors have adequately addressed limitations in a dedicated paragraph.

Reviewer CrCc2023-08-18

Response

I have read your response as well as the responses to the other reviewers. I would suggest the authors take a closer look at their construction/justification as a generalization of CDGPs before revising this work, and consider changing the way this is pitched. I would like to make it clear that I think the idea is a good one, but the presentation needs work.

Authorsrebuttal2023-08-18

We are glad you think our paper's idea is good. We would be grateful for further details/suggestions on how to improve the specific issues you noticed and how you would suggest changing our pitch, as we want to address your concerns and present our ideas in the best possible way.

Reviewer CrCc2023-08-19

Theorem 3.1

My main apprehension is with Theorem 3.1 as I stated in the initial review. The justification as a generalization of CDGPs seems flimsy at best. While it is true that bias terms exist elsewhere in machine learning, introducing an artificial term that allows you to introduce the other model, then turning your model off, and then claiming that this makes CDGPs a special case of TDGPs seems dishonest. It would be a different story if the intermediate models performed well, even if the best performance is at the endpoints. However if that were the case you would be writing about the whole family of models rather than the model with the `bias' term removed. This is reflected in the experiments you mentioned in your response. In my view the inclusion of this justification weakens the entire paper. It seems too forced, when no such theoretical justification as a generalization of CDGPs is required in order for your model to be a valid contribution, and it is not an important fact. While your construction is more refined that interpolating between your model and an arbitrary one, then setting parameters to turn the other one off, it has the same flavor and cheapens your otherwise good work.

Authorsrebuttal2023-08-20

Recontextualizing Theorem 3.1 for the next revision of the main text

Thank you for detailing your concerns. We want to highlight that this claim is only present in select sections, as part of the contribution section, the theorem statement, and the conclusion. Nonetheless, our goal with this theorem was to prove that our proposed model (TDGP without bias) has a deep connection with the usual CDGP model, as both can be considered different limits of a more general model (TDGP with bias). This property is not shared with other non-stationary models, such as DNSGP, and we believe that this perspective of the deep models as locally affine maps is of interest to the DGP community. Regardless, we still object to the impression that TDGPs as a generalization of CDGPs is dishonest, as the introduction of the $d(\cdot)$ terms is not artificial or forced by us but a consequence of the well-known connection between linear transformations (TDGP) in $R^d$ and affine transformations (CDGP) in the projective space of $R^{d+1}$. We emphasize that the only modeling choice in the theorem is adding bias and setting the last row of $\tilde{W}$ to $[0, 0, \ldots, 1]$ (otherwise the bias dimension would not be 1 in the next layer) and that these choices are valid under the description of the TDGP model in equation 8, for example. However, in the interest of dispelling possible misunderstandings, we would like to rephrase the lines where this theorem is referenced: * (Lines 53 & 54) **We prove that TDGPs and compositional DGPs are the limits of a more general DGP construction. Thus, we establish a new perspective on standard CDGPs while reaping the benefits of inducing a lengthscale field.** * (Line 130) **Importantly, however, TDGP are related to CDGPs as both can be seen as locally affine deformations of the input space (proof in Appendix A)** * (Lines 258-260) **Since TDGP is based on deformation kernels and is related to CDGPs, we believe it can leverage many of these advances in inference, which is left to future investigations.** * (Lines 272-274) **Additionally, while TDGP shares a connection with CDGP, our prior does not concentrate on "flat" samples when the number of layers increases — as is the case with CDGP** Would these changes ease your concerns and elevate the overall contents of the paper?

Reviewer 6DJk2023-08-18

Dear authors, Thank you very much for your elaborate answers and clarifications, and many apologies for not getting back to you sooner. After reading your (impressive!) rebuttals to myself and the other reviewers, I feel much more positive about the paper. You have clarified many things for me and addressed my concerns in an impressive way. In particular, if you can add discussions like the one you have provided in your response to my questions regarding the induced covariance matrix (W1) and deeper models (W4), I think your paper will be stronger. I understand that you might want to keep the theoretical analysis to just two layers, but showing experiments with more, as you have done in your rebuttal, would at least reassure the reader that the model is not limited (in practice) to two layers. Having these in the supplementary would be just fine. The additional analyses and explanations you have provided to all of us here have given me a significantly better grasp of the paper, and I encourage you to incorporate them into your paper (or appendix). I have updated my score and will recommend acceptance. A minor thing I have just noticed, in line 172, you refer the reader to the appendix for additional information regarding the synthetic experiment, but I can't seem to find it there.

Authorsrebuttal2023-08-18

Thank you for recommending the acceptance of our paper and alerting us about the missing details concerning the synthetic experiment in the appendix; we will correct this issue and reference the additional figures in the main text and the supplementary material.

Reviewer DvNb2023-08-18

Acknowledgement of rebuttal.

Thank you for your detailed replies - I have read the other reviews as well as the consequent discussions, and I still consider this paper to be a valuable addition to the literature on (deep) GPs. Coupled with the fact that the paper was already well-written and presented in its original submission, for the time being I am keeping my score for the paper.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC