Approximation-Aware Bayesian Optimization

High-dimensional Bayesian optimization (BO) tasks such as molecular design often require 10,000 function evaluations before obtaining meaningful results. While methods like sparse variational Gaussian processes (SVGPs) reduce computational requirements in these settings, the underlying approximations result in suboptimal data acquisitions that slow the progress of optimization. In this paper we modify SVGPs to better align with the goals of BO: targeting informed data acquisition rather than global posterior fidelity. Using the framework of utility-calibrated variational inference, we unify GP approximation and data acquisition into a joint optimization problem, thereby ensuring optimal decisions under a limited computational budget. Our approach can be used with any decision-theoretic acquisition function and is compatible with trust region methods like TuRBO. We derive efficient joint objectives for the expected improvement and knowledge gradient acquisition functions in both the standard and batch BO settings. Our approach outperforms standard SVGPs on high-dimensional benchmark tasks in control and molecular design.

Paper

Similar papers

Peer review

Reviewer 6opG8/10 · confidence 3/52024-06-25

Summary

The authors present an extension of the sparse Variational Gaussian Process framework (SVGP) that is better suited for Bayesian optimization (BO) tasks. This is achieved by a new optimization criterion (EULBO) which aims to optimize the parameters of SVGPs s.t. the training data is fit well while achieving a better performance in selecting new solution candidates. Two concrete instantiations of EULBO (with Expected Improvement and Knowledge Gradient) are derived. An extensive empirical evaluation is conducted to demonstrate the effectiveness of the proposed approach.

Strengths

- though being dense, the paper is clearly written and straightforward to follow - the proposed approach is technically solid and mathematically sound - a relevant problem is addressed by an elegant solution - both, methodology and experiments are presented well

Weaknesses

## Methodology - there are some details unclear/missing regarding figure 1: - which points were used as inducing points? - why is the uncertainty relatively constant at the right side of the plot (left plot)? Depending on the choice of the inducing points, I'd expect the predicted variance (and thus the EI) to increase towards the right edge of the plot - what was the exact problem definition leading to Fig. 1? It would be good to see the exact parameterizations for reproducibility as it would help to gauge whether the motivational example is rather cherry-picked or whether the shown issue manifests across different parameterizations - missing $t$ in Eq. 2? shouldn't it be $\pi(f | \mathcal{D}_t)$? - in the equation after line 221 it seems that the sum does not depend on $j$. shouldn't it be $\mathbf{x}_j$? ## Experiments - why were the baselines only applied in conjunction with EI? (see line 259) - it seems that, depending on the choice of the objective and method, there are quite significant differences in the variance of EULBO. For example, in Fig. 2 "Media Molecules 1", TuRBO + ELBO EI has much higher variance than TuRBO + EULBO EI. Do you account this observation to EULBO?

Questions

see weaknesses

Rating

8

Confidence

3

Soundness

4

Presentation

4

Contribution

3

Limitations

the authors discussed the limitations of their approach in a detailed waysssssssssssssssssssssss

Reviewer jWXp7/10 · confidence 3/52024-06-27

Summary

The paper proposes a new method for training sparse Gaussian processes (GPs) for large-budget Bayesian Optimization (BO), designed to facilitate the sequential decision tasks inherent in BO. This is achieved by introducing the “expected utility lower bound (EULBO),” which formulates sparse GP training as a joint optimization problem of BO queries and posterior approximation. The proposed framework is compatible with other significant advancements in the field, such as Expected Improvement (EI) and Knowledge Refinement (KR) acquisition functions, TuRBO, and batch optimization. This compatibility was demonstrated across several BO benchmarks.

Strengths

The method proposed in the paper is novel and original. One of the main strengths is how the newly proposed EULBO integrates with important points in the literature. The authors effectively demonstrate how EULBO can be used in various contexts, such as with EI, KR acquisitions, and batch optimization. This helps distinguish their work from others in the field. The submission is technically sound, with derivations and experiments convincingly supporting the claims of the paper. The work is significant, and I anticipate it will be extended in the future or that some of the ideas presented will be adopted elsewhere.

Weaknesses

While the paper is generally well-written, many derivations were abbreviated and some of the notation was not properly explained, making the paper more difficult to read than necessary. These issues, however, are easily amendable. I provide some suggestions for improvement in the following subsections.

Questions

Introduction: (line 51): '\emph{minorize-maximize}' Section 2: (Fig. 1): It would be beneficial to show the SVGP model fit for EULBO as well. (line 110): Could you define \lambda, m and S? (Eq. between 114-115): q_\lambda is initially presented with one argument, then two. If the intention is for q_\lambda to represent all approximate distributions (in this case, p(f, u) ), please state this explicitly. It is natural to assume that q_\lambda(.) is just the Gaussian pdf (as defined in line 110), which creates confusion. (line 116): k(.,.) as a function also needs to be explicitly defined. Section 3: (line 123): This should probably refer to the equation between lines 114-115, not Equation 3. (Eq 4): Please define \Lambda. (Eq 6): Please use double integrals to indicate nested integration. Additionally, I don't understand the emergence of l(D_t |f) and the normalizing constant Z. I was under the impression that l(.|.) is the Gaussian likelihood as defined in line 113. Notice that D_t, as defined in line 92, contains both x and y. (line 147): 'corresponds to' (line 165): Please clarify where exactly the requirement for the utility to be strictly positive plays a role or comes from. (lines 173-175): Would using Gauss-Hermite quadrature for EI make EULBO significantly slower than ELBO? (line 189): What is y^{+}_t and what its relation to y_t? Question: Because you formulate the BO queries and GP approximation as a joint optimization problem without necessarily reducing the dimensionality of the problem, how much slower is this method compared to ‘approximation-ignorant’ BO?

Rating

7

Confidence

3

Soundness

3

Presentation

2

Contribution

4

Limitations

The authors are open about the limitations of their work, which they leave for future exploration. Specifically, the increased computational complexity seems to be a primary limitation. It would be beneficial if the authors could quantify this increased complexity, perhaps through reporting empirical runtimes or other relevant metrics.

Reviewer TCW17/10 · confidence 4/52024-07-07

Summary

This paper proposes a modification to sparse variational Gaussian processes (SVGPs) used in Bayesian optimization (BO) to better align the SVGP posterior approximation with the goal of optimizing an acquisition function. The key idea is to jointly optimize the SVGP and the acquisition function using a unified objective called the expected utility lower bound (EULBO). This approach ensures the posterior approximation is well-suited for the downstream decision-making task. The authors derive efficient EULBO objectives for the expected improvement (EI) and knowledge gradient (KG) acquisition functions and demonstrate improved performance over standard SVGPs on several high-dimensional optimization benchmarks.

Strengths

The paper presents a novel approach to aligning SVGP approximations with the goals of BO by jointly optimizing the posterior approximation and acquisition function. This is a creative combination of ideas from variational inference and decision theory. The proposed method is well-motivated and grounded in sound theoretical principles. The derivations of the EULBO objectives for EI and KG are clear and technically sound. The experiments are comprehensive, covering a range of high-dimensional optimization tasks, and the results convincingly demonstrate the effectiveness of the approach. The paper is well-written and easy to follow. The authors provide a clear exposition of the problem, the proposed solution, and the experimental setup. The figures and tables effectively communicate the key results. Scaling BO to high-dimensional spaces is an important problem, and this work represents a significant step towards more efficient and effective BO in such settings. The proposed approach is general and could potentially be applied to other acquisition functions and GP approximations, making it a valuable contribution to the field.

Weaknesses

The paper focuses specifically on SVGPs, but it would be interesting to explore whether the proposed approach can be extended to other sparse GP approximations, even those without a tractable ELBO. This would broaden the applicability of the method and strengthen the contribution. In Figure 2, the exact GP performs worse than the proposed method and some other baselines in the BO setting. This is counterintuitive, as one would expect the exact GP to be the gold standard when computationally feasible. The authors should provide more discussion and insights into this unexpected result. The paper presents several variations of the proposed method (e.g., EI-EULBO, KG-EULBO, batch versions), but it lacks a clear recommendation on which variant to use in practice. A more in-depth discussion of the trade-offs between these variants and guidance on when to use each one would enhance the practical value of the work.

Questions

Can the proposed EULBO approach be extended to other sparse GP approximations beyond SVGPs, even if they do not have a tractable ELBO? If so, what challenges would need to be addressed, and how might the optimization problem be formulated? Why does the exact GP perform worse than the proposed method and some other baselines in the BO experiments (Figure 2)? Is this due to the limitations of the exact GP in high dimensions, or are there other factors at play? What are the key factors to consider when choosing between the different variants of the proposed method (e.g., EI-EULBO, KG-EULBO, batch versions)? In what scenarios would one variant be preferred over another?

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors have addressed the limitations of their work in Section 6, acknowledging the increased computational cost of the EULBO approach compared to standard SVGPs. They also mention the need for multiple optimization tricks and the potential instability of the EULBO optimization problem. These are important limitations that users should be aware of when considering this method. However, the authors do not discuss any potential negative societal impacts of their work. While the proposed method is primarily methodological and does not pose immediate societal risks, it would be valuable for the authors to briefly comment on any broader implications of improved BO in high-dimensional spaces, such as the potential for misuse or unintended consequences in certain application domains.

Reviewer veTg7/10 · confidence 4/52024-07-09

Summary

The paper proposes a new approach for scaling Bayesian Optimisation to large datasets. Contrary to previous approaches, which fitted a sparse GP and optimised acquisition function independently, the paper proposes a method, which jointly optimises the variational parameters of sparse GP and searches for the next point to query. This is done by adding a utility term to ELBO. Authors discuss how this objective can be connected to generalised Bayesian Inference. Authors evaluate the proposed method on a number of benchmark with high number of datapoints and show it outperforms other baselines.

Strengths

- The paper touches on a very important subject of scaling Bayesian Optimisation to large datasets. I find the proposed solution to be very elegant. I also think the connection to generalised Bayesian Inference is very interesting. - The method seems to be delivering a significant improvement in sample efficiency over baselines. I believe the choice of benchmarks is sufficiently diverse and the number of baselines compared against is sufficient. - The paper is well-written and I really like that authors share the details of the tricks they used for improved optimisation of EULBO. I also like the fact that authors are very open about admitting the limitations of the proposed method.

Weaknesses

- As mentioned by the authors, optimisation of the EULBO objective is a bit cumbersome and seems this process is also slower than the optimisation of standard ELBO. However, it might be worth to sacrifice a bit of computational efficiency for an increase in the sample complexity. - While authors cite previous work, which proved that the selected action satisfies convergence guarantee, it does not directly prove anything about the performance of optimisation process choosing action in such a way (e.g. regret bound or convergence to optimum). The paper could be made much stronger if authors managed to connect those notions in some way. However, I also do not think it is critical, as the main contribution of the paper is empirical (although it would be greatly appreciated).

Questions

- Can the authors provide the running time of the proposed method (and baselines) ? It is ok if the proposed method is slower, but it would be good to quantify exactly how much slower it is. - Do authors have any expectations on how would the method perform in comparison to an exact GP on a smaller dataset? While I understand that the method is particularly designed for large datasets, it would be interesting to see how the utility VI-based acquisition strategy compares to standard acquisition functions like EI. It would be nice to have at least such experiment in the camera-ready version to judge whether the improvement delivered by the method comes from better acquisition strategy or better modelling

Rating

7

Confidence

4

Soundness

4

Presentation

4

Contribution

3

Limitations

Authors admit that the increased difficulty of optimising EULBO is a limitation of their work.

Reviewer 6opG2024-08-08

Answer to Rebuttal

Thank you very much for addressing the points I raised! The rebuttal clarified my points. There is only one question left from my side regarding motivation: Could you provide intuition on whether there are certain circumstances in which using ELBO instead of EULBO heavily leads to performance deterioration of BO with SVGPs? In the experiments, one can see that there are problem instances in which EULBO leads to significantly better results than ELBO. Still, in other problem instances, the difference is not that large. Overall, I'm happy to keep my score.

Authorsrebuttal2024-08-08

Regarding your question about motivation

This is a good question that we're not sure we have enough data to confidently answer. The benchmark problems we consider in this paper are a mix of problems that some were originally introduced in BO papers using non variational GPs, and others using them. For the ones that have already used variational GPs, there is obviously some survivorship bias where, if SVGPs catastrophically failed on those problems, we wouldn't have seen them. The other problems that originally did not use variational GPs like Rover give us at least some evidence about the performance degradation when switching to SVGPs with ELBO, and it at least does not seem to be catastrophic. We haven't encountered a problem where SVGPs with the ELBO have totally failed compared to known exact GP results, in part because they are often used in situations where evaluation budgets mandate them.

Reviewer 6opG2024-08-09

Answer to question about motivation

Thank you for the detailed answer, this clarifies my question.

Reviewer jWXp2024-08-09

Rebuttal Acknowledgement

I would like to thank authors for their response and addressing my concerns. I am happy to keep the assessment of their work the same.

Reviewer veTg2024-08-11

Thank you very much for responding to my review. I am happy with the response and I believe the paper is worthy of publication.

Reviewer TCW12024-08-11

Thank you for your response, which resolves my concerns and raises some very interesting and insightful discussion on the svgp.

Program Chairsdecision2024-09-25

Decision

Accept (spotlight)

© 2026 NYSGPT2525 LLC