Efficient Robust Bayesian Optimization for Arbitrary Uncertain Inputs

Bayesian Optimization (BO) is a sample-efficient optimization algorithm widely employed across various applications. In some challenging BO tasks, input uncertainty arises due to the inevitable randomness in the optimization process, such as machining errors, execution noise, or contextual variability. This uncertainty deviates the input from the intended value before evaluation, resulting in significant performance fluctuations in the final result. In this paper, we introduce a novel robust Bayesian Optimization algorithm, AIRBO, which can effectively identify a robust optimum that performs consistently well under arbitrary input uncertainty. Our method directly models the uncertain inputs of arbitrary distributions by empowering the Gaussian Process with the Maximum Mean Discrepancy (MMD) and further accelerates the posterior inference via Nystrom approximation. Rigorous theoretical regret bound is established under MMD estimation error and extensive experiments on synthetic functions and real problems demonstrate that our approach can handle various input uncertainties and achieve state-of-the-art performance.

Paper

References (32)

Scroll for more · 20 remaining

Similar papers

Peer review

Reviewer UjwV5/10 · confidence 3/52023-07-03

Summary

The paper proposes a new method for Bayesian optimization under uncertain inputs, where the distribution of an input can be complex and unknown but can be sampled from. The paper proposes an MMD based kernel between probability distributions and the use of the Nystrom approximation to make the computations tractable. A UCB-type algorithm is used. The paper provides regret bounds accounting for the approximation error due to sampling / Nystrom approximation, and compares the algorithm to some baselines.

Strengths

1. The motivation of requiring to do BO with uncertain inputs with complex distributions that may not be known in closed form is compelling. 2. The regret bound incorporating approximation error and the corresponding implications on the sampling size $m$ are useful and interesting. In general I believe the work has potential given that the technical issues raised in the Weaknesses section are sufficiently addressed.

Weaknesses

1. **Validity of kernel**. The paper designs an MMD-based kernel between probability distributions $\hat k = \eta(\text{MMD}(P, Q))$. When designing a new kernel, it is important to prove that it is a valid kernel. However, no such proof is given (or references to previous literature that does something similar). $\eta$ is said to be a "normalizing function with range $[0, 1]$". What conditions does $\eta$ need to fulfill in order for $\hat k$ to be a valid kernel? 2. **Empirical evaluation**. The empirical evaluation is unconvincing for the following reasons: * The Nystrom approximation is adopted for efficiency. It is important to measure how much using this approximation affects performance beyond simply a qualitative comparison by looking at the posteriors in Figure 3. The empirical evaluation should do an ablation study that includes MMDGP without the Nystrom approximation, at a sampling size that results in the same inference time as that with the Nystrom approximation. What if removing the Nystrom approximation results in better performance with no decrease in efficiency? * The closest work seems to be Oliveira et. al. (2019). Why isn't their algorithm one of the baselines? * The error bars are simply too large and overlap too much. Any performance improvement could be due to randomness; please run with more trials to decrease the error bars so that the performances can be meaningfully compared. 3. **Writing**. The writing has much to be improved: * Line 174 "One important theoretical guarantee to conduct GP model..." and in Theorem 1 "...running Gaussian Process with acquisition function..." I believe you mean "Bayesian optimization" in these contexts instead of "Gaussian process". The GP is the model, the algorithm is BO which relies on a GP. The thing being run is the algorithm, not the model. * Line 151 "sampling size $N$" do you mean $m$? * Many grammatical and typo errors: the last word in the title is not capitalized; line 178 "For $\hat k$ be radial kernels" and "For $\hat k$ be linear kernel", should be, for example, "If $\hat k$ is a radial kernel"; Lines 201 and 205 are missing periods; Line 256 "can be more pronounced impact" should be "can have". * [24] and [25] point to the same reference.

Questions

No additional questions.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

1 poor

Contribution

3 good

Limitations

Yes.

Reviewer JQUf5/10 · confidence 4/52023-07-05

Summary

This work focuses on the situations where input uncertainty arises and the input values are unobservable, and introduces to measure the distance of uncertain inputs through MMD when training the Gaussian process surrogate. The authors theoretically and empirically demonstrate the effectiveness of the proposed method.

Strengths

1. The proposed method is sound and MMD can be a metric to measure the distances of uncertain inputs. 2. The experiments are extensive to show the effectiveness of the proposed method.

Weaknesses

1. MMD-GP needs to query m times more samples than GP, which can restrict its application since BO is usually applied to the tasks where evaluating a query can be time-consuming. What about the time cost of this work compared to other methods for input uncertainty? 2. After sampling m times for one query x, we can also get m rewards/performance. I wonder how to deal with the m rewards/performance? Line 139 shows that $D_n=${$(x_i, y_i)|x_i \sim P_i$}. However, it seems that $y_i$ also should be a distribution. 3. Other typos: 3.1 Eq. 2 use $\xi$ to denote the noise while line 92 utilize $\epsilon$. 3.2 Line 151: “the computation and space complexities of the empirical MMD estimator scale quadratically with the sampling size N” Should N be m? 3.3 What does the training and testing samples stand for in line 162? Do you mean the number of observations and the number of queries?

Questions

Please see the weakness.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

No broader societal impacts are provided.

Reviewer JQUf2023-08-21

Thanks for your response and I have read the rebuttal. The authors resolve my concerns and I would like to maintain the rating.

Reviewer Yh2J6/10 · confidence 4/52023-07-06

Summary

The paper proposes a novel Bayesian Optimization (BO) algorithm that explicitly tackles input uncertainty by introducing a new integral probabilistic metric (IPM)-based kernel. The algorithm also utilizes an efficient and stable Nystrom estimator to approximate the Maximum Mean Discrepancy (MMD), which serves as the adopted IPM. Furthermore, the paper extends the GP-UCB framework by incorporating the proposed kernel and derives the corresponding upper bound for the cumulative regret. The empirical study illustrates the effectiveness of the proposed approximation and the BO algorithm.

Strengths

1. The paper is generally well-organized and illustrative with figures and tabular results. 2. The theoretical result extends the existing UCB work with reasonable approximation to tackle the input uncertainty. 3. The limitation of the proposed AIRBO on dealing with high-dimensional input and lack of discussion over other IPMs is clearly stated.

Weaknesses

1. The theoretical bound is arguably sound. Due to the numerical approximation of the Maximum Mean Discrepancy (MMD), it may potentially result in pseudo-metric or even worse outcomes in practice. Therefore, it remains unclear whether the upper bound of the maximum information gain $\gamma_T$ still holds and guarantees that the UCB cumulative regret is sublinear. 2. The essential justification for the proposed Nystrom estimator lies in assumption 1. While the author claims in lines 210 and 211 that the approximation error can be fairly small, the paper lacks sufficient discussion or substantiation to support this claim. 3. The empirical results are limited. The regret curve lacks statistical significance, and more extensive empirical studies on real-world applications would be desirable.

Questions

1. What is SKL-UCB mentioned in line 262? Is it referring to the algorithm discussed in line 45 from [19]? 2. What is the corresponding reference for Oliveira et al. mentioned in line 50? 3. Despite the explicit treatment of input uncertainty, it is unclear to me whether tolerating the input uncertainty by fixing a larger output noise level could provide a simpler solution. The author claims that the GP model overfits (line 222) and justifies the use of MMD-GP. However, one would expect that fixing a larger noise level for the GP model would help mitigate it as well. Additionally, the existing method addressing heteroscedastic noise proposed in [1] potentially aids in mitigating the input uncertainty purely on the output side. It would be valuable to hear the authors' comments on this. 4. Could the author provide more intermediate quantitative evidence showing the effectiveness of the proposed MMD-based kernel? For example, any evidence showing that the uncertainty quantification is better than the naïve GP model that doesn't explicitly deal with the input uncertainty?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

LImitations are mentioned in the comments above.

Reviewer jZyz6/10 · confidence 4/52023-07-08

Summary

The paper tackles the problem of robust Bayesian Optimization (BO) with uncertain inputs, i.e., the input values are deviated from the intended value before evaluation. The paper proposes a new technique, namely AIRBO (Arbitrary Input uncertainty Robust Bayesian Optimization), that can model the uncertain input and incorporate this uncertainty into the surrogate model, and thus can be used to guide the search of the objective function global optimum. The paper further proposes to use Nystrom approximation to reduce the computational cost. Theoretical analysis is conducted to guarantee the performance of the proposed technique. Experiments are conducted on some synthetic and one real-world problem to evaluate the performance of the proposed method.

Strengths

+ The paper’s writing is generally clear and easy to understand. Illustration and figures are plotted nicely to explain the problem setting, as well as the property of the proposed method. + The proposed method of using MMD to construct a kernel that can incorporate uncertainty of the inputs seems to be interesting to me. + Theoretical analysis is conducted to guarantee the convergence property of the proposed method (although note that I have some concerns regarding the theoretical analysis). + The experiments are described very detailed. Various experiments are conducted to understand the performance of each component within the proposed method.

Weaknesses

+ I think the application of this problem setting should be motivated much better as it’s unclear on the significance of the problem tackled in this paper. In the Introduction, the paper only mentions that this problem is quite common for robotics and process controls, but no references or further explanations are provided. + Some notations are quite confusing, making it hard for me to follow the proposed method. In Eq. (8), the paper shows how to compute MMD(P,Q) from the m samples {x_i}_1^m, {y_i}_1^m – but y is defined as the objective function value. Then how can we compute the kernel \hat{k}(P_{x_i}, P_{x_j}) in Eqs. (10) and (11)? + I also have some concerns with the theoretical analysis. In the theoretical analysis, there is no formal proof to prove the proposed kernel is actually a valid kernel. There is just one paragraph in Lines 170-173 briefly mentioning about this but no formal proof is given to know if the proposed kernel is valid, and in which scenarios it will be a valid kernel. Assumption 1 is too strong, it assumes that we already know that the error function e(P,Q) can be uniformly upper-bounded. In practice, can this assumption be true? How can we know it can be true? Theorem 1 also doesn’t have much meaning to me. In Theorem 1, \tilde{\beta}_n is defined with the maximum information gain \tilde{I} within the formula. This is not really a standard analysis in BO’s theory, normally \beta is defined by some constants. Is this maximum information gain \tilde{I} bounded? + Experiments are mostly conducted with synthetic functions. There is only one real-world problem being used in the experiment, and it’s quite simple to me. It’s just a 3-dim problem. + What is the time cost of the proposed method?

Questions

Please answer my comments and questions in the Weaknesses section.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

I can’t find any dedicated section that describes about the limitations of the proposed technique. There are some future work mentioned in the Conclusion, but there are no limitations mentioned there.

Reviewer UjwV2023-08-11

Additional concern

Thank you for your response, most of my previously raised concerns are decently addressed. I have increased my score to reflect this. I have one more qualm regarding the comparison of your kernel to that developed in Oliveira et. al. (2019), and to the kernels developed in "Learning from Distributions via Support Measure Machines" by Muandet et. al. (2012) and "Universal Kernels on Non-Standard Input Spaces" by Christmann and Steinwart (2010). The kernel developed in Oliveira et. al. (2019) has the form $k(P, Q) = \langle \psi_P, \psi_Q \rangle_{\mathcal H}$, and they state below this definition that "Besides the linear kernel in Equation 9, many other kernels on $\mathcal P$ can be defined via $\psi$, e.g. radial kernels using $\lVert \psi_P - \psi_Q\lVert_{H}$ as a metric on $\mathcal P$ (Muandet et al., 2012). " Since your kernel is defined as $k(P, Q) = \exp(-\alpha \lVert \psi_P - \psi_Q\lVert_{H})$ this seems to be the same as what was suggested. Muandet et. al. (2012) seems to have tested this also as a "Level 2 RBF", and seems to be very similar to that developed in Christmann and Steinwart (2010) Lemma 2.3. My concerns concretely are the following: 1. The fact that the proposed kernel is not new must be clearly stated and these links to previous works must be clearly discussed. On reading the paper in its current form, one has the impression that the proposed kernel is part of the work's novel contribution. 2. The work of Muandet et. al. (2012) suggests that there is an entire family of kernels over probability measures, of which the one in Oliveira et. al. (2019) and yours are special cases. The Nystrom approximation will be able to decrease the computation time for all of them, since they all rely on inner products between sums of kernel matrix entries and therefore all rely on the kernel matrix. I believe that this work can be made significantly more general by not focusing on the case of RBF with MMD as a metric, and since Theorem 1 does not rely on this specific kernel anyway. Or is there a good reason that there is a particular focus on this specific kernel?

Authorsrebuttal2023-08-12

Response to kernel design

We appreciate the reviewer's thorough understanding of this paper and insightful comments! + We come to this MMD-based kernel design from an intuition that, if we can find a good metric that can characterize the distance btw probability measures with a rich expressive power, we should be able to devise a robust GP based on it and guide the search for robust optimum. + This intuition turns us into a family of Integral Probabilistic Metrics (IPMs). Among them, the MMD grabs our attention because of its intrinsic connection with distance measurement in RKHS and high coincidence with the kernel trick (ref. Section 3.1 of our paper), which eventually leads us all the way to the current idea. + Given two distributions P and Q, the MMD [1] is defined as:$MMD(P, Q) = \\sup\_{f \\in \\mathcal{F}} \\big( \\mathbb{E}\_{x}[f(x)] - \\mathbb{E}_{y}[f(y)] \\big)$. By setting the function class $\mathcal{F}$ to a unit ball in an RKHS $\mathcal{H}$, the MMD can be expressed as a distance btw the mean embeddings in RKHS. In this case, our kernel becomes $k(P, Q)=\exp{(-\alpha \Vert \psi_P -\psi_Q \Vert_H)}$, which converges to the kernel family in [2] (The uGP kernel in [4] is a specialization of [2]). + Though our theoretical analysis is developed upon the RKHS, **MMD is not limited to RKHS and our kernel can go beyond the kernel family in [2]**. In fact, any function class $\mathcal{F}$ that comes with uniform convergence guarantees and is sufficiently rich can be used, which gives different expressions of MMD [1]. For example, MMD converges to the Kolmogorov metric if $\mathcal{F}$ is a class of bounded variations. Also, MMD can be expressed as other Earth-mover distances, say Wasserstein distance, with proper choice of function class $\mathcal{F}$. In this field, there are some active researches, e.g., [5, 6, 7], to design novel kernels with these IPM metrics. **In the updated manuscript, we will clarify this point clearly and discuss the connections with these existing works as the reviewer suggested.** Again, thanks for the constructive suggestion on the generalization direction. As pointed out by the reviewer, our theoretical analysis in theorem 1 does not rely on an assumption of the kernel in its current form and can be generalized. As the main scope of this paper is a practical method for robust Bayesian optimization, we will **add a discussion for this generalization direction and leave the comprehensive exploration in future work.** **Reference** [1] Gretton, Arthur, et al. "A kernel two-sample test." *The Journal of Machine Learning Research* 13.1 (2012): 723-773. [2] Muandet, Krikamol, et al. "Learning from distributions via support measure machines." *Advances in neural information processing systems* 25 (2012). [3] Christmann, Andreas, and Ingo Steinwart. "Universal kernels on non-standard input spaces." *Advances in neural information processing systems* 23 (2010). [4] Oliveira, Rafael, Lionel Ott, and Fabio Ramos. "Bayesian optimization under uncertain inputs." *The 22nd international conference on artificial intelligence and statistics*. PMLR, 2019. [5] Carriere, Mathieu, Marco Cuturi, and Steve Oudot. "Sliced Wasserstein kernel for persistence diagrams." *International conference on machine learning*. PMLR, 2017. [6] Oh, Jung Hun, et al. "Kernel wasserstein distance." *arXiv preprint arXiv:1905.09314* (2019). [7] De Plaen, Henri, Michaël Fanuel, and Johan AK Suykens. "Wasserstein exponential kernels." *2020 International Joint Conference on Neural Networks (IJCNN)*. IEEE, 2020.

Authorsrebuttal2023-08-15

A warm reminder on the last day of the rolling discussion

We kindly ask the reviewers if they have any outstanding questions or clarifications regarding our paper. We are happy to engage in a dialogue and conduct any additional requested work in the remaining discussion period. Thank you!

Reviewer UjwV2023-08-15

Thank you for your response. While other distances such as the Wasserstein distance may be a special case of the MMD depending on the function class $\mathcal F$, in the context of your work, if $\mathcal F$ is not the RKHS associated with $k$ and the MMD is not a sum of kernel matrix entries, the Nystrom approximation is not applicable. Your work seems to only be applicable to the kernel family, of which uGP and the MMD in your paper are special cases. Once again, I believe that the work can be made much more general by properly discussing this fact. The empirical section ideally would then involve a few members of this kernel family, but I understand that that will take time. I believe that you will incorporate the lengthy discussion we have had into the next iteration of the paper and properly position your work in the context of the BO and kernel methods literature. I have increased my score further to reflect this.

Authorsrebuttal2023-08-15

Thank you for the kind review

We thank the reviewer again for the considered comments and in particular for suggesting the generalization direction of our idea! We will clarify these points clearly and discuss the connections to the existing works as suggested in the updated paper.

Reviewer Yh2J2023-08-15

I appreciate the authors' detailed response to the concerns regarding the soundness of the theoretical and empirical results. I believe the new results could help alleviate the issues and enhance the overall presentation. However, the new results in Figure 8 demonstrate an improvement in uncertainty quantification but do not seem to contribute significantly to the optimization, especially when compared to uGP. Additionally, the results in Figure 9 still lack significance. I still have some reservations about the contribution of explicitly learning the input uncertainty for general optimization. Nonetheless, I appreciate the general contribution of this paper within this context.

Authorsrebuttal2023-08-15

Additional explanation

Thanks for your feedback. Here we would like to explain a little bit more about the experiment setting and results, hoping it can help dispel some concerns. In Figure 8, we aim to compare the modeling performances of different surrogates given the same set of training points. We design the input distribution to be a "step-changing" Chi-square distribution, whose $df = 0.5$ if $x\in[0.0, 0.6)$) and $df$ changes to 7.0 when $x\in[0.6, 1.0)$ [1]. Due to this sudden change, the uncertainty at point $x=0.6$ is expected to be asymmetric: i) as the Chi-square distribution is quite lean and sharp when $df=0.5$, the distance from $x=0.6$ to its LHS points, i.e., $x_{lhs} \in[0.0, 0.6)$ are relatively large thus their covariances are small, resulting a fast-growing uncertainty. Meanwhile, ii) when $x\geq 0.6$, the $df$ changes to 7.0, rendering the input distribution a quite flat one with a long tail at RHS. Therefore, the distances btw $x$ and its RHS points become relatively small, which leads to large covariances and small uncertainties for points in $[0.6, 1.0]$. As a result, we expect to observe an asymmetric posterior uncertainty at $x=0.6$. We have employed several surrogates in this test, including: + *MMDGP-nystrom(160/10)* is our method with a sampling size $m=160$ and sub-sampling size $h=10$. Its complexity is $O(MNmh)$, where $M$ and $N$ are the sizes of training and test samples (Note: all the models in this experiment use the same training and testing samples for a fair comparison). + *uGP(40)* is the surrogate from [2] and employs an integral kernel with sampling size $m=40$. Due to its $O(MNm^2)$ complexity, we set the sampling size $m=40$ to ensure a similar complexity as the *MMDGP-nystrom(160/10)* . + *uGP(160)* is also the surrogate from [2] but uses a much larger sampling size ($m = 160$). Given the same training and testing samples, its complexity is 16 times higher than our *MMDGP-nystrom(160/10)*. + *skl* is another robust GP surrogate equipped with a symmetric KL-based kernel, as described in[13]. + *ERBF* employs an expected RBF kernel from [4] (Both *skl* and *ERBF* are designed for Gaussian inputs). + *GP* utilizes a noisy Gaussian Process model with a learnable output noise level. According to Fig 8, we observe that: + **our method, MMDGP-nystrom(160/10), can comprehensively quantify the sudden change of the input uncertainty**, evidenced by its abrupt posterior change at $x=0.6$. + However, **the *uGP (40)* with the same complexity fails to model the uncertainty correctly**. We suspect this is because *uGP* requires much larger samples to stabilize its estimation of the integral kernel and thus can perform poorly with insufficient sample size, so we also evaluate the *uGP(160)* with much higher complexity (sampling size $m=160$) and it does successfully alleviate this issue. + Apart from this, the **noisy GP model with a learnable output noise level are not aware of this uncertainty change at all**, this may be because it treats the inputs as the exact values instead of random variables. This test is only designed to examine the uncertainty quantification, but not their optimization performance. We further compare the optimization performances on the 1D synthetic functions and a real-world problem in Figure 9, as well as a 10D problem under circular distribution in Figure 7b. According to these tests, we found that: + In all experiments, **the noisy GP with learnable output noise level fails to locate the robust optimum and can get stuck at a local optimum**. + In problems with complex distributions and high dimensions, **our method outperforms the others and works quite, while the *uGP* at the same computation cost suffers the instability caused by insufficient sampling and stumbles over iterations**. This observation can be supported by the mean and standard deviation of regret in Figures 7b and 9c. Moreover, the discussion on the benefits of modeling the input uncertainty explicitly can be founded in these works: [2, 3, 4, 5] (which are already cited in our manuscript). **Reference** [1] Chi-squared distribution, Wikipedia, https://en.wikipedia.org/wiki/Chi-squared_distribution [2] Oliveira, Rafael, Lionel Ott, and Fabio Ramos. "Bayesian optimization under uncertain inputs." The 22nd international conference on artificial intelligence and statistics. PMLR, 2019 [3] Moreno, Pedro, Purdy Ho, and Nuno Vasconcelos. "A Kullback-Leibler divergence-based kernel for SVM classification in multimedia applications." *Advances in neural information processing systems* 16 (2003). [4] Dallaire, Patrick, Camille Besse, and Brahim Chaib-Draa. "An approximate inference with Gaussian process to latent functions from uncertain data." *Neurocomputing* 74.11 (2011): 1945-1955. [5] Beland, Justin J, and Prasanth B Nair. “Bayesian Optimization Under Uncertainty.” In *Advances in Neural Information Processing Systems*, 5, 2017.

Reviewer Yh2J2023-08-15

Thank you for the swift response and detailed elaboration, which has further clarified the context of the results presented in Figure 8. I acknowledge the notion that an enhancement in uncertainty quantification is demonstrated around x=0.6, and there exists a series of works supporting the effectiveness of broader explicit input uncertainty treatments in optimization. What I would kindly request the authors to have second thoughts on is that the figures in the second row of Figure 8, illustrating the acquisition function, do not directly convey the concept that the proposed input uncertainty treatment better benefits the optimization process specifically. While I don't fully dismiss the improvements shown in Figure 9, focusing solely on Figure 8 leads me to contend that its construction should be revised to more clearly and directly reflect the advancements in Bayesian optimization-oriented uncertainty quantification. I hold the belief that with a more extended period of revision, it is possible that the author can orchestrate a scenario that better conveys the underlying concept that the proposed method facilitates the optimization and effectively motivates the algorithm in a similar format. The presentation would be much stronger with the improved illustration.

Authorsrebuttal2023-08-15

Thanks a lot for the constructive and kind suggestions. We understand current experiment setup in Figure 8 can be further improved to better illustrate our motivation and will update it in the revision as the reviewer suggested.

Authorsrebuttal2023-08-15

Reminder in the last day of the Rolling Discussion

We kindly ask the reviewer if they have any outstanding questions or clarifications regarding our paper. We are happy to engage in a dialogue and conduct any additional requested work in the remaining discussion period. Thank you!

Reviewer jZyz2023-08-18

Thank you for your response

Dear authors, Thank you for your response. The response has addressed most of my concerns, so I decided to increase my score. I can't increase more because I'm not too sure regarding the setup of the optimization problem targetting in this paper. Perhaps, the authors could consider to motivate it much better.

Authorsrebuttal2023-08-19

Concrete motivating cases

We appreciate the reviewer's feedback. Here we provide two concrete examples of robust optimization, hoping they can better clarify the motivation of our paper. **[Case 1] Design & Manufacture of wireless antenna** In wireless communication systems (e.g., 4G/WiFi), the antenna is an important device for sending & receiving electromagnetic waves. It typically consists of conductive elements, such as wires, rods, or metal surfaces. These elements are precisely shaped and positioned to determine the antenna's characteristics, *e.g.*, the radiation pattern, polarization, frequency range, and etc. However, during its manufacturing or fabrication process, machine errors occur inevitably because of various factors, such as inaccuracies in the machining equipment, variations in material properties, and imprecise assembly techniques. These errors deviate the design parameters from the desired specifications and can have significant effects on antenna performance [1]. For example, suppose we want to optimize the length of the antenna $x$ to maximize its gain $y$ at a frequency range of $[1.424, 2.75)$ GHz. Let the machine error be described by a probability distribution $P(x)$, the antenna gain of the final product is $f(x^* +\delta), \delta \sim P(x^*)$, where $x^*$ is the global optimum of the target function. For some antennas, the global optimum can be quite "unstable": a small perturbation in $x$ can degrade its performance dramatically (e.g., the RKHS function in Figure 4a of the paper). In these cases, instead of finding the global optimum, we aim to find a robust optimum whose average performance is best under the machine errors: $$ x^r = \text{argmax}\_{x} \int_\delta f(x+\delta) d\_{P(x)} $$ Moreover, as the machine errors derive from multiple random sources, its probability distribution $P(x)$ can be quite complex. **[Case 2] Robust robot grasp** Yet another motivating case comes from the robot grasp [2]. In this problem, we aim to control a robot hand to grasp an object. In particular, we need to decide the optimal translations and rotations of the robot hand: $\mathbf{x}=(\Delta_x, \Delta_y, \Delta_z, \theta_x, \theta_y, \theta_z)$. However, controller errors and execution noises happen during grasp and deviate hand configuration $x^{\prime} = \mathbf{x} + \mathbf{\delta}, \mathbf{\delta} \sim P(\mathbf{x}) $, preventing the precise positioning of the robot hand in the desired pose for a stable grasp. In addition, the grasp performance is often measured via a wrench space analysis [3] and its discontinuous nature sometimes renders it quite sensitive to a small configuration change, which further complicates this problem. As such, when planning for a robotic grasp, we must consider the impact of the uncertainty in the input space (positioning errors) to ensure a robust grasp: $$ \mathbf{x}^r = \text{argmax}\_{x} \int_{\delta} f(x+\delta) d\_{P(x)} $$ ### Summary In general, both of these two cases involve the design parameters being perturbed before the evaluation by random errors and the targeting function can be quite unstable, i.e., a small change in $x$ may result in a large fluctuation in $y$. This kind of problem is quite common in the real world, say semiconductor design [4], and a more complete application list can be found in [5]. Locating the robust optimum in these tasks can be formed as an optimization problem in the expected form and robust optimization methods can be used. Compared with the other methods, our solution can model complex input distributions comprehensively and perform the optimization efficiently. **Reference** [1] Stutzman, Warren L., and Gary A. Thiele. Antenna theory and design. John Wiley & Sons, 2012. [2] Nogueira, José, et al. "Unscented Bayesian optimization for safe robot grasping." *2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)*. IEEE, 2016. [3] Weisz, Jonathan, and Peter K. Allen. "Pose error robust grasping from contact wrench space metrics." *2012 IEEE international conference on robotics and automation*. IEEE, 2012. [4] Ng, Tsan Sheng, Yang Sun, and John Fowler. "Semiconductor lot allocation using robust optimization." *European Journal of Operational Research* 205.3 (2010): 557-570. [5] Gabrel, Virginie, Cécile Murat, and et al. “Recent Advances in Robust Optimization: An Overview.” *European Journal of Operational Research* 235, no. 3 (June 2014): 471–83.

Authorsrebuttal2023-08-15

A warm reminder in last day of rolling discussion

We kindly ask the reviewers if they have any outstanding questions or clarifications regarding our paper. We are happy to engage in a dialogue and conduct any additional requested work in the remaining discussion period. Thank you!

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC