Offline black-box optimization aims to maximize a black-box function using an offline dataset of designs and their measured properties. Two main approaches have emerged: the forward approach, which learns a mapping from input to its value, thereby acting as a proxy to guide optimization, and the inverse approach, which learns a mapping from value to input for conditional generation. (a) Although proxy-free~(classifier-free) diffusion shows promise in robustly modeling the inverse mapping, it lacks explicit guidance from proxies, essential for generating high-performance samples beyond the training distribution. Therefore, we propose \textit{proxy-enhanced sampling} which utilizes the explicit guidance from a trained proxy to bolster proxy-free diffusion with enhanced sampling control. (b) Yet, the trained proxy is susceptible to out-of-distribution issues. To address this, we devise the module \textit{diffusion-based proxy refinement}, which seamlessly integrates insights from proxy-free diffusion back into the proxy for refinement. To sum up, we propose \textit{\textbf{R}obust \textbf{G}uided \textbf{D}iffusion for Offline Black-box Optimization}~(\textbf{RGD}), combining the advantages of proxy~(explicit guidance) and proxy-free diffusion~(robustness) for effective conditional generation. RGD achieves state-of-the-art results on various design-bench tasks, underscoring its efficacy. Our code is at https://github.com/GGchen1997/RGD.
Paper
Similar papers
Peer review
Summary
The paper proposed RGD, a novel method for integrating classifier guidance into classifier-free guidance diffusion models for solving offline MBO problems. Experiment results and ablation studies validate that the method outperforms state-of-the-art baselines and each proposed component is resonable.
Strengths
- Idea is intuitive and easy to follow - Motivating example in the introduction makes the reader easy to understand the limitations of the prior method and the advantages of the proposed method - Strong experiment results and detailed ablation studies make the proposed method more convincing
Weaknesses
- For the diffusion-based proxy refinement part, it seems that there are several estimations to compute the distance between $p_{\phi}(y\vert \hat{x})$ and $p_{\theta}(y\vert \hat{x})$. Furthermore, it incurs additional hyperparameter $\alpha$, which should be carefully tuned.
Questions
- For the diffusion-based proxy refinement part, have the authors tried other approaches for estimating $p(y)$? I wonder if the choice of estimation affects the performance significantly. - To compute the regularization loss term in Eq (15), we need to collect samples from the adversarial distribution. I cannot find the detailed procedure for collecting adversarial samples ($M$, $\eta$, ...). Could authors elaborate more on that part?
Rating
6
Confidence
3
Soundness
4
Presentation
3
Contribution
3
Limitations
There are a few minor comments on the manuscript. - For figure 2, it seems that $\tilde{s}(x_T, y, \omega)$ should be written as $\tilde{s}(x_T, y, \hat{\omega})$. Furthermore, at first, it makes me confusion that RGD conducts classifer-guidance. However, that misleading part has been resolved after reading the manuscript.
Thank you for your detailed feedback and I keep my positive rating. There are some minor comments. > For the diffusion-based proxy refinement part, have the authors tried other approaches for estimating p(y) As the authors say, ablation studies on the choice of estimating p(y) enhance the robustness of the proposed method. I also recommend that authors conduct the ablation study across at least two tasks for the claim.
Thanks for your prompt feedback and continued support.
Thank you for your prompt feedback and continued support. To further assess the robustness of our method against the choice of p(y), in addition to the Ant task, we have conducted experiments on the TFB8 and TFB10 tasks. The performance was 0.974 with the original method and 0.975 with GMM on the TFB8 task, and it was 0.694 with the original method and 0.692 with GMM on the TFB10 task. These consistent results reinforce the robustness of our estimator choice. We will ensure that all discussions are meticulously incorporated into the final manuscript, as suggested.
Summary
In this paper, the authors proposed to combine both classifier guidance and classifier-free guidance for offline black-box optimization. In addition, the authors propose a Proxy Refinement procedure by minimizing KL divergence between the Proxy distribution and diffusion distribution regarding $y$.
Strengths
1. The paper is well-written and well-organized. 2. The paper introduces several refinement procedures to boost the offline optimization performance. The proposed Diffusion-based Proxy Refinement procedure is interesting.
Weaknesses
1. **Technical contribution seems to be incremental** Employing diffusion models for offline black-box optimization is not new. The technical contribution of this paper seems to be incremental. The draft extends the paper "Diffusion Models for Black-Box Optimization" [1]. However, detailed discussions about the relationship between the proposed method and the paper [1] are missing. [1] Siddarth Krishnamoorthy, Satvik Mashkaria, and Aditya Grover. "Diffusion Models for Black-Box Optimization." ICML 2023. 2. **Part of the technical details are not clear.** (a) In Equation (12), the concrete computation procedure of $p_\theta (\hat{\boldsymbol{x}} | y)$ and $p_\theta (\hat{\boldsymbol{x}})$ via diffusion model is not clear. (b) The derivation of Equation (10) is not given. It seems that Equation (10) is from the forward pass of the diffusion model. However, the forward pass (Eq.32-32 in [10]) is regarding the distribution. And the concrete $\boldsymbol{x} _ t $ is constructed via the backward pass with $s_\theta(\boldsymbol{x}_k,k)$ for $k \in T,\cdots, t+1$. In addition, how to choose $\mu(t)$ and $\sigma(t)$ in Equation (10) is not clear. 3. **The additional proxy training, sample refinement procedure and proxy refinement procedure increase the computation cost** The additional proxy training, sample refinement procedure and proxy refinement procedure increase the computation cost. However, the time comparison with baselines is missing. 4. **The additional proxy training, sample refinement procedure and proxy refinement procedure bring many additional hyperparameters, which may overfit the offline BBO task** In the offline BBO tasks, the offline dataset is provided. The evaluation is the black-box function value at the generated query at one time. The long-term convergence properties and exploration/exploitation balance are not considered. As a result, there are risks that overfit the evaluation metric for the offline tasks. The paper Introduces lots of additional hyperparameters, which increases the overfitting risks.
Questions
1. Please provide more discussion about the technical details in Weakness 2. 2. What is the computation time of the proposed method? Please provide time comparisons with baselines. 3. Please provide explanations about the overfiting issue.
Rating
5
Confidence
4
Soundness
2
Presentation
3
Contribution
2
Limitations
Additional computation cost and overfitting risk may be additional limitations besides the limitations discussed.
Please reply to the rebuttal.
Dear Reviewer, Please reply to the rebuttal. AC.
Clarification on Model Robustness and Key Components
We appreciate the reviewer’s detailed feedback and are glad that most of the concerns have been addressed. We are uncertain about your use of the term 'overfitting.' We interpret this as the possibility that our proposed designs might overfit the trained proxy. Please let us know if we have misunderstood your concern. **The first key component** enhancing our model's robustness is the use of proxy-free diffusion guidance. In this framework, the denoising step cannot be interpreted as a gradient-based adversarial attack, since it does not rely on proxy gradients to directly modify the input design. This concept is thoroughly discussed in the seminar work [A], particularly in Equation (6). However, proxy-free diffusion guidance alone is insufficient, as it lacks direct guidance from the proxy, limiting its ability to extrapolate effectively. This limitation is illustrated in Figure 1 of our manuscript, titled 'Motivation for Explicit Proxy Guidance.' To address this, we introduce explicit proxy guidance as **the second key component**. This component aims to direct the sampling process toward high-property regions. Direct application of proxy gradients to the input space would result in out-of-distribution samples, potentially leading to what might be perceived as overfitting. Therefore, we apply proxy gradients to the scalar strength parameter $\omega$, which modulates both condition and diversity. This approach of optimizing scalar parameters rather than the design itself is explored in the ICML 2023 paper [B], specifically in Section 4.5 on Adaptive-$\gamma$. It demonstrates how supervision signals from the proxy can effectively update scalar hyperparameters, thereby enhancing robustness without directly modifying the input design. In essence, our model combines (1) the robustness afforded by proxy-free diffusion—which does not rely on proxy gradients—with (2) the targeted guidance from a trained proxy, which influences only the scalar strength parameter $\omega$ to mitigate overfitting risk. **These two key components** form our **proxy-enhanced sampling module**, which is crucial for enhancing the model’s resilience against overfitting and are instrumental in achieving superior outcomes. Additionally, in our **diffusion-based proxy refinement module**, we propose utilizing diffusion-derived distribution priors to refine the proxy, an approach not previously explored. This is also **an important component**. While prior work, such as COMs and ROMA, employed simple intuitive priors like conservative estimation and smoothness to refine proxy, our method leverages diffusion-derived distributions, providing a more relevant and impactful signal for refining the proxy, which has been proved to be more effective. The detailed comparisons with COMs and ROMA are discussed in detail in Appendix E, 'Further Ablation Studies'. Have we adequately addressed your concerns? We truly appreciate your comprehensive feedback. [A] Ho J, Salimans T. Classifier-free diffusion guidance[J]. arXiv preprint arXiv:2207.12598, 2022. [B] C. Chen. et al. Bidirectional Learning for Offline Model-based Biological Sequence Design. ICML 2023
Thanks for the authors' detailed response. I now have a better understanding of how each proposed component works and how they relate to one another. I am not certain about the experiments, but I think the proposed method makes sense. Therefore, I decided to increase my score.
Thanks for your support
Thank you for your supportive feedback. We are pleased that our explanations have helped clarify the aspects of our work. We will ensure that these discussions are incorporated into the revised manuscript.
Additional question about Eq.(10) in the draft
Dear authors, I find an issue when I try to derive Eq.(10) using Tweedie’s formula suggested by the author. Note that $X_0 \sim p_0(X_0)$ and $X_t = X_0 + \mathcal{N}(0, \sigma(t)^2 \boldsymbol{I} )$ from forward pass of diffusion model (corresponding VE SDE Eq.(31) in [r1]) . From the Tweedie’s formula, we achieve the following Equation: $$ \mathbb{E}[X_0|X_t=x_t] = x_t + \sigma(t)^2 \nabla _{x_t} \log p_t (x_t) $$ It can then be approximated using the trained score function $s_\theta(x_t)$ as $$ \mathbb{E}[X_0|X_t=x_t] \approx x_t + \sigma(t)^2 s_\theta(x_t) $$ However, the Eq.(10) in the submission drops the conditional expectation w.r.t. $X_0$ and directly obtain $$ x_0 \approx x_t + \sigma(t)^2 s_\theta(x_t) $$ Did I miss anything to achieve Eq.(10)? Could the authors explain more about how to derive Eq.(10)? [r1] Song et al. Score-Based Generative Modeling through Stochastic Differential Equations. 2021.
Clarification on Eq.(10).
Thank you for your insightful question regarding Eq.(10). We will now derive Eq.(10) step by step to address any concerns. We follow Eq (33) from [r1] where $p(x_t|x_0) = N(x_t; \mu(t) x_0, \sigma^2(t) I)$. Given this, we can sample $x_t$ from $x_0$ using: $x_t = \mu(t) x_0 + \epsilon \sigma(t)$. To recover $x_0$ from $x_t$, we need to know $\epsilon$, which approximates as $\epsilon \approx -\sigma(t) \cdot s_{\boldsymbol{\theta}}(x_t)$. Using this approximation, we derive $x_0 = \frac{x_t - \epsilon \sigma(t)}{\mu(t)} \approx \frac{x_t + s_{\boldsymbol{\theta}}(x_t) \cdot \sigma^2(t) }{\mu(t)}$. This approach originates from [r1], and we utilize the implementation framework detailed in another seminal work [r2]. Specifically, our code, available at https://anonymous.4open.science/r/RGD-7DBB/lib/sdes.py , implements this process as follows: - Line 24 implements $\mu(t)$ - Line 27 implements $\sigma^2(t)$ - Line 37 describes the sampling process: $x_t = \mu(t) x_0 + \epsilon \sigma(t)$ - Line 112 optimizes: $\epsilon \approx -\sigma(t) \cdot s_{\boldsymbol{\theta}}(x_t)$, where $\epsilon$ is the target, $\sigma(t)$ is the std, and $a$ is $s_{\boldsymbol{\theta}}(x_t)$. Apologies for any confusion. Considering that most readers and reviewers come from an offline MBO background, these advanced concepts of diffusion models can be challenging. We will add a statement following Line 178, "For a more detailed derivation, please refer to the Appendix." In the Appendix, we will include the discussion outlined above. Have we adequately addressed your concerns? References: - [r1] Song et al. Score-Based Generative Modeling through Stochastic Differential Equations. 2021. - [r2] Huang C W, Lim J H, Courville A C. A variational perspective on diffusion-based generative models and score matching. Advances in Neural Information Processing Systems, 2021, 34: 22863-22876.
Additionally, it's worth noting that Eq.(10) in our submission aligns closely with Eq.(15) from the seminal work DDPM[r3]. In DDPM, they present the equation $x_0 \approx \frac{x_t - \sqrt{1-\alpha_t} \epsilon_{\boldsymbol{\theta}}(x_t)}{\sqrt{\alpha_t}}$, which is derived in a discrete setting. [r3] Ho J, Jain A, Abbeel P. Denoising diffusion probabilistic models[J]. Advances in neural information processing systems, 2020, 33: 6840-6851.
Thanks for the authors' detailed response. From the authors' new response, I find that the derivation of Eq.(10) does not come from Tweedie’s formula suggested by the authors in the previous rebuttal. The Tweedie’s formula calculates the posterior expectation $X_0$ given $X_t = x_t$, i.e. $\mathbb{E} [X_0 | X_t = x_t]$, instead of a concrete sample $x_0$. The derivation of Eq.(10) is actually through reparametrization of $X_t$ and relies on an approximation of the Gaussian noise $\epsilon \approx -\sigma(t) \cdot s_\theta(x_t)$. I got the derivation. Thanks again for the authors' detailed response. My concern is well addressed.
Thank you for your continued engagement
Thank you for your continued engagement and for raising these important points regarding the derivation of Eq.(10). The reference to Tweedie's formula in our previous rebuttal was intended as a high-level citation, acknowledging the foundational idea of recovering samples from imperfect data. It was not directly used to derive Eq.(10), but rather to highlight the conceptual framework. To enhance understanding, we will include a citation of the seminar work DDPM [r3], where similar concepts are more explicitly detailed. Regarding the approximation $\epsilon \approx -\sigma(t) \cdot s_{\boldsymbol{\theta}}(x_t)$, the essence of diffusion models is to learn a model that can predict the noise vector, thereby enabling the denoising of samples from pure noise to realistic data. In our specific case, we train the diffusion model $s_{\boldsymbol{\theta}}(x_t)$ to predict $-\frac{\epsilon}{\sigma(t)}$. This is operationalized by optimizing the loss function mentioned in Line 112 of our sde.py, where $\epsilon$ is the target, $\sigma(t)$ is the std, and $a$ is $s_{\boldsymbol{\theta}}(x_t)$. In essence, starting with $x_0$, we sample a noise vector $\epsilon$ as the target, add the correponding noise to $x_0$ to generate $x_t$, and then aim to train $s_{\boldsymbol{\theta}}(x_t)$ to more closely approximate $-\frac{\epsilon}{\sigma(t)}$. ~~Have we adequately addressed your concerns?~~. Based on the latest feedback, it appears that the derivation is now clear, so we have removed this question sentence. Thank you again for your thoughtful engagement with our work.
Summary
The paper proposes a framework called Robust Guided Diffusion for the problem of Offline Black-box Optimization. The key idea is to formulate the solution as conditional generation of high-performance designs using a diffusion model which has explicit guidance from a proxy (surrogate) model. This proxy model is also refined/updated via a proxy-free diffusion procedure. Experimental analysis is shown on multiple tasks from design-bench benchmark.
Strengths
- Overall, I like the paper because it includes two simple changes to an existing approach (DDOM) that shows improved performance and the changes are validated by ablation choices.
Weaknesses
- One major premise (repeated multiple times in the paper) in the paper is that proxy guidance conditional generation is more robust than updating the design with standard gradient ascent on the proxy. However, it is not immediately clear why this should be true and the justification for this key point is somewhat limited. If true, this will be much bigger insight going beyond black-box optimization. If it is only about the exploration/exploitation balance driven by w, we could also make standard gradient have this property by optimizing a upper/lower confidence bound on the objective. Please describe why this is the case either via some empirical experiment or theoretical insight. Also, in equation 11, we might evaluate the proxy far away from the training data depending on the values of s_\theta(x_t), \sigma(t), \mu(t). - The related work coverage and corresponding experimental analysis of the paper can be improved. This problem has seen an extensive body of work recently. Please see the references below and discuss/compare them appropriately. Some of them are included in references but not compared in the experiments ([1], [2], [3]): - [1] Yuan, Ye, et al. "Importance-aware co-teaching for offline model-based optimization." Advances in Neural Information Processing Systems 36 (2023). - [2] Kim, Minsu, et al. "Bootstrapped training of score-conditioned generator for offline design of biological sequences." Advances in Neural Information Processing Systems 36 (2023). - [3] Nguyen, Tung, Sudhanshu Agrawal, and Aditya Grover. "ExPT: Synthetic pretraining for few-shot experimental design." Advances in Neural Information Processing Systems 36 (2023). - [4] Chemingui, Yassine, et al. "Offline model-based optimization via policy-guided gradient search." *Proceedings of the AAAI Conference on Artificial Intelligence*. Vol. 38. No. 10. 2024. - [5] Yao, Michael S., et al. "Generative Adversarial Bayesian Optimization for Surrogate Objectives." arXiv preprint arXiv:2402.06532 (2024).
Questions
Some of the tasks in design-bench benchmark has errors which makes them not so informative for evaluation. For example, the offline dataset in superconductor task has multiple copies of the same inputs but with different outputs. As a result, the random forest oracle which is fit on this offline data is not reliable. It is mentioned that "This issue has now been rectified by the development team." How is it fixed?
Rating
6
Confidence
4
Soundness
2
Presentation
2
Contribution
2
Limitations
Please see weaknesses section.
Please reply to the rebuttal.
Dear Reviewer, Please reply to the rebuttal. AC.
Thank you for your detailed feedback
Thank you for your detailed feedback, which provides constructive insights for refining our paper. We will incorporate these points into the revised version. - In the paper [A], searching the term 'adversarial' reveals critical arguments supporting the robustness of proxy-free diffusion guidance over proxy diffusion guidance. The text notes, > Furthermore, because classifier guidance mixes a score estimate with a classifier gradient during sampling, classifier-guided diffusion sampling can be interpreted as attempting to confuse an image classifier with a gradient-based adversarial attack. as mentioned in the second paragraph of the introduction. Additionally, descriptions related to Equation (6) state, > Eq. (6) has no classifier gradient present, so taking a step in the $\epsilon$ direction cannot be interpreted as a gradient-based adversarial attack on an image classifier. These points suggest that proxy-free diffusion guidance surpasses proxy diffusion guidance in robustness. Acknowledging that generating images for a specific class is simpler than guiding towards the optima of a function, the latter scenario necessitates more robust guidance mechanisms. This is where proxy-free guidance excels, as directly using proxy diffusion guidance may lead to out-of-distribution issues, whereas proxy-free diffusion guidance provides more robust guidance. We will incorporate these discussions in Line 121 of our manuscript where we introduce guided diffusion. - Thank you for your comment. Yes, in our ablation study, we kept all elements except the proxy refinement part constant across the three methods. We will emphasize this in Line 482 of our manuscript where we perform further ablation studies. Have we adequately addressed your concerns?
Response
Thanks for the response. I am not fully convinced with the robustness of proxy-free diffusion guidance argument but I don't want to nitpick now and would be happy to increase the score towards acceptance. Please add these points in the main paper along with proper discussion of the related work.
Thanks for your prompt response and support.
Thank you for your willingness to consider our arguments and for your constructive suggestions. We will ensure to add these points and enhance the discussion of related work in the main paper.
Summary
The paper introduces a robust guided diffusion framework for offline black-box optimization, combining proxy and proxy-free diffusion for conditional generation. Key improvements include proxy-enhanced sampling and diffusion-based proxy refinement to address out-of-distribution issues. Experiments on the Design-Bench benchmark show the method outperforms existing techniques, validated by ablation studies.
Strengths
- The regularization of the proxy using the diffusion model is interesting. Additionally, optimizing the alpha parameter in an offline manner aligns well with the offline setup, enhancing the method's consistency and applicability. - Experiments and ablations on four continuous and three discrete tasks validate the effectiveness of the proposed RGD method, showing improved performance and robustness.
Weaknesses
- The paper lacks comparison with relevant approaches like ICT [1] and TRI-mentoring [2]. Despite referencing the latter in the related work section, it’s overlooked in the results. - It is unclear why the results without proxy-enhanced sampling still achieve competitive outcomes, surpassing the dataset y_max. This contradicts the claims in lines 40-46. Where does the out-of-distribution (OOD) problem arise then? What is the distribution of the generated 128 candidates with and without the sampling? - The BDI reported results are significantly lower than in the original paper, especially for the ANT and TFBIND8 tasks. This also seems to be the case for BONET results. Did the authors change the evaluation setup? [1]: Importance-aware Co-teaching for Offline Model-based Optimization, https://arxiv.org/abs/2309.11600 [2]: Parallel-mentoring for Offline Model-based Optimization, https://arxiv.org/abs/2309.11592
Questions
- How are the initial 128 designs selected? Do you generate N designs and then use the proxy to select 128? - Are the diffusion parameters like T the same for DDOM? - Can you show, for a task, how the values (proxy/oracle) of the generated designs progress throughout the diffusion steps? - How were the discrete tasks handled? Were logits used, or were they kept discrete? - Previous methods typically evaluate on the Hopper task; why was it removed and Rosenbrock is added instead?
Rating
3
Confidence
4
Soundness
2
Presentation
3
Contribution
2
Limitations
The authors address the limitations and potential negative impacts in their paper.
reply to the rebuttal
Dear Reviewer, Please reply to the rebuttal. AC.
Thank you for your detailed feedback. Regarding the use of BDI results from the TRI-mentoring paper, we have now included proper citations and incorporated TRI-mentoring into our benchmarks for a comprehensive comparison. We initially viewed TRI-mentoring primarily as an advanced ensemble method, which seemed distinct from our model’s approach, and thus only include it in the related work section. However, recognizing the importance of clarity and completeness, we have addressed this in our revised submission. As for the reduced network architecture, we followed the specifications outlined in the published NeurIPS2023 paper TRI-mentoring and use the reported results in this NeurIPS2023 paper. Our approach was consistent with the standard three-layer MLP used across all methods in this comparison, ensuring a fair and uniform basis for evaluation. We hope these clarifications address your concerns. Could you please let us know if there are any further issues or suggestions you have?
Appeal for Reassessment Based on Review Feedback
Given your recognition that we have addressed most of the initial concerns: > While you've addressed most of my concerns, we were surprised by the decision to downgrade our submission from 'borderline reject' to 'reject,' particularly since the remaining issues were, in our assessment, relatively minor. We are confident that the revisions we have implemented thoroughly address the issues raised. In light of these efforts, we kindly request a reevaluation of our paper.
Thank you for your follow-up and revisions, and for admitting the oversight. However, transparency is most effective when it’s proactive, not reactive. The initial omission of TRI-mentoring while using their BDI results raise serious concerns about the credibility of your work. Even if corrected later, selective reporting undermines trust and cannot be overlooked. Therefore, I will maintain my decision to reject the paper. I strongly urge you to prioritize transparency in future submissions.
Thank you for your prompt feedback. As previously mentioned, we acknowledge the TRI-mentoring method and have discussed it in the related work section. We initially did not include a direct comparison with TRI-mentoring because (1) it employs an advanced ensemble approach, distinctly different from our methodology with a single proxy, and (2) our analysis already included a comprehensive set of 14 baselines. Following your insightful recommendation, we have now incorporated a comparative analysis with TRI-mentoring. We also recognize and regret the inadvertent omission of the citation for the BDI results from TRI-mentoring, which has been duly corrected. We wish to clarify that **our omission was limited to the citation of results; we did not manipulate the baseline results, and thus this should not be considered as selective reporting.** We appreciate the emphasis on transparency and fully agree with its importance. We understand that **peer review should focus on the scientific content and contributions of a manuscript**. While unintentional oversights are unfortunate, they have been rectified and should not overshadow the substantive scientific evaluations of the work. We hope that the changes implemented demonstrate our commitment to transparency and scientific rigor.
Summary
The paper introduces a new method, named RGD, for Offline Black-box Optimization (BBO). RGD incorporates an improved proxy to guide the previous proxy-free method (i.e. DDOM[4]). Key technical innovations includes (1) improving the robustness of the proxy function against adversarial samples by consistency regularization with the diffusion process; (2) dynamic per-sample reweighting between proxy-guided and proxy-free sampling. Compared to previous approaches, RGD demonstrates superior performance on Design-Bench [3].
Strengths
Methodology: RGD integrates forward and reverse approaches for BBO, in a way that they can help with each other (e.g. using forward proxy to guide the reverse sampling and using the diffusion process to improve the forward proxy), which is technically sound and interesting. Experiment: RGD demonstrates superior performance on Design-Bench, compared to the baselines. Ablation: Ablations on different components of RGD are provided.
Weaknesses
The reviewer would prefer some clarifications on the method and the experiments i) Algorithm 1, Line 4, how to identify the adversarial examples? From Line 187-188, it looks like gradient ascent is utilized to find the x that maximize y, it is unclear to the reviewer that how to determine if the obtained x is an adversarial example ii) Algorithm 1, Line 7, refine the proxy function via eq 15. It would be best if the author could provide further details on how to optimize eq (15), e.g. number of validation and adversarial samples, number of iterations for the bi-level optimization discussed in Appendix B. iii) Algorithm 1 Line 13, optimizing \omega. Again, it would be best if the author could provide extra info on how to optimize \omega. From Algorithm 1, it looks like \omega is time dependent and optimized for each time step. How many training iterations are required for each time step. The reviewer also wonder if the obtained \omega are dramatically different between different time steps. iv) From Line 257-258, it looks like the baselines shown in Table 1 & 2 were re-implemented. If this is the case, the authors are encouraged to include more implementation details, e.g. the model architecture for the score function, etc. This could help follow-up works to reproduce the reported results. The reviewer also wonders if the source code will be made public.
Questions
Please refer to the weaknesses.
Rating
5
Confidence
4
Soundness
3
Presentation
3
Contribution
2
Limitations
Limitations have been discussed in the appendix
reply to the rebuttal
Dear Reviewer, Please reply to the rebuttal. AC.
Thank you for your detailed responses. While you've addressed most of my concerns, two key issues are troubling: (1) The use of BDI results from the TRI-mentoring paper without proper citation, coupled with the initial omission of TRI-mentoring from the benchmarks, and (2) The results from a reduced network architecture, which leads to an unfair comparison that may inadvertently favor your method. These issues affect the credibility of the work, prompting me to revise my score to a rejection. I hope these concerns can be addressed in any future submissions to ensure a fair and accurate presentation.
Response to rebuttal
Thanks for taking the time to respond to my questions. Please see points related to your response below: - Regarding `This significant insight aligns with findings from [A], which argues that proxy-free diffusion guidance surpasses proxy diffusion guidance in robustness. [A] Ho, J. and Salimans, T. Classifier-free diffusion guidance.` Unless I am missing something, the main premise in [A] is to "attain a trade-off between sample quality and diversity **similar** to that obtained using classifier guidance". I am not sure how this reference paper [A] findings conveys that ` proxy-free diffusion guidance surpasses proxy diffusion guidance in robustness.` Moreover, guiding diffusion models to generate images for a specific class is a relatively easier/different problem that guiding towards the optima of a function. In the former, we are interested in sampling any point from a class conditional distribution whereas in the latter, we explicitly want to find the optima (rare sample in the distribution). - Thanks for the ablation comparing diffusion-based proxy refinement module with COMs/ROMa strategy. Since the ablation show final evaluation performance on the tasks, this ablation is useful only if the proxy refinement part of the three methods is changed and everything else is kept the same. For example, this requires fixing either gradient ascent or proxy enhanced sampling for search/generating candidates for evaluation. Is this the case? - Thanks for including the discussion about new related work and fixing the error in superconductor task.
Reply to Authors' rebuttal
Thanks for the authors' detailed clarification and responses. Most of my concerns have been addressed. I still have some concerns regarding the overfitting risk. I acknowledge the different focuses of offline black-box optimization and online black-box optimization and explain why the authors preclude the exploration-exploitation considerations for long-term behavior. However, I am not sure what the key component among the several proposed ones is that makes the whole model more robust against the overfitting. In addition, what is the key component to achieve a better score beyond the maximum in the dataset and outperforms baselines?
Dear Reviewer, As the discussion period is nearing its conclusion, we kindly ask you to engage in the discussion and provide notes on any concerns that have not yet been addressed, along with the reasons why. Thank you for your attention to this matter. AC.
Urgent: please respond to rebuttal
Dear Reviewer, As the discussion period is nearing its conclusion, we kindly ask you to engage in the discussion and provide notes on any concerns that have not yet been addressed, along with the reasons why. AC.
Decision
Reject