Pretrained Optimization Model for Zero-Shot Black Box Optimization

Reply to new discussion (1/3)

First and foremost, I would like to express my sincere gratitude for your comments and suggestions regarding our manuscript submitted to NeurIPS 2024. We have diligently revised our paper in accordance with your feedback. We earnestly hope that you will reconsider our revised manuscript and appreciate the improvements made. If you find that our efforts and the results now meet the standards of the conference, we kindly request that you consider increasing the score of our paper. We highly value your assessment and look forward to your final approval. Thank you once again for your valuable time and expertise. ## Q1: I still believe the writing can be significantly improved for... Thank you for your valuable suggestions, which have helped us improve the expression of our paper. To better introduce the background knowledge and clarify the motivation behind the design of POM, we have followed your advice and added the following content in Section 3 before detailing the mechanism of POM (see Section 3.2 for details). " ***3.2 Classic Population Optimization Algorithm*** *In this section, we use Differential Evolution (DE) as an example to review classic evolutionary algorithms. DE [20,43] is a prominent family within evolutionary algorithms (EAs), known for its advantageous properties such as rapid convergence and robust performance [44,45]. The optimization strategy of DE primarily involves mutation and crossover operations.* *The classic DE/rand/1 crossover operator is illustrated in Eq. (1) (additional examples are listed in Appendix A.2). Each mutation strategy can be viewed as a specific instance of Eq. (2); Further details are provided in Appendix A.2. Additionally, we represent the mutation strategy in a matrix form, as shown in Eq. (3). The matrix $\mathbf{S}$ evolves with the generation index *t*, indicating that the mutation strategy adapts across different generations. Consequently, we propose a module to enhance the performance of the mutation operation, which leverages the information from the population of the *t*th generation to generate $\mathbf{S}^t$. This serves as the motivation for our design of the LMM.* $\mathbf{v}\_i^t = \mathbf{x}\_{r1}^t+F\cdot(\mathbf{x}\_{r2}^t-\mathbf{x}\_{r3}^t)$     (1) *In the crossover phase at step *t*, DE uses a fixed crossover probability $cr_i^t$ ∈ [0,1] for each individual $\mathbf{x}_i^t$ in the population, as shown in Eq. (9). The crossover strategy for the entire population can then be expressed as a vector $\mathbf{cr}^t = (cr_1^t, cr_2^t, ..., cr_N^t)$. Our goal is to design a module that adaptively generates $\mathbf{cr}^t$ using the information from the population. This approach allows for the automatic design of the crossover strategy by controlling the parameter *cr*. This serves as the motivation for our design of LCM.*

Reply to new discussion (2/3)

## Q2: Moreover, the methodology section is a bit too verbal and mathy, and some equations are unnecessary. I do not see a difference between Eq (6) and (10) and the standard feed forward and attention layers. In this section, we present detailed formulas to ensure the reproducibility of POM, which are also crucial for elucidating the mechanism of POM. We will then discuss in detail the differences between these formulas and the standard self-attention and feed-forward layers. ### The difference between Eq (6) and standard self-attention **A. Self-Attention** 1. **Input Transformation**: $$ \mathbf{Q}= \mathbf{X}\mathbf{W}^Q, \quad \mathbf{K} = \mathbf{X}\mathbf{W}^K, \quad \mathbf{V} = \mathbf{X}\mathbf{W}^V$$ 2. **Attention**: $$\text{Self-Attention}(\mathbf{Q}, \mathbf{K}, \mathbf{V}) = \text{softmax}\left(\frac{\mathbf{Q}\mathbf{K}^T}{\sqrt{d_k}}\right)\mathbf{V}$$ 3. **Output**: $$\text{Output} = \text{Self-Attention}(\mathbf{Q},\mathbf{K},\mathbf{V})$$ **B. Eq. (6)** 1. **Input Transformation**: $$ \mathbf{\hat{H}}^t=\text{Tanh}(\mathbf{H}^t\mathbf{W}_{m1}+\mathbf{b}\_{m1}),\quad \mathbf{Q}^t=Tanh(\mathbf{\hat{H}}^t\mathbf{W}\_{m2}+\mathbf{b}\_{m2}), \quad \mathbf{K}^t=Tanh(\mathbf{\hat{H}}^t\mathbf{W}\_{m3}+\mathbf{b}\_{m3}) $$ 2. **Attention**: $$\text{LMM-Attention}(\mathbf{Q}, \mathbf{K}, \mathbf{X})= \text{mask}\left(\text{softmax}\left( \text{Tanh}\left(\frac{\mathbf{Q}\mathbf{K}^T}{\sqrt{d_k}}\right)\right),r_{\text{mask}}\right)\mathbf{X}$$ 3. **Output**: $$\text{Output} = \text{LMM-Attention}(\mathbf{Q}, \mathbf{K}, \mathbf{X})$$ The differences between Eq (6) and standard self-attention are evident as follows: 1. **No Calculation of V**: In Eq (6), we do not compute $\mathbf{V}$; instead, we use $\mathbf{X}$ in the attention calculation. This approach is crucial for achieving good performance in our optimization task, whereas standard self-attention struggles to adapt to our problem scenario. 2. **Different Processing of Input Information**: We introduce the Tanh activation function and a bias term, which helps enhance the performance of POM. This is a significant departure from the standard self-attention mechanism. 3. **Attention Matrix Mapping and Masking**: For the computed attention matrix, we apply the Tanh function to map it to the range $[-1,1]$. Additionally, we use a random mask operation, setting elements to zero with a probability of $r_{mask}$. In summary, there are several key differences between our approach and standard self-attention. These differences are crucial for the performance of POM, and it is essential to detail them thoroughly. This is not a shortcoming but rather a necessary distinction. ### 2 Eq (10) and feed forward layer **A. feed forward** $$\text{Output} = \mathbf{W}_2 \cdot \text{ReLU}(\mathbf{W}_1 \cdot \mathbf{X} + \mathbf{b}_1) + \mathbf{b}_2$$ **B. Eq (10)** $$\mathbf{cr}^t = \text{sigmoid}\left(\text{layernorm}\left(\tanh\left(\mathbf{Z}^t \times \mathbf{W_{c1}} + \mathbf{b_{c1}}\right) | \mathbf{\tau}\right) \times \mathbf{W_{c2}} + \mathbf{b_{c2}}\right)$$ It is worth noting that, although the two are very similar in form, Eq (10) provides a detailed implementation scheme. Any modifications to Eq (10), such as replacing the sigmoid function with ReLU, result in a significant decline in POM's performance. This is because we rely on the sigmoid function to map the output to the range $[0,1]$, representing probability values. Changes to other activation functions or hyperparameters in Eq (10) also lead to performance degradation. ## Q3: Regarding the baselines, TNP, ExPT, and LICO can all be used for zero-shot optimization because they do not require further fine-tuning after pretraining. However, they do require the new objective to lie in the same dimension/domain as the pretraining tasks. It is clear that TNP, ExPT, and LICO require retraining in different dimensional scenarios when faced with tasks of varying dimensions. This does not align with the definition of Zero-shot Optimization, which is "an optimizer that can be directly applied to solve a continuous black-box optimization problem $f$ without any tuning." ***"Definition 1 Zero-shot Optimization** : Zero-shot optimization refers to an optimizer that is applied directly to solve a continuous black-box optimization problem $f$ without any tuning. This means that the optimizer does not require any contextual information about $f$ and can be directly used to handle problems of any dimensionality."*

Reply to new discussion (3/3)

## Q4: Can the authors explain why POM does not have the conflicting issue I mentioned in the original review... Conflicts between optimization objectives do not necessarily lead to conflicts in optimization strategies. POM focuses on learning the mapping from the ranking information of individuals within a population to the search strategies, rather than adopting a fixed optimization strategy to suit all tasks. Specifically, the LMM module adaptively generates the strategy matrix $\mathbf{S}^t$ using the population's ranking information ($\hat{r}^t_i$) and the information representing their relative advantages $\hat{f}_i^t$ (normalized from $f(\mathbf{x}_i^t)$, representing individual $i$. Note that we can convert all problems to minimization problems by adding a negative sign). The LCM module uses the population's ranking information ($\hat{r}^t_i$), the relative advantage information of individuals within the population $\hat{f}_i^t$, and the similarity information between parent and offspring $sim_i^t$ to adaptively generate the crossover strategy. This means that POM is not fitting a specific problem $f$, nor is it searching for an optimal $\mathbf{x}^* \in \mathbb{R}^d$ in its solution space. Instead, it searches for optimal POM strategy parameters $\mathbf{\theta}^*$ in the strategy space through these training tasks. Therefore, even if there are conflicting objective functions, it does not affect the performance of POM. Furthermore, the training algorithm proposed by POM, MetaGBT, integrates gradient information from a set of tasks, making the training process more stable and easier. Let's simulate the situation you mentioned. Suppose there is a function $f(\mathbf{x})=\sum_i{x_i}$, with task one being $f_1=\mathop{\arg\max}\limits_{\mathbf{x}}f(\mathbf{x})$ and task two being $f_2=\mathop{\arg\min}\limits_{\mathbf{x}}f(\mathbf{x})$. Clearly, these two tasks are in direct conflict. Assuming that at the current moment, the populations for both tasks are the same, they are in the following state: $$ \mathbf{X}= \begin{bmatrix} 1 & 2 & 3 \\\\ 4 & 5 & 6 \\\\ 7 & 8 & 9 \\\\ 1 & 3 & 4 \\\\ \end{bmatrix} \mathbf{X}_1= \begin{bmatrix} 7 & 8 & 9 \\\\ 4 & 5 & 6 \\\\ 1 & 3 & 4 \\\\ 1 & 2 & 3 \\\\ \end{bmatrix} \mathbf{X}_2= \begin{bmatrix} 1 & 2 & 3 \\\\ 1 & 3 & 4 \\\\ 4 & 5 & 6 \\\\ 7 & 8 & 9 \\\\ \end{bmatrix} $$ $$ \mathbf{U}_1=\mathbf{CR}\cdot\mathbf{V}_1+(1-\mathbf{CR})\cdot\mathbf{X}_1= \left[ \begin{matrix} 6.96 & 7.96 & 8.96 \\\\ 4.54 & 5.55 & 6.55 \\\\ 3.25 & 4.79 & 5.79 \\\\ 4.51 & 5.65 & 6.64 \\\\ \end{matrix} \right] $$ $$ \mathbf{U}_2=\mathbf{CR}\cdot\mathbf{V}_2+(1-\mathbf{CR})\cdot\mathbf{X}_2= \left[ \begin{matrix} 1.02 & 2.03 & 3.03 \\\\ 1.14 & 2.89 & 3.89 \\\\ 2.68 & 3.83 & 4.83 \\\\ 2.27 & 3.54 & 4.54 \\\\ \end{matrix} \right] $$ Finally, after performing selection between $\mathbf{X}$ and $\mathbf{U}$, the resulting population is: $$ \mathbf{X}_1'= \left[ \begin{matrix} 7.00 & 8.00 & 9.00 \\\\ 4.54 & 5.55 & 6.55 \\\\ 3.25 & 4.79 & 5.79 \\\\ 4.51 & 5.65 & 6.64 \\\\ \end{matrix} \right] \mathbf{X}_2'= \left[ \begin{matrix} 1.00 & 2.00 & 3.00 \\\\ 1.14 & 2.89 & 3.89 \\\\ 2.68 & 3.83 & 4.83 \\\\ 2.27 & 3.54 & 4.54 \\\\ \end{matrix} \right] $$ **The average fitness for Task 1 improved from 13.25 to 17.8175, demonstrating an enhancement in the performance of Task 1. Conversely, the average fitness for the population of Task 2 decreased from 13.25 to 8.9025, which also indicates an improvement in the performance of Task 2.** **This shows that even with conflicting objective functions, Their optimization strategies do not necessarily conflict. This demonstrates that POM learns a universal optimization strategy, and there is no conflict as you claimed.**