Thank you for your response. We sincerely appreciate your valuable feedback. We would like to provide further clarification and address these concerns.
Q1: The settings of attacker
**Moving one item**
From the perspective of the practical setting of online BPP, the packing policy can observe multiple items, but it can only pack the first item into the container. Let's consider a scenario where the attacker permutes the entire sequence of observable items at timestep $t$. In this case, regardless of the permutation applied to the remaining items at timestep $t$, at timestep $t+1$, the remaining items will combine with a new incoming item to form a new sequence that will be further permuted. Therefore, any permutation of the remaining items at timestep $t$ will be ignored and subsequently permuted with the new incoming items.
From the perspective of the influence on the packing policy, let's consider a simplified value function $V(C_{t}, B_{t}) = r_t + V(C_{t+1}, B_{t+1})$, where $C_{t}$ and $B_{t}$ represent the bin configuration and the permuted observable item sequence, and $r_t$ is the reward. It tells us that the value of the packing policy at timestep $t$ depends on both the reward $r_t$ and the value function $V(C_{t+1}, B_{t+1})$. And $r_t$ and $C_{t+1}$ are influenced by the first item in $B_{t}$. The remaining items in $B_{t}$ will combine with a new item to form a new sequence, which will undergo further permutations as $B_{t+1}$. Consequently, the permutation of the remaining items at timestep $t$ will be disregarded in the attack process.
From the perspective of the implementation, if we allow the attacker to permute the entire sequence of observable items, the action space will grow exponentially as the number of observable items increases. This large search space may have a detrimental effect on the convergence of the RL-based attacker.
**Item distribution**
In our paper, we specifically focus on the impact of item permutation on the packing policy, rather than the item distribution. To ensure that our study remains focused on the randomness introduced by item permutation, we design the permutation-based attacker, which allows us to avoid any influence stemming from changes in the item distribution. Furthermore, as mentioned in PCT, larger items can simplify the scenario, while smaller items can trickily make the task more challenging. Therefore, we aim to prevent the attacker from simply learning to selectively favor certain types of items to create challenging instances. Instead, we want the attacker to genuinely learn how to permute the item sequence in order to identify and create challenging scenarios. Hence, we have designed an attacker that is unable to alter the item distribution in order to maintain the integrity of our study.
Q2: The evaluation
**The performance of $\beta=0$**
We would like to emphasize that there are inherently challenging instances present in the nominal dynamics ($\beta=0$). Therefore, one approach to improving performance when $\beta=0$ is to enhance the performance specifically on those challenging instances while maintaining performance on others. When using a smaller value of $N_B$, the distribution of training data (mixture instances) and testing data ($\beta=0$) exhibits less deviation. This enables the policy to effectively handle both challenging instances and moderate instances, thereby improving generalization. However, when $N_B=20$, the substantial deviation between the distributions can negatively impact performance when testing on $\beta=0$. And we have confidence in the strength of the baseline methods chosen due to the fair experimental comparison settings.
**Standard Deviation**
We would like to clarify that in a high uncertainty environment, such as online 3D BPP, robustness specifically refers to the ability of a policy to consistently perform well despite the presence of uncertainties. It is intuitive that the RARL policy will consistently exhibit a conservative behavior, regardless of the problem instances provided. This conservative behavior ensures a consistent performance and reduces the variance of the policy. On the other hand, the AR2L(1) policy demonstrates a less conservative behavior, resulting in relatively larger variance.
Q3: The hyperparameter
We have the flexibility to set $\alpha > 1$. However, based on observations from our empirical study, we found that larger $\alpha$ can actually lead to a degradation in performance in the nominal dynamics. This suggests that AR2L with larger $\alpha$ tends to overprioritize worst-case scenarios, similar to previous robust methods. Given that our objective is to achieve a desired balance between performance in the nominal and worst-case dynamics, it is not recommended to set $\alpha > 1$. Doing so may result in an excessive focus on worst-case performance, compromising the overall performance in the nominal dynamics.