>**Q:** Thank the authors for the rebuttal. R1 addresses my concern about the scalability of the proposed method since the results on Tiny-ImageNet-200 are provided. However, I can't entirely agree with R2. Wasserstein DRO also suffers from overly pessimistic issues as indicated by [1]. And f-divergence DRO is an important baseline for evaluation, but the author did not seem to provide the results of f-divergence DRO for comparison.
**A:** Thank you for your constructive comment! We are glad to hear that your concern regarding scalability has been sufficiently addressed. With respect to f-divergence, we regret the oversight in our manuscript. Notably, our competitors, **DROLT**([a]) and **ADVShift**([b]), employ DRO based on KL-divergence to enhance model robustness, and our methods consistently outperform theirs across all experiments.
To our knowledge, [c] reveals a phenomenon quite different to that described in [d]. The former suggests that in Wasserstein DRO, the training process may collapse, leading the model to a trivial solution, even when the perturbation radius $\epsilon$ is relatively small compared to the distance between training distribution and true distribution.
To check whether this anomaly manifests in our setting, we conduct a simple two-step experiment on CIFAR datasets. In the first step, we estimate an upper-bound of Wasserstein distance between training and testing distributions. In the second step, we set $\epsilon$ to be larger than the estimated distance, and see if similar issue occurs in our context.
**Step 1:** Assume that both training and test sets of the original CIFAR are sampled from the same distribution, and our corrupted CIFAR datasets are sampled from the "real distribution". Directly solving the Wasserstein DRO between these distributions remains intractable. However, given that the corrupted datasets are generated from the original testing sets, a good estimate of the Wasserstein distance can be obtained by calculating the mean example-wise distance between corrupted and clean datasets, resulting in:
**The estimated Wasserstein distance between CIFAR and different levels of CIFAR-C:**
|Datasets|Level1|Level2|Level3|Level4|Level5|
|:-:|:-:|:-:|:-:|:-:|:-:|
|CIFAR10|2.80|4.07|4.67|5.58|6.63|
|CIFAR100|2.83|4.11|4.67|5.57|6.61|
**Step 2:** Train our model with $\epsilon \in \\{1,2,3,4,5,6,7,8\\}$, which is much larger compared to our experimental parameters. In the 16 groups of experiments we run on CIFAR10 and CIFAR100, only $\epsilon=7$ in CIFAR100 fails to converge. However, by choosing smaller learning rate, our model can evade collapsing into a trivial solution.
This experiment proves that the training collapse problem is not that fatal in our setting, even when $\epsilon$ is substantially larger than the distance to the true data distribution, and can be avoided by simply tuning the learning rate.
However, shifting the discussion about overly-pessimistic to a broader perspective, beyond a specific setting, we must probe if our methods encounter performance degradation under large, unconstrained uncertainty set. This becomes similar to what we discussed in Prop. 2, i.e., if we place no constraint on the distributional attacker, it will tend to attack the tail class, potentially causing label noise. This might typify the over-pessimistic problem inherent in our setting. Nonetheless, we have proposed our solution by adding structural constraint, namely DRAUC-Da, in our paper.
[a]Samuel, Dvir, and Gal Chechik. "Distributional robustness loss for long-tail learning." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.
[b]Zhang, Jingzhao, et al. "Coping with label shift via distributionally robust optimisation." arXiv preprint arXiv:2010.12230 (2020).
[c]Frogner, Charlie, et al. "Incorporating unlabeled data into distributionally robust learning." arXiv preprint arXiv:1912.07729 (2019).
[d]Hu, Weihua, et al. "Does distributionally robust supervised learning give robust classifiers?." International Conference on Machine Learning. PMLR, 2018.