Distributionally robust optimization (DRO) is an effective approach for data-driven decision-making in the presence of uncertainty. Geometric uncertainty due to sampling or localized perturbations of data points is captured by Wasserstein DRO (WDRO), which seeks to learn a model that performs uniformly well over a Wasserstein ball centered around the observed data distribution. However, WDRO fails to account for non-geometric perturbations such as adversarial outliers, which can greatly distort the Wasserstein distance measurement and impede the learned model. We address this gap by proposing a novel outlier-robust WDRO framework for decision-making under both geometric (Wasserstein) perturbations and non-geometric (total variation (TV)) contamination that allows an $\varepsilon$-fraction of data to be arbitrarily corrupted. We design an uncertainty set using a certain robust Wasserstein ball that accounts for both perturbation types and derive minimax optimal excess risk bounds for this procedure that explicitly capture the Wasserstein and TV risks. We prove a strong duality result that enables tractable convex reformulations and efficient computation of our outlier-robust WDRO problem. When the loss function depends only on low-dimensional features of the data, we eliminate certain dimension dependencies from the risk bounds that are unavoidable in the general setting. Finally, we present experiments validating our theory on standard regression and classification tasks.
Paper
Similar papers
Peer review
Summary
This paper empowers WDRO with the ability to resist outliers, building upon the outlier-robust Wasserstein distance $W_p^\epsilon$. The excess risk of the solution to both the outlier-robust WDRO and its empirical version is given. An improved bound of the excess risk is derived for the setting of low-dimensional features. The optimization algorithm for the outlier-robust WDRO is developed with the dual form of the minimax problem. Empirical study validates the effectiveness of the proposed method with a simple regression setting involving both Wasserstein and Total Variance contaminiation.
Strengths
1. DRO is known to be over-pessimistic in the existence of outliers. Empowering Distributionally Robust Optimziation with robustness against noisy labels is important. The outlier-robust Wasserstein distance is a well-developed revision of the original Wasserstein distance to account for certain degree of contamination. Thus, constructing the uncertainty set of DRO with $W_p^\epsilon$ is both significant and reasonable. 2. The theoretical analysis of the excess risk of the proposed method is sound and comprehensive. The results quantify how the solution adapts to given geometric and TV contamination. The effectiveness of the empircial algorithm is also guaranteed by the empirical bound of excess risk.
Weaknesses
1. The effectiveness of outlier-robust WDRO is theoretically guaranteed by the excess risk bound. My main concern is over the **advantage** of outlier-robust WDRO v.s. WDRO. As is stated by the authors, the excess risk of outlier-robust WDRO is upper bounded by the $W_p$ regularizer with a larger Wasserstein radius. The authors also recognize that the same upper bound could be achieved by a radius-expanded WDRO. Two advantages of WDRO given in Remark 1 are not convincing. - The authors claim that a heavy preprocessing step is required by WDRO to estimate the exact radius. However, in practical settings beyond simulated experiments, the exact parameters of contamination levels $\rho, \epsilon$ are also latent. Either an estimate or tuning of the radius is necessary for both WDRO and the outlier-robust version. - The authors prove an improved bound of the excess risk for outlier-robust WDRO in the setting of low-dimensional features and claim that the bound of WDRO could not be improved, which is unsupported. A tight bound of the excess risk of WDRO with low-dimensional features might be given to consolidate the author's claim. Otherwise, an inequality between the excess risk of WDRO and its outlier-robust version might be given similarly to Eq.4. Furthermore, the proposed low-dimensional feature setting is somewhat impractical because the exact dimension $k$ is typically unknown in real datasets. In the experiment section, I suppose the radius for WDRO is selected to be the true contaminaton radius. The curve of performances of WDRO and outlier-robust version with increasing radius might demonstrate their gap more convincingly. 2. I am concerned if the formulation of outlier-robust WDRO (eq.3) is well defined. Consider a simple case where all the samples of $\tilde \mu$ are discretely distributed on $2/\epsilon$ points, one of which is denoted by $(x_0,y_0)$. By arbitarily modifying $y_0$ to $y'$ we get a new distribution $\nu(y')$. According to Line 131 we have $W_p^\epsilon(\tilde \mu, \nu) =0$. Thus, all the $\nu(y')$ is included in the uncertainty set of the correspondong $W_p^\epsilon$ DRO. However, the risk of a given predictor on the group of $\nu(y')$ could be unbounded since $y'$ is unconstrained. As a consequence, there might be no solution to eq.3. Intuitively, though eq.3 incorporates the clean distribution into the uncertainty set, it might also include more dirty distributions since the outlier-robust Wasserstein distance tolerates a small fraction of outliers, but the risk on these outliers could be unbounded. Therefore, I'm concerned if the porposed outlier-robust DRO would be biased towards more dirty distributions instead of recovering the clean one. 3. I am also concerned with the selection of the radius $\rho$ of outlier-robust WDRO. As is indicated by Theorem 1, larger $\rho$ leads to higher excess risk while Line 203 states that the excess risk bound only stands for $\rho \geq \rho_0 + W_p(\mu, \hat \mu_n)$, implying that $\rho$ shall not be too small. Since both $\rho_0$ and $W_p(\mu, \hat \mu_n)$ are unavailable in practical settings, the selection of the parameter seems tricky.
Questions
1. Why doesn't the dual form of outlier-robust WDRO in Proposition 7 reduce to that of WDRO when $\epsilon \rightarrow 0$, even if neglecting $\lambda_1$? Specifically, the operator $[\cdot]_+$ claimed to be vital for outlier-robust WDRO does not disappear when outlier-robust WDRO reduce to WDRO. Some typos: - Line 158: $\nu$ v.s. $\mu$ - Supplementary Line 565: In second equality, it might be $s_i\geq r_i$ and $s_i \geq 0$.
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
4 excellent
Contribution
3 good
Limitations
NA
Summary
It is well known that Wasserstein distances do not commute well with total variation distance - a slight perturbation in TV can change the Wasserstein distance by a lot. This means that models that are robust to corruptions in the data distribution in Wasserstein distance can still be vulnerable to outliers or corruptions in TV sense. This paper addresses this problem by considering robustness with respect to both TV and Wasserstein. The paper studies worst-case excess risk w.r.t. corruptions in a new distance termed the 'outlier robust Wasserstein distance'. The authors derive upper and lower bounds on the worst-case excess risk for families of distributions that are either sub-Gaussian or have bounded covariance. The upper bounds depend on a recently introduced term called the "resilience" of a probability measure. Intuitively, resilience of a measure quantifies the deviation in expectation of a function (in this case the loss function) when taken w.r.t. a measure in a probability ball around the original measure. The authors also propose a tractable reformulation of the min-max problem via strong duality. Finally, the authors tighten their results for the case when the true distribution lies on a low dimensional linear subspace, by extending their results to corruptions in outlier robust max-sliced Wasserstein distance. The authors also verify their excess risk bounds on a toy dataset for linear regression problem with mean absolute deviation loss.
Strengths
- significance: the problem of incorporating outlier robustness into the framework of wasserstein distributionally robust optimization (WDRO) is of significance to both ML and optimization communities, and has already received some recent interest. This paper makes progress on this significant problem. - novelty: I think the upper and lower bounds on the min-max excess risk are novel. The strong duality result appears to be a generalization of Gao and Kleywegt's result on strong duality for WDRO, but I think it is is non-trivial.
Weaknesses
- **Contextualizing the work within related literature**: Although the paper is overall well written, I wish it did a better job at contextualizing their results properly by comparing it with the two special cases of WDRO without TV corruption and of TV robustness without WDRO. For WDRO without TV corruption for example, a natural point of comparison is [this](https://optimization-online.org/wp-content/uploads/2016/04/5396.pdf) paper by Gao and Kleywegt. One point of comparison could be the existence and the form of the worst-case distribution in the min-max risk. This question is answered for the WDRO without TV corruption in Gao and Kleywegt's paper but this paper does not address it at all, which seems odd to me. For TV corruption without WDRO (aka Huber contamination), there are several works from the robust statistics community. - **Usefulness on top of stricter WDRO**: I think Remark 1 deserves much more discussion. From proposition 1 it is clear that the new compound model of outlier robust WDRO can be subsumed under plain vanilla DRO by increasing the budget of the Wasserstein contamination in proportion to the resilience of the true distribution. Then, how much additional utility do we gain by analyzing the min-max excess risk under the compound model in detail? I must admit I don't understand what the authors mean by the expensive pre-processing step for WDRO. Please explain this to me satisfactorily, and I am willing to change my opinion on this particular weakness.
Questions
- Proposition 3: Is this lower bound valid for p = 2, or only for p = 1? The loss function family is only assumed to be Lipsclitz and not $\alpha$-smooth, whereas for the upper bound, the loss is assumed to be $\alpha$-smooth for the case of p = 2. - Proposition 4 could just be a remark. - Something is broken in reference [38]. Are you sure this is the correct reference? I did not find a Proposition 3.4 in it. Also, as I have stated previously, I would very much like a closer comparison of the strong duality result with that of Gao and Kleywegt. Do both the results become identical if one of the lambda's is zero? - Do the results of section 4 still go through if the max-sliced wasserstein distance is replaced with the sliced wasserstein distance that takes expectation over all slices instead of max? - Lemma 5, which seems crucial to the proof of Theorem 1, is taken from [12], supposedly a forthcoming paper, which I am unable to access anywhere. So, I am unable to check Theorem 1 fully.
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
Yes.
Summary
This paper introduces an outlier-robust Wasserstein Distributionally Robust Optimization (DRO) framework that aims to capture both geometric uncertainties and non-geometric perturbations, such as adversarial outliers. By utilizing the outlier-robust Wasserstein distance, the proposed framework allows for the arbitrary corruption of a fraction of the data. The authors design an uncertainty set using a robust Wasserstein ball and derive minimax optimal excess risk bounds. They also establish a strong duality for efficient computation. The resulting problem involves tuning three parameters: - the bounded covariance parameter $\sigma$, - the radius of the ambiguity set $\rho$, - and the contamination parameter $\varepsilon$. Moreover, the authors address dimension dependencies in risk bounds for low-dimensional features by introducing the projection robust optimal transport. The paper concludes with experimental validation of the theory on regression and classification tasks.
Strengths
- The paper replaces the Wasserstein distance with the outlier-robust Wasserstein distance to tackle the case that the observed distribution is contaminated with outliers. - The authors establish the excess risk bounds of decisions for the cases $p=1,2$. - The authors derive tractable reformulation by replacing $\mathcal{G}_{\text{cov}}$ and leveraging dual of the problem. - The low-dimensional features are considered to address the problem of dimension dependency.
Weaknesses
- In Remark 3 and Appendix E, the paper briefly touches on the selection of the parameter $\varepsilon$. However, the overall discussion on parameter selection is limited. The model introduced in the paper involves several parameters, such as $\rho$, $\varepsilon$, and $\sigma$, which may not be fully independent. Therefore, a more comprehensive discussion about the tuning of these parameters is warranted. - In the experiments, the paper exclusively uses the standard Wasserstein DRO (WDRO) as the baseline for comparison. However, considering that both models aim to address the outlier challenge, it would be valuable to include DFO [A], which is another model specifically designed for handling outliers, as a reasonable baseline for comparison. - The presence of a few missing references should be addressed in order to enhance the completeness and accuracy of the paper. In Section 4, it would be beneficial to include references [B, C, D], which introduce the concepts of Wasserstein projection pursuit and projection robust Wasserstein, respectively. [A] Jiang, Nan, and Weijun Xie. "DFO: A Framework for Data-driven Decision-making with Endogenous Outliers." (2022). [B] Huang, Minhui, Shiqian Ma, and Lifeng Lai. "A riemannian block coordinate descent method for computing the projection robust wasserstein distance." *International Conference on Machine Learning*. PMLR, 2021. [C] Paty, François-Pierre, and Marco Cuturi. "Subspace robust Wasserstein distances." *International conference on machine learning*. PMLR, 2019. [D] Niles-Weed, Jonathan, and Philippe Rigollet. "Estimation of wasserstein distances in the spiked transport model." *Bernoulli* 28.4 (2022): 2663-2688.
Questions
- Would it be possible to provide further elaboration on the process of determining suitable parameter values for $\rho$, $\varepsilon$, and $\sigma$? - In the proof of Proposition 7 (Appendix B.8), is the constraint $\sum_{i\in [n]}m_i = 1$ included to ensure that $\mu^\prime$ is a valid probability distribution? It appears that in the problem mentioned above l566, the constraint should be $0 \leq r_i \leq s_i$ rather than the constraint $0 \leq s_i \leq r_i$. - In lines 48-49, it appears that the term $W_p^\varepsilon$ is defined in reference to outlier-robust optimal transport, rather than the partial optimal transport as described in [D], which focuses on transporting partial mass from the source distribution to the target distribution.
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
2 fair
Limitations
Unless I missed it, I believe the authors do not expand on the limitations of their approach. It would help to add a short section discussing that. This work does not have a negative societal impact.
Summary
This paper introduces a novel approach to make Wasserstein distributionally robust optimization problems robust to adversarial outliers including geometric perturbations and non-geometric contamination of the data. This goal is achieved by considering relevant Wasserstein ball which includes both type of adversarial attacks. Later, the authors provides strong duality results which allows them to improve the computation complexity of the proposed outlier-robust WDRO problem.
Strengths
This is a well written paper with novel contributions. The robustness of WDRO to the outliers is a very natural problem and the authors proposed a Wasserstein distance based constraints to provide robustness. This is particularly hard if we do not make any presumptions on the distribution of data; therefore, I find the contributions of this paper to be significant to the literature. There might be some concerns that I have mentioned in the weaknesses part, but, overall I am impressed by the work done here.
Weaknesses
As I have mentioned at the strengths part, I believe this paper has a novel contribution to the robust optimization literature. Some few weaknesses that I observed was that authors assumed the readers have prior knowledge on relation between adverserial attacks and Wasserstein and TV perturbations. These are not so obvious for me and might be useful to include some discussion/motivation about how these attacks can be associated with claimed perturbations in metric space. I believe it would strengthen the paper if authors can obtain some results on larger datasets. Considering the time constraint of the conference, I believe it would be sufficient to add one large dataset to experiments.
Questions
- What is the definition for $\theta$ performing uniformly well over Wasserstein ball (stated in Line 29)? - I could not see the definition of $\Sigma_\mu$ in the paper.
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
The Assumption 2 can be challenging to maintain/check in practice. Moreover, this application might also be hard to implement on large datasets such as classification problems on MNIST or FMNIST. That being said, I don't see these possible limitations as a major problem considering the theoretical novelty of this work.
Thanks for the authors for their response. My concern over Weakness 2,3 and Question 1 have been addressed. Notably, the authors made a major mistake in the dual reformulation of DRO's objective and the empirical implementation. Fortunately, the authors have corrected both the theoretical and empirical results. My major concern over Weakness 1 around the advantage of outlier-robust WDRO over WDRO remains. The authors provide the excess risk of vanilla WDRO formulated as $\||\ell\|| (\sqrt{d}\rho + \sqrt{d\epsilon})$, in contrast to that of outlier-robust WDRO formulated as $\||\ell\|| (\rho + \sqrt{d\epsilon})$ in Corollary 1. However, the gap between two DROs is $(\sqrt{d}-1)\rho$ which does not depend on the TV contamination level $\epsilon$. Why does outlier-robust WDRO outperform WDRO when $\epsilon=0$ and only geometric contamination exists? The result does not validate WDRO's superiority in the case of TV contamination. I believe the stronger bound of the excess risk for vanilla WDRO with any radius, which is ongoing work, might elucidate my concern. Since the paper is motivated from WDRO under TV contamination, I insist that a thorough comparison is important. I would like to keep my score for now until further results or clarification is made by the authors.
When $\varepsilon = 0$, one may center standard WDRO around the observed distribution $\tilde{\mu}$ with radius $\rho$, since a Wasserstein ball of radius $\rho$ about $\tilde{\mu}$ will contain the true distribution $\mu$. In this case, the performance of outlier-robust WDRO and standard WDRO are the same. However, as soon as $\varepsilon > 0$ (no matter how small), standard WDRO on its own is no longer sufficient, because we have no bound on $\mathsf{W}_p(\tilde{\mu},\mu)$ (indeed, $\mathsf{W}_p(\mu,(1-\varepsilon)\mu + \varepsilon \delta_z) \to \infty$ as $||z|| \to \infty$). To remedy this efficiently, we proposed above to recenter standard WDRO around the efficiently computable estimate $\check{\mu}$ produced via iterative filtering. However, this estimate can only be guaranteed to satisfy $\mathsf{W}_p(\check{\mu},\mu) \leq \sqrt{d}\rho + \sqrt{d\varepsilon}$, hence the degraded risk bound. We note that this guarantee for $\check{\mu}$ is tight and novel, and in general that no approach using standard WDRO existed for our problem before this work. We will update the reviewer if we can prove the mentioned stronger lower bound w/in the discussion period. In any case, we will add this filter + standard WDRO approach as a baseline for comparison to our experiments in the final version.
Thanks for the authors’ response. My concern over parameter selection has been addressed and the advantage over WDRO is proven for a remedy of WDRO utilizing iterative filtering. Thus, I would like to raise the score. My remaining concern is over a direct comparison of the excess risk in the general condition where the clean distribution might not be included in the uncertainty set.
Below we provide a separation between our approach and standard WDRO centered around the iterative filtering estimate (with any radius). Consider the setting where clean data is distributed according to $\mu = \delta\_0$ and the learner observes $\tilde{\mu} = (1-2\varepsilon)\delta\_0 + \varepsilon \mathcal{N}(0,\frac{\rho^2}{\varepsilon^2} e\_1 e\_1^\top) + \varepsilon \mathcal{N}(0,\frac{\rho^2}{2\varepsilon^2}I\_d)$. Here, the second component involves a Wasserstein step of size $\leq \rho$ and the third component is the TV corruption. Iterative filtering seeks to remove an $\varepsilon$ fraction of mass to drive down the operator norm of the covariance matrix, and as a result, the filtered estimate $\check{\mu}$ based on $\tilde{\mu}$ will mostly remove mass from the second component. Based on this guarantee of filtering, we can prove that $$\mathsf{W}\_1(\check{\mu},\mu) \geq \mathbb{E}\_{\check{\mu}}[\\|Z\\|] \gtrsim \varepsilon\, \mathbb{E}\_{\mathcal{N}\big(0,\tfrac{\rho^2}{2\varepsilon^2}I\_d\big)} [\\|Z\\|]\gtrsim \sqrt{d}\rho =: A.$$ Now suppose the learner is faced with deciding between $\ell_0(z) =\\|z\\|$ and $\ell_1(z) = A - \\|z\\|$. If they perform standard WDRO with any radius centered around $\check{\mu}$, they will choose $\ell_1$ (since the Lipschitz regularizer resulting from WDRO is identical for these two functions). However, $\mathbb{E}\_\mu[\ell_1] = A \gg \mathbb{E}\_\mu[\ell_0] = 0$, i.e., filtering + WDRO yields an excess risk $\Omega(\sqrt{d}\rho)$. We note that a choice of $\mu$ with unit variance could drive this risk up to $\sqrt{d}\rho + \sqrt{d\varepsilon}$. In any case, this $\sqrt{d}\rho$ component of the risk is highly suboptimal when $d$ is large. This establishes the desired separation between the proposed outlier-robust WDRO approach and standard WDRO with expanded radius. A formal statement of this separation, along with the proof and corresponding discussion, will be added to the revised paper to supplement the current Remark 1.
Thank you for your kind response. We were wondering if there are any other additions to the text that the reviewer would like to see that could further improve their assessment of the work?
I just realized that I did not respond to this comment and I apologize. The main reason I would like to keep my rating is due the fact that I believe there is more room for improvement on the experiments. In the context of outlier detection, I believe there could have been more done with MNIST data such as various digits (more than two) as outlier and playing with the training flip rate. Moreover, another alternative is to use two or more digits as normal and picking one as outlier could also be used for comparison as well. I am very satisfied with the discussion so far and it looked like most of the reviewer also provide similar ratings. So, I would still like to keep mine if that would not majorly effect area chairs final decision.
There is a more precise relation between adversarial attacks and Wasserstein perturbations.
Upon reading the review of vYVj and your response, I would like to point out that there is a more precise relation between adversarial attacks and Wasserstein perturbations besides the intuition provided in your response. Reference [*] establishes the equivalence between robustness against adversarial attacks with perturbation radius $\epsilon$ and distributional robustness in a ball of radius $\epsilon$ w.r.t. $W_\infty$ metric. I wonder if your strong duality result holds as $p\to \infty$. [*] Pydi, M. S., & Jog, V. (2021). The many faces of adversarial risk. Advances in Neural Information Processing Systems, 34, 10000-10012.
Indeed, there is an equivalence between point-wise adversarial attacks and $\mathsf{W}\_\infty$ perturbations for standard WDRO. Fix observed data $\tilde{\mu}\_n = \frac{1}{n} \sum\_{i=1}^n \delta\_{\tilde{z}\_i}$, and write $\mathcal{B}\_\infty := \\{ \nu \in \mathcal{P}(\mathcal{Z}) : \mathsf{W}\_\infty(\nu,\tilde{\mu}\_n) \leq \rho \\}$. By Lemma EC2 of [A], $\mathcal{B}\_\infty$ admits the equivalent representation $$ \mathcal{B}\_\infty = \left\\{\ \frac{1}{n}\sum\_{i=1}^n \delta\_{z\_i} : \\|z\_i - \tilde{z}\_i\\| \leq \rho, z_i \in \mathcal{Z} \right\\}, $$ and we have $$ \sup\_{\nu \in \mathcal{B}\_\infty} \mathbb{E}\_\nu[\ell(Z)] = \mathbb{E}\_{\tilde{\mu}\_n}[\bar{\ell}(Z)], $$ where $\bar{\ell}(z) := \sup\_{z' \in \mathcal{Z}, \\|z' - z\\| \leq \rho} \ell(z')$. Our theory also extends naturally to this $p \to \infty$ limit. For the robust Wasserstein ball $\mathcal{B}\_\infty^\varepsilon := \\{ \nu \in \mathcal{Z} : \mathsf{W}\_\infty^\varepsilon(\nu,\tilde{\mu}\_n) \leq \rho \\}$, we can similarly prove $$ \sup\_{\nu \in \mathcal{B}\_\infty^\varepsilon} \mathbb{E}\_\nu[\ell(Z)] = \mathrm{CVaR}\_{\tilde{\mu}\_n}^{1-\varepsilon}[\bar{\ell}(Z)], $$ where $\mathrm{CVaR}$ is the conditional variance at risk appearing in our corrected dual. Enforcing our moment constraints, we can prove $$ \sup\_{\nu \in \mathcal{B}\_\infty^\varepsilon \cap \mathcal{G}\_2(\sigma,z\_0)} \mathbb{E}\_\nu[\ell(Z)] = \inf\_{\lambda \geq 0} \lambda \sigma^2 + \mathrm{CVaR}\_{\tilde{\mu}\_n}^{1-\varepsilon}[\bar{\ell}\_2(Z)], $$ where $\bar{\ell}\_2(z) := \sup\_{z' \in \mathcal{Z} : \\|z' - z\\| \leq \rho} \ell(z') - \lambda\\|z' - z\_0\\|^2$, and $$ \sup\_{\nu \in \mathcal{B}\_\infty^\varepsilon \cap \mathcal{G}\_\mathrm{cov}(\sigma,z\_0)} \mathbb{E}\_\nu[\ell(Z)] = \inf\_{\Lambda \succeq 0} z\_0^\top \Lambda z\_0 + \sigma^2 \mathrm{Tr}(\Lambda) + \mathrm{CVaR}\_{\tilde{\mu}\_n}^{1-\varepsilon}[\bar{\ell}\_\mathrm{cov}(Z)], $$ where $\bar{\ell}\_\mathrm{cov}(z) := \sup\_{z' \in \mathcal{Z} : \\|z' - z\\| \leq \rho} \ell(z') - z'^\top \Lambda z'$. Note that both cases recover the standard dual as $\varepsilon \to 0$ and $\sigma \to \infty$. [A] : Rui Gao, Xi Chen, and Anton J. Kleywegt. Wasserstein Distributionally Robust Optimization and Variation Regularization. Operations Research, 2022, to be published. doi:10.1287/opre.2022.2383
Thank you for your response. I did not notice the missing constraint in your original Lagrangian formulation, but it is good that this error is fixed and we indeed recover the pure WDRO result as $\epsilon \to 0$. I also appreciate your response to my second weakness. Regarding my comment on your response to reviewer vYVj, could you also perhaps comment on what effect does taking p in Wp to infinity have on your results? I am especially curious to see what happens to the strong duality result as $p\to\infty$. I understand this question may be non-trivial to answer, but my interest in this is stemming from the work of Pydi & Jog Neurips 2021 who showed a connection between $W_\infty$ perturbations and adversarial attacks.
Response to authors
Thank you for providing clarifications that address my concerns. I'm pleased to see the correct dual problem formulation. After carefully reading all the reviews and rebuttals, I have decided to revise my score from 4 to 6.
Decision
Accept (poster)