Automatic Outlier Rectification via Optimal Transport

Summary

A framework for regression tasks with outliers (see robust statistics) is introduced, based on an optimal transport cost function. A novel concave cost function directly facilitates the rectification of outliers. Mathematical proofs are given and evaluation is performed on two simpler and one real world task with state-of-the-art results.

Strengths

- The theory and field of robust statistics is well explained and brought into context nicely with adjacent topics, with especially Fig. 1 and its explanation providing a good lens to view these problems - The paper is organized exceptionally well, despite its size (incl. the huge appendix) - All proofs in Appendix C are generally easy to understand, even without strong mathematical background. I was even able to thoroughly verify proofs in C.1 myself - All experiments are explained in great depth and detail in the appendix material, making both a surface and deeper understanding of all topics possible.

Weaknesses

- Some of the tasks involve artificially constructing outliers (such as the cluster of outliers in mean estimation & raising the price p to 10 * p in the deep learning volatility experiment), which may not correspond to real-world data corruption - as the code/data seems to be proprietary, no publication is intended, which may hinder reproducibility Typos and Formatting Issues: - the authors falsely reference eq. (3), when really they want to link eq. (4) in lines, 145, 147, 149, 180, 183 (non exhaustive list) - missing citation in line 160 - formula slightly clips into the text at line 156 - typo in line 746: "esstimator" - typo in line 874: "exporation," - wrong citation: Nietert et al "Outlier-robust wasserstein dro" appeared in NeurIPS 2023, not 2024

Questions

Q1: why are there no other "robust" variants of regular estimatiors used as baselines for the experiments? (i.e. mean estimation task,...) Q2: Mean&LAD (App.E): The tasks look relatively simple with two very clear clusters of inliers and outliers. Why are there no experiments where there are many outliers sparsely distributed across the data with high variance? These are the tasks I usually see in robust model fitting (i.e. for homography estimation in Computer Vision). Would the estimator struggle to transport many points different amounts of smaller distances, to stay within the "long haul" metaphore? Q2.1: The same goes for the Deep Learning comparison. Without knowing the variance of the price, going from p to 10p seems like a huge step. Was an ablation on the factor of the price (here 10) done?

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

- no potential negative societal impact exist - currently limited to regression tasks, although the authors point towards possible extensions to classification in Appendix B - evaluation is currently limited to the authors toy tasks and a few tasks from finance. the broad applicability of their method could have also contributed to more extensive experiments

Summary

The paper presents a new framework for outlier detection using optimal transport with a concave cost function. Unlike traditional methods that separate outlier detection and estimation, this approach integrates both into a single optimization process, improving efficiency. The concave cost function helps accurately identify outliers. The method outperforms traditional approaches in simulations and empirical analyses for tasks like mean estimation, least absolute regression, and fitting option implied volatility surfaces.

Strengths

- The paper is well-written - The proposed method is novel and theoretically sound - Extensive experiments are conducted to support the claims

Weaknesses

- Could the authors discuss the relationship between the proposed method and M-estimator and RANSAC?

Questions

Please see Weaknesses

Rating

5

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

N/A

Summary

The paper presents a novel method for outlier rectification in robust statistics. In particular, taking inspiration from distributionally robust optimization (DRO) methods, this paper proposes to extend the formulation to the field of robust statistics. To this end, a rectification set is constructed using optimal transport distances with concave cost functions. The authors have proven theoretically that the reformulation is eventually equivalent to an adaptive quantile (regression) estimator with the quantile controlled by the budget parameter for mean estimation and least absolute regression. Experiments are conducted to demonstrate the effectiveness of the proposed method on various tasks, including mean estimation, least absolute regression, and option implied volatility surface estimation.

Strengths

Theoretically, the paper explores an interesting extension of DRO in the context of robust statistics. The introduction of optimal transport distance with concave loss functions is well-grounded. The resulting algorithm admits an intuitive explanation.

Weaknesses

1. It seems to me that theoretically, the presentation is not particularly clear regarding how certain designs, such as optimal transport and concave cost functions, affect the final algorithm. At the core of the connection is Proposition 1 in the paper, but the proof is not provided. 2. Experiment-wise, I find that the baseline methods are somewhat weak. The baseline methods in Sec 5.2.1 are very old, and in Sec 5.2.2 the baseline method does not consider robustness in estimation. It would be good if the adaptive quantile estimator is included for comparison. It is also advised to not put the main experimental results of Sec 5.1 in Appendix. Other comments: Eq.3 is wrongly referenced (I believe it should be Eq.4) in multiple sentences, such as line 145. reference is missing in line 160

Questions

Please respond to the two points in Weaknesses.

Rating

5

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

I cannot find a discussion of limitations in the main paper or appendix.

Summary

This paper is, essentially, about robust model fitting. In this paper, it is proposed to optimize an optimal transport problem. As loss function, the authors choose concave functions ||z-z‘||^r with r \in (0,1). The idea behind choosing this shape of functions is that it allows to move outliers farther due to decreasing increases of transportation costs. The authors provide a proof that outliers can be identified easily by solving a linear program. Alternatively, they propose to use the quick select algorithm. Experimental results of the proposed algorithm are presented on option volatility prediction, a problem in finance.

Strengths

* I like that the paper presents a mathematically concise framework how to identify outliers in data. This is different from many practical algorithms that propose ad-hoc heuristics. * I like that the paper is written in a relatively accessible way. * The problem addressed here is important in many real-world problem.

Weaknesses

* As far as I can can tell, the idea is novel yet somewhat similar to what has been proposed in this paper: Chin et al., Accelerated Hypothesis Generation for Multistructure Data via Preference Analysis, TPAMI 2012 It would be nice if the authors could cite and discuss this paper. * The authors state in their abstract that the commonly used 2-step approach (first outlier removal followed by model fitting) does not provide information about the outliers to the model fitting stage. While that is true, I wonder which information and how their algorithm provides to the model fitting as there is no flow of information (for instance, by backpropagation) between outlier removal or knot point identification and the gradient step.

Questions

* What is delta in Fig 2? Is this the budget? If so, please mention this in the caption of the figure. * I would have liked to see more figures on the selected knot points and lambda, for instance some examples for which the proposed algorithms (LP vs QSelect) work well or fail.

Rating

7

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

I do think so. I believe that this algorithm constitutes an important contribution to the field and can inspire new papers by other researchers.

Summary

A new approach for robust estimation is proposed based inspired by optimal transport. The general approach is introduced and then new estimators are derived for three particular cases : mean estimation, linear fitting and surface regression. The estimators are compared numerically with standard estimators on synthetic data for the first two cases and on financial datasets for surface regression. Many details, proofs and explanation are provided in the appendix which is 25 pages long.

Strengths

The proposed approach seems very original and interesting. A new robust estimator may have a major impact on all the learning research field.

Weaknesses

The paper is well presented and explained but the author material is so large that an article is not enough. There is probably material for a book. This raise the question to publish a paper with so large appendix ? The link with neural networks needs to be reinforced. A few details : page 4, line 160 : reference missing. page 6, line 227 : distirbution => distribution

Questions

Is this works may help to interpret and better understand the performances obtained by back-propagation during learning ? Details : page 5, line 200 : where is delta in problem (3) ? page 6, line 209 : is it equation (4) or (3) ?

Rating

8

Confidence

3

Soundness

3

Presentation

3

Contribution

4

Limitations

The proposed approach being very generic, the limitations are quite reduced. There are probably no potential negative societal impact.

Rebuttal reminder

Hello, it has been about two days since we've posted our rebuttal, and we wanted to gently remind the reviewers to view it. We politely request that the reviewers either provide further feedback, or, if they agree that our response has addressed their concerns, to please increase their score. Thank you!

Response to Official Comment by Reviewer qw3a

We greatly appreciate the adjusted score and the additional positive feedback. We also appreciate the further questions on your question Q2. We understand the concern regarding the clustering of outliers, and we agree that an additional example with non-clustered outliers will improve the paper. We will add this example to the camera-ready version.