Summary
This paper tackles the challenges associated with noisy and imbalanced training data in a scenario of adversarial training. The proposed method is composed of an oracle training process (resampling, refurbishment & split, and contrastive learning) and adversarial training (distribution estimation and logit adjustment). Experimental results using CIFAR-10 and 100 shows much better clean accuracy (+20% points) and robust accuracy (+10% points) than the baseline methods.
Strengths
+ This paper is the first trial to achieve better clean and robust accuracy for adversarial training under the noisy and imbalanced conditions.
+ Multiple techniques are efficiently combined for better adversarial training and the key components in this work are all reasonable.
+ The effectiveness of the proposed method is clearly shown with CIFAR-10 and 100 by significantly outperforming the baseline methods in terms of both clean accuracy and robust accuracy.
Weaknesses
1. I am afraid the technical novelty is relatively weak.
- Data-resampling (over-sampling, sub-sampling, SMOTE, etc) can be found in many papers.
- The label refurbishment and dataset split using k-NN seems essentially the same as in the following paper. If not, the authors should clarify the novelty more clearly.
[a] Dara Bahri, Heinrich Jiang, and Maya Gupta. Deep k-nn for noisy labels, Proceedings of Machine Learning Research, 2020.
[b] Chen Feng, Georgios Tzimiropoulos, Ioannis Patras. SSR: An Efficient and Robust Framework for Learning with Unknown Label Noise, BMVC, 2022.
-- Contrastive self-supervised learning is also well-known.
-- Label distribution estimation, i.e., label reassignment, is commonly used as in [b].
-- Logit adjustment is simply taken from [Menon, ICLR21]
2. The experimental validation is weak
- The experiments are conducted only with CIFAR-10 and 100. I admit these two datasets are standard, but the authors might want to show more results with larger datasets to give greater impact to the community.
- The baselines are all simple adversarial training methods. As the authors might be aware, there are already many methods for noisy labels and imbalanced data. The authors might want to at least apply such methods to the baselines. Otherwise, I am afraid that I cannot say the comparison is fair.
Questions
As pointed out in the weakness part, I have concerns about the technical novelty. Please clarify which part was technically novel.
Just as a reference, an adversarial training method for imbalanced datasets has been presented recently. Since this paper was officially published in Oct. 2023, I am NOT requesting the authors to compare their work to this one.
Wentao Wang, Harry Shomer, Yuxuan Wan, Yaxin Li, Jiangtao Huang, and Hui Liu. 2023. A Mix-up Strategy to Enhance Adversarial Training with Imbalanced Data. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management (CIKM '23).
Rating
5: marginally below the acceptance threshold
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.