Models trained with empirical risk minimization (ERM) are prone to be biased towards spurious correlations between target labels and bias attributes, which leads to poor performance on data groups lacking spurious correlations. It is particularly challenging to address this problem when access to bias labels is not permitted. To mitigate the effect of spurious correlations without bias labels, we first introduce a novel training objective designed to robustly enhance model performance across all data samples, irrespective of the presence of spurious correlations. From this objective, we then derive a debiasing method, Disagreement Probability based Resampling for debiasing (DPR), which does not require bias labels. DPR leverages the disagreement between the target label and the prediction of a biased model to identify bias-conflicting samples-those without spurious correlations-and upsamples them according to the disagreement probability. Empirical evaluations on multiple benchmarks demonstrate that DPR achieves state-of-the-art performance over existing baselines that do not use bias labels. Furthermore, we provide a theoretical analysis that details how DPR reduces dependency on spurious correlations.
Paper
Similar papers
Peer review
Summary
To address the issue of spurious correlations when bias labels are unavailable, the work proposes a new method to mitigate the spurious correlations by minimizing the maximum average loss over bias-aligned and bias-conflicting groups. Additionally, they introduce the disagreement probability/sampling probability, which weights samples in the training objective, to achieve group-robust learning.
Strengths
1. The organization of this paper is good, and it is written in a clear and understandable manner. 2. This work is comprehensive as it covers both theoretical and experimental analyses. 3. The experiments are extensive, considering a rich variety of datasets and baselines.
Weaknesses
1. The method proposed in this paper, DPR, is not novel. Although the authors introduce several concepts such as sampling probability and group indicator, the essence of DPR can be seen as first using ERM with GCE to predict pseudo spurious attribute labels, and then applying the Group DRO method. Similar approaches (pseudo group labels from ERM + invariant learning) can be found in several methods like JTT [1] and CnC [2]. The essence of the author's emphasis on disagreement probability seems to be the predicted probability of ERM that a sample belongs to the minority group (i.e., bias-conflicting groups). Similarly, the reweighting of samples using the predicted probabilities by ERM can be seen in methods like EIIL [3] (the main difference is EIIL focuses more on the regularization term). 2. The use of data augmentation helps alleviate spurious correlations, while other baseline models JTT do not utilize data augmentation, which may lead to unfair comparisons in experiments. 3. The experimental section lacks unity, as evidenced by inconsistent metrics and experimental datasets. It is recommended to focus on the worst group (such as C-MNIST, MB-MNIST) to illustrate the effectiveness of DPR in mitigating spurious correlations. Additionally, I noticed that the average accuracy of CivilComments-WILDS decreases significantly compared to other baselines, which contradicts the theoretical claim in the article of simultaneously reducing the loss for bias-aligned and bias-conflicting groups. Can the authors provide insights into this? 4. I appreciate the authors' demonstration of section Identifying group via disagreement probability, which shows the effectiveness of biased models in predicting bias-conflicting groups. This is an important evidence for measuring the effectiveness of DPR. However, showcasing it only on C-MNIST is not sufficient. Can you provide experimental results on other datasets, especially real-world datasets? If authors could provide reasonable replies, I am willing to further increase the score. [1] Liu, Evan Z., et al. "Just train twice: Improving group robustness without training group information." International Conference on Machine Learning. PMLR, 2021. [2] Zhang, Michael, et al. "Correct-n-contrast: A contrastive approach for improving robustness to spurious correlations." arXiv preprint arXiv:2203.01517 (2022). [3] Creager, Elliot, Jörn-Henrik Jacobsen, and Richard Zemel. "Environment inference for invariant learning." International Conference on Machine Learning. PMLR, 2021.
Questions
see my weaknesses.
Rating
5
Confidence
4
Soundness
2
Presentation
3
Contribution
2
Limitations
see my weaknesses.
Dear Author, Thank you for your response and the additional experimental results. Regarding Q1, although EIIL and DPR use different terminologies, in the presence of spurious features, EIIL encourages the model to violate the principles of invariant learning by maximizing the EI objective in the first stage, thereby learning the spurious attribute labels of the samples (i.e., biased model). The model probabilities (in a binary classification scenario) are then used as sample weights for invariant learning in the next stage (i.e., debiased model). Despite differences in algorithm details, EIIL is close to DPR. For Q2, could you please confirm where JTT and CnC indicate that they use data augmentation techniques? For instance, in the case of JTT, the original paper explicitly states in Section A Training Details that the experimental settings for Waterbirds and CelebA are "no data augmentation" (please correct me if I am wrong). Best, Reviewer GDUK
We appreciate your comments and provide answers to each question. For Q1, the differences between EIIL and the proposed DPR are as follows: First, in terms of the algorithm, there are distinct differences between EIIL and DPR. The most significant difference is that in DPR, the weighted loss function used to train the debiased model is mathematically derived directly from the training objective under an easily achievable assumption. We also utilized the characteristics of the biased model (i.e., relatively low loss for the bias-aligned group) to satisfy this assumption, ensuring that the loss function used to train the debiased model aligns with the training objective. This process of deriving the loss function and the resulting loss function itself are novel and have not been proposed in previous debiasing papers, including EIIL. Therefore, this newly derived loss function leads to a new algorithm, DPR, which is distinct from EIIL. Although it may use similar components (i.e., biased model, reweighting), DPR creatively combines these components, as it is derived by the newly derived loss function. Secondly, the group definitions used by EIIL's objective function and our proposed objective function are different. Upon examining EIIL's objective function and algorithm, it seems that EIIL has no restrictions on the number of groups (i.e., environments) and this is not explicitly specified. In contrast, we explicitly defined two groups - bias-aligned and bias-conflicting - and based our objective function on these two groups. For example, regardless of whether a dataset consists of 6 classes or 10 classes, or irrespective of the dataset's composition and characteristics, we propose dividing it into two groups in every situation and building our objective function upon these two groups. We would like to point out that there is a difference between our approach and EIIL's, as EIIL did not start by dividing into two groups to set up its objective function, but rather proposed an objective function and algorithm for an arbitrary number of groups. The differences mentioned above have indeed led to performance disparities. As you can see in the comparison experiment table above for CelebA and CivilComments-WILDS, there is a significant performance gap, showing that our DPR outperforms EIIL. We believe this highlights the differences from EIIL and demonstrates the superiority of our method. For Q2, we followed their experimental settings, including data augmentation policies (e.g., whether to use it, types, etc.). Therefore, since JTT and CNC did not use data augmentation for CivilComments-WILDS and CelebA, we also refrained from using it.
Dear author, Thank you for your detailed reply. After reading the reply and the opinions of other reviewers, I choose to keep my score. I still think the novelty of this work is limited. Besides, I am concerned about the fairness of the experiments. As far as I know, data augmentation (which helps alleviate spurious correlations) was not used for the baselines such as JTT and CnC on CelebA and CivilComments-WILDS. I suggest the author should further explain in detail in the main text or appendix what experimental settings they used to ensure that the comparison is reasonable. To sum up, I choose to maintain my score. Best, Reviewer GDUK
Thank you for taking the time to read my response and reply. Additionally, I would like to clarify something regarding the experimental setup. When we said the experimental settings were the same, we meant that comparative experiments were conducted in the same experimental environment as the baselines. Although different data augmentation policies may have been applied to each dataset, all baselines, including our proposed DPR, were tested using the same data augmentation policy for each dataset. In other words, for CelebA and CivilComments-WILDS, no data augmentation was used for all other baselines including JTT and CNC, as well as the proposed DPR. For other datasets including BFFHQ, experiments were conducted using data augmentation for all baselines and the proposed DPR. This setting was used to ensure fairness in comparative experiments. As you suggested, we will include this information in the appendix of the revised version. We appreciate your careful comments.
Thank you for clarifying the experiment. Please add the description of the experimental setting in the revised version. According to the author's explanation, I am willing to increase my score.
We sincerely thank you for your insightful suggestions and the decision to raise the score. We will include the detailed experimental settings in the revised version.
Summary
The authors mainly target fairness without accessing bias labels. They suggest a new learning objective that minimizes the loss of the bias group showing the highest ERM and demonstrate that minimizing this objective decreases the upper bound of the expected average loss. To utilize this loss when the bias labels are not accessible, they derive the oversampling method from the new learning objective. In experiments, they show that their method outperforms baselines on various datasets.
Strengths
* They suggest a new learning objective and provide theoretical support for the proposed objective. * The authors derive the sampling weights from the perspective of the new learning objective. * They demonstrate that their method outperforms baselines on various benchmark datasets.
Weaknesses
* Since the sampling weights are derived from the proposed learning objective, it's important to demonstrate the effectiveness of the proposed learning objective itself. Could you confirm if it is superior to GroupDRO in scenarios where bias labels are provided on CelebA? * The effectiveness of the proposed method compared to baselines seems weak. Except for the synthetic datasets and BFFHQ, their performance gain is marginal. * There is no clear explanation of how to choose hyperparameters or select the model from a specific epoch during training. In the absence of an unbiased validation set, such as BFFHQ, how were these selections made? * In situations where the ratio of bias-conflicting samples is low, using oversampling methods might make training unstable. Could you show how the test accuracy of bias-conflicting samples changes every 10 epochs during the whole training process on BFFHQ (160 epochs as mentioned in the Appendix)?
Questions
* According to Section 4.3, the authors introduce additional augmentation for the diversity of minor class samples. Did the other algorithms compared in the experiment also use these augmentation techniques?
Rating
6
Confidence
3
Soundness
3
Presentation
3
Contribution
3
Limitations
The authors properly address the limitations in Section 7.
Summary
This paper addresses the critical issue of spurious correlations in machine learning models, where models often rely on easy-to-learn bias attributes rather than the intended target features, leading to poor generalization on minority groups where these spurious correlations are absent. The authors propose a novel method called Disagreement Probability based Resampling for debiasing (DPR), which aims to mitigate the effects of spurious correlations without requiring explicit bias labels during training. The key contributions of this paper are: 1. A new training objective designed to achieve consistent model performance across both bias-aligned and bias-conflicting groups, encouraging robustness against spurious correlations. 2. Development of DPR, a practical resampling method derived from the proposed objective. DPR leverages the disagreement probability between the target label and the prediction of a biased model to identify and upsample bias-conflicting samples. 3. Theoretical analysis demonstrating that DPR minimizes losses for both bias-aligned and bias-conflicting groups while reducing the disparity between their losses. 4. Extensive empirical evaluation on six benchmark datasets (including both synthetic and real-world data) showing that DPR achieves state-of-the-art performance compared to existing methods that do not use bias labels. 5. Ablation studies and analyses that provide insights into the effectiveness of various components of the proposed method, such as model initialization, generalized cross-entropy loss, and data augmentation.
Strengths
### Originality: 1. Novel objective formulation: The authors introduce a new training objective designed to achieve consistent performance across bias-aligned and bias-conflicting groups. This approach differs from previous works that typically define groups based on combinations of target labels and bias labels. 2. Creative use of disagreement probability: The method leverages the disagreement between target labels and biased model predictions as a proxy for identifying bias-conflicting samples. This is an innovative way to address the challenge of not having explicit bias labels. 3. Unique combination of existing ideas: DPR creatively combines ideas from biased model training, resampling techniques, and theoretical analysis to create a cohesive and effective debiasing method. ### Quality: 1. Comprehensive empirical evaluation: The authors test their method on six diverse benchmarks, including both synthetic and real-world datasets, providing a thorough assessment of DPR's performance. 2. Theoretical foundations: The paper includes a rigorous theoretical analysis that supports the empirical results, demonstrating how DPR reduces loss disparity between groups and minimizes average loss. 3. Ablation studies: The authors conduct detailed ablation studies to understand the contribution of each component of their method, showing a commitment to thorough scientific investigation. 4. Comparison with state-of-the-art: DPR is compared against multiple recent baselines, consistently showing superior or comparable performance. ### Clarity: 1. Clear problem formulation: The authors provide a clear definition of the problem and their approach to solving it. 2. Step-by-step derivation: The development of DPR from the initial objective to the final algorithm is presented in a logical, easy-to-follow manner. 3. Visual aids: The paper includes helpful figures and tables that illustrate key concepts and results. 4. Detailed experimental setup: The authors provide comprehensive information about their experimental setup, facilitating reproducibility. ### Significance: 1. Addressing a crucial challenge: Spurious correlations are a major issue in machine learning, and this work provides a novel approach to mitigating them without requiring bias labels, which is often a practical constraint in real-world scenarios. 2. Broad applicability: The method is shown to be effective across various types of data (image and text) and problem setups, suggesting its potential for wide adoption in different domains. 3. Theoretical and practical relevance: By providing both theoretical guarantees and strong empirical results, the paper bridges the gap between theory and practice in addressing spurious correlations. 4. Potential impact on fair ML: The proposed method could contribute to the development of fairer and more robust machine learning models, which is a critical goal in the field.
Weaknesses
Nothing
Questions
Nothing
Rating
8
Confidence
4
Soundness
4
Presentation
4
Contribution
4
Limitations
Authors have discussed the limitations sufficiently.
Thank you very much for your comments and for taking the time to review our responses. Additionally, We really appreciate your review and your decision to maintain your score.
Summary
This paper proposes a re-sampling approach based on disagreement between bias predictions and target label predictions. First, a biased model is trained using generalized cross-entropy. Then, sample-wise weights are determined by calculating the probability of disagreement between the bias predictions of the biased model and the target label predictions of the target model.
Strengths
1. This paper tackles the significant and pressing issue of robust learning, which is crucial for effectively using trained models in real-world applications. 2. The proposed sampling method is both simple and straightforward, making it easy to understand and implement.
Weaknesses
1. The method extends the approach of upweighting misclassified samples in the existing Just Train Twice (JTT) by utilizing generalized cross-entropy (GCE) and the disagreement between bias predictions and target label predictions. However, this extension does not appear novel. Furthermore, do there not exist approaches that use bias predictions? If so, discussing these studies would provide context and underscore the novelty of the proposed method. 2. Color jitter is employed as a data augmentation technique that can directly solve the color bias in CMNIST. However, the ablation study does not show significant performance improvement. What is the reason for this outcome? Additionally, for a fair evaluation, the same data augmentation should be applied to the baselines. 3. CMNIST is a relatively easy dataset, and BAR is not widely used recently. Therefore, additional experiments on other widely used datasets, such as CIFAR-10C, Waterbird, or NICO, are needed. Demonstrating similar performance improvements on these datasets would better support the experimental effectiveness of the proposed method. 4. The related works section does not discuss some recent studies, such as [1, 2]. [1] Lee et al., "Surgical fine-tuning improves adaptation to distribution shifts.", ICLR'23 [2] Jung et al., "Fighting Fire with Fire: Contrastive Debiasing without Bias-free Data via Generative Bias-transformation", ICML'23
Questions
Please see the Weaknesses
Rating
5
Confidence
5
Soundness
2
Presentation
2
Contribution
2
Limitations
Please see the Weaknesses
Thank you for your detailed responses. Many of my concerns have been clarified. However, the concern regarding color augmentation still remains. As you mentioned, applying the same color augmentation to the baselines may ensure a fair comparison within that specific setting. However, this does not necessarily equate to an accurate evaluation of each method's debiasing effects. The vast majority of debiasing studies have generally omitted color augmentation because its inclusion can obscure an accurate assessment of the methods' effectiveness. It is well-known that color bias in colored MNIST and corruption bias in CIFAR-10C can be partially mitigated through color augmentation. Therefore, to ensure the reliability of future research on spurious correlations, I believe it is important to avoid using color augmentation in experimental settings. Given these concerns, I strongly recommend including main results without color augmentation (a comparison between the proposed method and state-of-the-art methods would suffice, considering the limited time available) as well as ablation studies specifically addressing the impact of color augmentation. If this concern is addressed, I will raise my score. However, if not, I cannot overlook the contribution of color augmentation to the main results, and therefore, I cannot agree to accept this paper.
We appreciate your comments. As you suggested, we conducted experiments on C-MNIST, CIFAR-10C, and additionally on the real-world dataset BFFHQ, after removing color jitter. We chose PGD [1] as the comparison baseline, as it was recently proposed and showed the highest performance among the baselines. We report the average accuracies (%) and standard deviations over three trials. The results are shown in the table below. | | C-MNIST (0.5%) | C-MNIST (1%) | C-MNIST (5%) | CIFAR-10C (0.5%) | CIFAR-10C (1%) | CIFAR-10C (5%) | BFFHQ (Unbiased) | BFFHQ (Conflict) | |:----------:|:--------------:|:------------:|:------------:|:----------------:|:--------------:|:--------------:|:----------------:|:----------------:| | ERM | 60.19 (0.96) | 79.01 (0.56) | 95.23 (0.07) | 22.90 (0.76) | 25.94 (0.69) | 39.14 (0.44) | 77.80 (0.61) | 56.00 (0.35) | | PGD | 96.83 (0.13) | 98.14 (0.11) | 98.40 (0.07) | 30.01 (1.25) | 41.55 (0.51) | 52.17 (0.37) | 84.17 (1.38) | 70.20 (1.91) | | DPR (Ours) | 97.45 (0.14) | 98.28 (0.13) | 98.44 (0.21) | 31.91 (0.55) | 43.31 (1.01) | 52.92 (0.20) | 87.13 (0.87) | 76.53 (2.20) | As you can see in the table above, even without color jitter, the proposed DPR outperforms ERM and PGD on all benchmarks (i.e., C-MNIST, CIFAR-10C, and BFFHQ). Additionally, we would like to mention that we followed the experimental settings of CNC and JTT respectively for CelebA and CivilComments-WILDS, which did not use any data augmentation. Therefore, color jitter was not used on them. Considering these points, we believe that DPR clearly demonstrates its effectiveness for mitigating spurious correlations. In addition, we would like to address the impact of color jitter in the experimental settings. As you can see from the results in the table above, the experimental results for CIFAR-10C shown in the previous answer, and Tables 1 and 2 in the paper, the presence or absence of color jitter does make a difference, but the difference is somewhat smaller than expected. We believe this may be due to the low intensity of color jitter used for data augmentation. For datasets other than CelebA and CivilComments-WILDS, we followed the experimental settings of Ahn et al. [1], which include various data augmentations including color jitter. The code for the color jitter used in the experiments is as follows, and you can find this in the code submitted with the paper as supplementary material. torchvision.transforms.ColorJitter(hue=0.05, saturation = 0.05) As shown in the code above, only hue and saturation were set, both at 0.05, and this was used for all experiments on C-MNIST, MB-MNIST, BAR, and BFFHQ. We believe that because a fairly low value of 0.05 was used for color jitter, it likely did not significantly alter the color characteristics of the data images. Furthermore, we think that even without color jitter, other data augmentations such as random rotation and random resize crop were sufficient to increase the diversity of bias-conflicting samples, thus achieving high debiasing performance. [1] Mitigating dataset bias by using per-sample gradient, ICLR 2023
I sincerely appreciate the additional experiments conducted in a short time, which have addressed many of my concerns. As a result, I will raise my score. Also, I strongly recommend that the issues discussed during the discussion period, particularly those regarding color augmentation, be incorporated into the revision to guide future research on spurious correlations in a more rigorous direction
Thank you for taking the time to read our responses and provide numerous comments on them. As you mentioned, we intend to incorporate the issues discussed during the discussion period into the revised version. We are very grateful for your decision to raise the score.
Thank you for providing a detailed response. Most of my concerns are resolved except for the hyperparameter selection. In A3, I hope to confirm whether the authors used the best validation accuracy on the BFFHQ, where the validation set is highly biased, for hyperparameter selection. Given that the validation set is biased, relying on its accuracy for hyperparameter tuning might not yield reliable results.
Thank you for your comments, and we would like to provide a response to your question. Yes, as you mentioned, we tuned the hyperparameters and reported the performance using a highly biased validation set. The reasoning behind this approach is as follows: 1. about the use of bias-aligned validation set **First, the results of other baselines reported for BFFHQ were taken from Ahn et al. [1]. To ensure a fair comparison, we conducted our experiments on BFFHQ following the experimental settings of Ahn et al. [1]. These settings include the dataset split.** Therefore, we used the same dataset split for our experiments and reported the performance of the proposed DPR using a bias-aligned validation set (composed only of bias-aligned samples). 2. about the reliability of the performance of DPR Even though we selected hyperparameters and reported performance using a bias-aligned validation set, DPR outperforms other baselines. We believe that the reason for these results is that **DPR is a debiasing method derived from a newly suggested training objective that aims to improve performance for both bias-aligned and bias-conflicting groups while reducing the performance gap between them.** This explains why DPR outperforms other baselines, even when performance is reported using a bias-aligned validation set. While the performance of DPR could improve further if evaluated using an unbiased validation set or a bias-conflicting validation set with the same distribution as the test set, we used the same dataset split as Ahn et al. [1] for a fair comparison. [1] Mitigating dataset bias by using per-sample gradient, ICLR 2023
Thank you for the detailed response. All of my concerns are resolved. I raise my score to weak accept.
Thank you for taking the time to read our responses and provide comments on them. We are pleased that all concerns have been resolved. Additionally, we are extremely grateful for your decision to raise the score.
Decision
Accept (poster)