Trained Models Tell Us How to Make Them Robust to Spurious Correlation without Group Annotation
Classifiers trained with Empirical Risk Minimization (ERM) tend to rely on attributes that have high spurious correlation with the target. This can degrade the performance on underrepresented (or 'minority') groups that lack these attributes, posing significant challenges for both out-of-distribution generalization and fairness objectives. Many studies aim to enhance robustness to spurious correlation, but they sometimes depend on group annotations for training. Additionally, a common limitation in previous research is the reliance on group-annotated validation datasets for model selection. This constrains their applicability in situations where the nature of the spurious correlation is not known, or when group labels for certain spurious attributes are not available. To enhance model robustness with minimal group annotation assumptions, we propose Environment-based Validation and Loss-based Sampling (EVaLS). It uses the losses from an ERM-trained model to construct a balanced dataset of high-loss and low-loss samples, mitigating group imbalance in data. This significantly enhances robustness to group shifts when equipped with a simple post-training last layer retraining. By using environment inference methods to create diverse environments with correlation shifts, EVaLS can potentially eliminate the need for group annotation in validation data. In this context, the worst environment accuracy acts as a reliable surrogate throughout the retraining process for tuning hyperparameters and finding a model that performs well across diverse group shifts. EVaLS effectively achieves group robustness, showing that group annotation is not necessary even for validation. It is a fast, straightforward, and effective approach that reaches near-optimal worst group accuracy without needing group annotations, marking a new chapter in the robustness of trained models against spurious correlation.
Paper
Similar papers
Peer review
Summary
This paper addresses the problem of subpopulation generalization, also known as spurious correlations. Building on the Last Layer Retraining (DFR) method, it removes the constraints on a small subset of annotations. The paper introduces the Environment-based Validation and Loss-based Sampling (EVaLS) method. Unlike DFR, EVaLS divides the validation set $D^{val}$ into two parts: (1) $D^{LL}$, where losses from an ERM-trained model are used as a proxy for identifying minority groups for retraining, and (2) $D^{MS}$, where environment inference methods are used for partitioning environments. The paper presents theoretical insights and empirical results demonstrating the effectiveness of EVaLS.
Strengths
* The paper is well-structured and presented in a clear and organized manner, making it easy to comprehend and follow along. * The proposed method is simple but effective and explores a relatively challenging area in existing literature (*i.e.* subgroup generalization without group annotations). * The authors provide some theoretical analysis to support their claims.
Weaknesses
* The novelty and contribution of the proposed method may be limited for the following reasons: 1) The paper combines multiple previously proposed methods (*i.e.* DFR [1], EIIL [2]) all at once, which inherently guarantees a nontrivial performance; (2) The primary technical contribution, at least from my perspective, is the loss-based sampling, which has been already explored extensively in the noisy label literature and has been used as tools for pseudo-labeling. * The paper fails to discuss recently proposed methods that also require no group annotations, such as SELF [3], BAM [4], and BPA [5]. In particular, SELF is also a direct follow-up of DFR. The authors are encouraged to discuss the limitations and strengths of loss-based schemes against the class-based schemes advocated by SELF and BAM. * More analyses can be included to provide further understanding of the selected loss-based samples. For example, given a threshold, how much percent of the high-loss and low-loss samples are indeed the minority and majority samples and how does this percentage change with the threshold? [1] Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations, ICLR 2023 [2] Environment inference for invariant learning. ICML 2021 [3] Towards Last-layer Retraining for Group Robustness with Fewer Annotations. NeurIPS 2023 [4] Bias Amplification Enhances Minority Performance. TMLR 2024 [5] Unsupervised learning of debiased representations with pseudo-attributes. CVPR 2022
Questions
* Is EVaLS sensitive to hyperparameters? * [minor] There seem to be typos in your corollary C.1. * Can the authors make clarifications on how the conditions in proposition 3.1 are met or relaxed in practice? How does the distribution of actual experimental benchmark datasets compare to your assumptions?
Rating
5
Confidence
4
Soundness
2
Presentation
3
Contribution
2
Limitations
Aforementioned in Weaknesses and Questions.
Response to Reviewer 7E4z
We deeply appreciate your consideration of our rebuttal. Thank you for suggesting the inclusion of the discussions in the final version of the paper; we will certainly do so. We have carefully addressed all your concerns based on your feedback. If there are any specific areas where you feel further clarification or justification is needed, we are eager to address them promptly. Your guidance helps us achieve the highest standards, and we hope this will be reflected in a higher score.
Summary
To address the issue of spurious correlations when group labels are unavailable, this paper proposes a new method called EVaLS. It first creates a balanced training dataset using loss-based sampling. Then, it evaluates the accuracy of the balanced training set based on the inferred environments from the validation set, and selects models accordingly.
Strengths
1. The paper is well-written, and includes a rich set of experiments with necessary theoretical explanations. 2. It is essential to discuss the multiple (unknown) spurious features case which has been overlooked in previous studies.
Weaknesses
1. Why is the approach of using high-loss points (considered as the minority group) more effective than directly using misclassified points (considered as the minority group) in methods like JTT? Intuitively, compared to misclassified points, high-loss points are more "implicit" and no obvious thresholds, which could potentially result in high-loss points actually belonging to the majority group, thus exacerbating the imbalance in resampling. 2. If the author can show the balance level of samples obtained through loss-based sampling compared to directly using labels (misclassified points), it could further illustrate the advantages of loss-based sampling. 3. In Section "Mitigating Multiple Shortcut Attributes", if color is treated as a known spurious attribute and shape as an unknown spurious attribute, how would the performance of EVaLS be affected? Based on my understanding, there is a possibility that simplicity bias could cause the model to prioritize learning the simpler feature, color, and struggle to learn the more complex shape attribute. Therefore, considering color as known and shape as unknown can better show the performance of EVaLS in handling complex spurious features.
Questions
See weaknesses.
Rating
6
Confidence
3
Soundness
2
Presentation
3
Contribution
2
Limitations
See weaknesses.
Summary
The paper studies how to improve the model’s robustness to multiple spurious correlations when the group labels (indicator for spurious correlation) are unknown in general. The proposed approach, EVaLS, leverages the loss from a base ERM model to sample a balanced subset to prevent learning from spurious correlations. In addition, a new synthetic dataset (Dominoes-CMF) for multiple spurious attributes is crafted. Empirically, the proposed approach sometimes has advantages over the rest of the baselines when using the same amount of additional information (group label).
Strengths
1. The main paper is generally well-written. 2. The theoretical analysis in Section 3.3 (with derivations and proofs in Appendix) shows that for one-dimensional Gaussian distributions, choosing the tails on the two sides of the distributions creates balanced groups, even though the original data distribution is skewed. 3. Environment inference technique is demonstrated to be useful for separating the dataset into groups with different distributions of the subpopulations and then for model selection. 4. The proposed technique only requires last-layer retraining on part of the validation set, which is generally more efficient.
Weaknesses
1. Figure 2 attempts to illustrate more minority samples have high loss while the majority samples have low loss. However, in each of the plots, only the % of one of the minority or majority groups is shown. The illustration can be improved by showing the % of both majority and minority groups in the same plot, and showing the actual distribution of the loss for the groups. 2. Though the idea is straightforward, it is unclear how the loss-based instance sampling is actually implemented. It is helpful to provide an algorithm or pseudocode to improve the presentation. 3. The theoretical analysis is generally sound but limited to a case without discussing the use of the loss (which may not be Gaussian) and the spurious correlations (which involve at least two dimensions of core and spurious features [1]). 4. The experimental results are less polished and sometimes the advantages are not so clear over other baselines. Some results are missing for datasets such as UrbanCars and MultiNLI. Only a few baselines are compared for the new dataset in Table 2. There is also no convincing and fine-grained analysis (e.g., ablation study) to understand how the proposed approach ensures data balancing and improves group robustness. 5. The paper initially focuses on improving group robustness when multiple spurious correlations are present, but the experimental results are lacking for these more challenging datasets. [1] Sagawa, Shiori, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. "An investigation of why overparameterization exacerbates spurious correlations." In *International Conference on Machine Learning*, pp. 8346-8356. PMLR, 2020.
Questions
1. Since it’s an essential prerequisite for loss-based sampling, how well are the majority/minority groups separated in the logit space? 2. How is the number of balancing samples $k$ chosen? How balanced are the samples when they are selected from the optimal $k$? 3. I am curious about why the result for Civilcomment dataset is so much higher than the other baselines. Is the evaluation consistent with the baseline methods? 4. Minor typo: line 233 should be without loss of generality (w.l.o.g.). 5. Other minor issues with the experiments: The best-performing results in each category of approaches should be bold. The column header “best” should be “average” in Tables 4 and 5.
Rating
6
Confidence
4
Soundness
3
Presentation
2
Contribution
2
Limitations
The authors have discussed the limitations in Section 5.
Dear Reviewer RE7s, We’re glad our clarification was helpful. It’s good to know that the pseudo-code and highlights have been effective in preventing confusion. We appreciate your suggestion to include them in the final version and will incorporate them. ### Regarding the limitations that you have stated As you mentioned, we state in the Discussion section (L356-357) that environment-based validation is limited to spurious correlation and not other types of subpopulation shifts. ### Regarding SELF [2] Please refer to the response to Reviewer 7E4z. As we described there and also in Appendix A-Related Work (L547-554), reference [2] in the paper proposes two methods: *Class-balanced (CB) last layer retraining*: See L552-554. As we discuss comprehensively in the response to Reviewer 7E4z, this method is not helpful in targeting robustness to spurious correlation, but *it improves robustness to class-imbalance, another type of subpopulation shift which is independent of spurious correlation*. For details, refer to the formulations in the response to Reviewer 7E4z. *Early-stop (ES) disagreement SELF*: As noted in the response to Reviewer 7E4z, ES disagreement requires group annotations for model selection, but not for retraining (L547-551). It also needs an early-stopped version of the trained ERM and the final model. In the following table, we restate the worst group accuracy for our methods and those of SELF on spurious correlation benchmarks. We also add the results of CB last-layer retraining and ES disagreement SELF for Dominoes-CMF. |Dataset|Group Info (Train/Val)|Waterbirds|CelebA|UrbanCars|Dominoes-CMF| |-|-|-|-|-|-| |ES disagreement (SELF)|$\text{x}/\checkmark$|$93.0_{\pm0.3}$|$83.9_{\pm0.9}$|$82.1_{\pm1.8}$|$60.5_{\pm1.3}$| |EVaLS-GL (ours)|$\text{x}/\checkmark$|$89.4_{\pm0.3}$|$84.6_{\pm1.6}$|$82.3_{\pm1.2}$|$63.6_{\pm1.3}$| |CB last-layer retraining (SELF)|$\text{x}/\text{x}$|$92.6_{\pm0.8}$|$73.7_{\pm2.8}$|$21.9_{\pm13.0}$|$53.3_{\pm5.0}$| |EVaLS (ours)|$\text{x}/\text{x}$|$88.4_{\pm3.1}$|$85.3_{\pm0.4}$|$82.1_{\pm0.9}$|$67.1_{\pm4.2}$| |ERM|$\text{x}/\text{x}$|$66.4_{\pm2.3}$|$47.4_{\pm2.3}$|$18.67_{\pm2.0}$|$50.6_{\pm1.0}$| *Waterbirds*: SELF’s methods outperform EVaLS. As SELF illustrates in their Sec. 3 (Preliminaries), since CB last-layer retraining is performed on validation data, which is group-balanced for Waterbirds, CB last-layer retraining is also group-balanced for Waterbirds. When the dataset for is not group-balanced, CB last-layer retraining achieves a worst group accuracy of $77.4_{\pm0.3}$ (Table 7 in reference [2] of the paper (SELF)). *CelebA*: As you can see, ***EVaLS and EVaLS-GL outperform both CB last-layer retraining and ES disagreement***. We believe there may have been an oversight in your response, as it incorrectly suggests their method outperforms ours. CB last-layer retraining underperforms all other benchmarked methods in Table 1. *UrbanCars*: EVaLS-GL outperforms all other methods. Both EVaLS and ES disagreement (with higher group supervision) show similar worst group accuracy. CB last-layer retraining shows no significant improvement over ERM. This is expected, as UrbanCars’ training data is class-balanced, unlike Waterbirds and CelebA. Therefore, further class-balancing for CB last-layer retraining makes no difference. *Dominoes-CMF*: EVaLS outperforms other methods. EVaLS-GL also outperforms methods in SELF [2]. CB last-layer retraining does not show any significant improvement compared to ERM. This is expected, as Dominoes-CMF also has class-balanced training data like UrbanCars. #### Conclusion: CB last-layer retraining targets class-imbalance shifts and is ineffective for spurious correlation. Except for the Waterbirds dataset, our methods ٍ(EVaLS and EVaLS-GL) outperform those of SELF [2] with similar group supervision on all other datasets. EVaLS shows comparable or better results (***e.g., in CelebA***) compared to methods with higher group supervision. See Table 1 in the paper for more comparisons.
Evaluation on CivilComments Dataset
Previously in Q3, I asked about why the performance of the proposed approach is much higher for the CivilComments dataset. The author responded that "The CivilComments dataset has class imbalance, so fine-tuning/retraining on class-balanced data can significantly improve group robustness." However, as I looked closely into the details, that might not be the case. In SELF[2], the worst-group accuracy is calculated over **4** groups by aggregating all identity categories into one spurious feature. In other benchmarks such as JTT, DFR, and AFR, the worst-group accuracy is calculated over **16** groups (8 attributes and 2 labels). Therefore, the result of EVaLS-GL in CivilComments column might not be directly comparable to each other except with SELF (added later) and the ERM/DFR implemented by SELF. Please look into this carefully and correct/rerun the misleading results if necessary. Relevant public GitHub repos: https://github.com/tmlabonte/last-layer-retraining https://github.com/AndPotap/afr
Dear Reviewer RE7s, We deeply appreciate your notice. Our setting follows the practice in the GitHub repository `izmailovpavel/spurious_feature_learning` in `datasets.py`, which is referenced in the official implementation of DFR as an extension of their code functionality. Both (line 266 in their code and line 129 in `data/civilcomments.py` in our code) collapse grouping as “Has Identities”/”No Identities” in each class (4 groups in total). We realize that all the methods that report results around ~79-80% worst group accuracy on CivilComments (like the class-balanced schemes suggested by Reviewer 7E4z and previously reported results for EVaLS-GL) use the 4-group setting of CivilComments, while the methods (as you mentioned) that report on the 16-group setting, result in around ~69-70% worst group accuracies. We carefully considered your notice and reran the experiment of EVaLS-GL on the CivilComments dataset with 16 groups. **EVaLS-GL on CivilComments with 16 groups has worst group accuracy and average accuracy (in parentheses) of $\boldsymbol{68.0_{\pm0.5}(89.2_{\pm0.3})}$%**. Evaluating ERM on worst accuracy for 16 groups also resulted in $56.3_{\pm4.8}$ (average accuracy does not differ with the 4 group setting). We should restate that, as mentioned by previous works (reference [4]-Table 2) and as seen in the following table, CivilComments is not an instance of a dataset with spurious correlation (L313). In other words, no identity attribute in CivilComments is predictive for any of the labels (see L316-318). However, loss-based sampling with ground-truth group annotations (EVaLS-GL) still significantly improves worst group accuracy compared to initial ERM. Proportion of attributes in each class for CivilComments dataset: |Toxicity (Class)|Male|Female|LGBTQ|Christian|Muslim|Other Religions|Black|White| |-|-|-|-|-|-|-|-|-| |0|0.11|0.12|0.03|0.10|0.05|0.02|0.03|0.05| |1|0.14|0.15|0.08|0.08|0.10|0.03|0.1|0.14| ### Conclusion EVaLS-GL results in $68.0_{\pm0.5}$% worst group accuracy when CivilComments is examined with 16 groups (identity attributes in the table above in each class), and $80.5_{\pm0.4}$% when it is examined with 4 groups (“Has Identities”/”No Identities” attributes in each class). We acknowledge the discrepancy in the reported results of EVaLS-GL on the CivilComments dataset and appreciate your feedback. We hope the above explanations provide clarity for a thorough comparison. We will incorporate this discussion regarding the EVaLS-GL results on CivilComments into the final version of the paper. Finally, we again greatly appreciate your attention to this matter and your notice regarding this. --- Below, we present the updated main table (Table 1) of the paper, including the results you requested to be added to it in this rebuttal thread for a more comprehensive comparison. If a method’s worst group accuracy is better than others with a similar or weaker level of assumption about the availability of group annotations during training/validation phases, it is underlined. Worst group accuracies are bolded if a method outperforms all other methods. SC stands for spurious correlation datasets. Results for CivilComments are calculated for the 16-group setting (∗ denotes results for the 4-group setting). Other notations are similar to the main table. We would like to remind you that environment-based validation only works for spurious correlation subpopulation shift (L316-324). |Method|Group Info (Train/Val)|Waterbirds (SC)|CelebA (SC)|UrbanCars (SC)|CivilComments|MultiNLI| |-|-|-|-|-|-|-| |GDRO|$\checkmark/\checkmark$|$91.4$|$\underline{\boldsymbol{88.9}}$|$73.1_{\pm 2.0}$|$69.9$|$\underline{\boldsymbol{77.7}}$| |DFR|$\text{x}/\checkmark\checkmark$|$92.9_{\pm0.2}$|$88.3_{\pm1.1}$|$79.6_{\pm2.2}$|$\underline{\boldsymbol{70.1_{\pm0.8}}}$|$74.7_{\pm0.7}$| —------------------------------- |GDRO+EIIL|$\text{x}/\checkmark$|$77.2_{\pm1.0}$|$81.7_{\pm0.8}$|$76.5_{\pm2.6}$|$67.0_{\pm2.4}$|$61.2_{\pm0.5}$| |ES disagreement SELF|$\text{x}/\checkmark$|$\underline{\boldsymbol{93.0_{\pm0.3}}}$|$83.9_{\pm0.9}$|$82.1_{\pm1.8}$|$79.1_{\pm2.1}^*$|$70.7_{\pm2.5}$| |JTT|$\text{x}/\checkmark$|$86.7$|$81.1$|$79.5_{\pm 5.1}$|$\underline{69.3}$|$72.6$| |AFR|$\text{x}/\checkmark$|$90.4_{\pm1.1}$|$82.0_{\pm0.5}$|$80.2_{\pm2.0}$|$68.7_{\pm0.6}$|$73.4_{\pm0.6}$| |EVaLS-GL (Ours)|$\text{x}/\checkmark$|$89.4_{\pm0.3}$|$84.6_{\pm1.6}$|$\underline{\boldsymbol{82.3_{\pm1.2}}}$|$68.0_{\pm0.5}$/$80.5_{\pm0.4}^*$|$\underline{75.1_{\pm1.2}}$| —------------------------------- |EVaLS (Ours)|$\text{x}/\text{x}$|$88.4_{\pm3.1}$|$\underline{85.3_{\pm0.4}}$|$\underline{82.1_{\pm1.0}}$|Not SC|Not SC| |Class-balanced last-layer retraining|$\text{x}/\text{x}$|$\underline{92.6_{\pm0.8}}$|$73.7_{\pm2.8}$|$21.9_{\pm13.0}$|$80.4_{\pm0.8}^*$|$64.7_{\pm1.1}$| |ERM|$\text{x}/\text{x}$|$66.4_{\pm2.3}$|$47.4_{\pm2.3}$|$18.7_{\pm2.0}$|$56.3_{\pm4.8}$|$64.8_{\pm1.9}$|
Reviewer Response
Thank you for promptly checking and running the results for CivilComments in the correct setting. Please make sure to present a clear and fair comparison with detailed experimental setup in the revised version to avoid potential confusion and unfairness. With no other major issues left, I have increased my score accordingly.
Dear Reviewer RE7s, Thank you for your constructive review. We’re pleased that our rebuttal successfully addressed your concerns. We ensure that the final version addresses these points to avoid any potential confusion or unfairness. Best regards, Authors of Submission 19478
Thanks for your response
Thank you for your detailed response. I think it generally addresses my concerns so I will increase my score. I would encourage the authors to include these additional discussions in the final version of the paper.
Reviewer Response
Thanks for providing the detailed responses including experimental results, pseudo code, useful statistics, etc. W3: To clarify, what I meant was that the proposed approach utilizes “**loss**-based sampling” (class-dependent), but the analysis in proposition 3.1 is based on the **logits** (class-independent). It’s perfectly fine to assume Gaussian distribution in the features space and hence logit space, but when the loss is involved, it becomes a bit more complicated. This is because now we have different classes even for logistic regression. The logistic loss is monotonic for each class but when the label flips it’s no longer the same function. As assumed there are two Gaussian distributions for the **logits**, for binary classification, there will be four resulting distributions on the **losses**, because there are two transformations $-log(\sigma(L))$ and $-log(1-\sigma(L))$. Does the analysis for logits still hold in this scenario with losses? Furthermore, I referenced [1] because they considered two feature dimensions (core and spurious), the majority/minority groups are clearly defined as spuriously-correlated/non-spuriously-correlated w.r.t. the features and labels. When you reduce the analytical setup to majority and minority distributions, the relationship with spurious correlation becomes blurred. That’s why I mentioned it as a limitation. W4: Thanks for filling in the missing results. Please also include the few baselines pointed out by reviewer 7E4z to make the empirical comparison more comprehensive, but just to clarify why CivilComments and MNLI are out of the scope of EVaLS?
Response to Reviewer RE7s
Thank you for considering our rebuttal and paying special attention to the theoretical aspects of our work. We have tried to answer all your concerns in our response and hope you find them helpful. Please let us know if there is anything that needs to be clarified, as we find these discussions very helpful. **W3**: We would like to thank you for clarifying your arguments. We think we now understand better what you meant. We should emphasize that **both** the loss-based sampling and theoretical analysis are **class**-dependent. We state that our theoretical arguments are class-dependent in L215-218, and also L584-587 in Appendix C. Consider the setting we have assumed for the classification task and the loss function (L214 in Sec. 3.3, L583 in Appendix C). As you have stated, in the case of using logistic loss (sigmoid function), loss values are calculated as $-log(\sigma(L))$ and $-log(1-\sigma(L))$ (each for one of the classes). Therefore, the order of samples in logit and loss spaces is monotonic (L215, L584) in **each class**. The only difference between the two classes is that the direction of the monotonicity is different for them. But as our analysis are **class**-dependent, there is no worry about applying two different transformations, as the transformations themselves are **class**-dependent. Please note that our theoretical analysis aims to show that loss-based sampling can create a balanced dataset. Focusing on logit space is sufficient to demonstrate sampling thresholds. Consequently, as seen in Proposition 3.1 (and C.1), we assume that we have $1-\epsilon$ spurious correlation, and minority (proportion $\epsilon$) and majority (proportion $1-\epsilon$) samples in each class are drawn from different Gaussian distributions in the feature and hence logit spaces (as you have noted). Additional assumptions about the feature space structure (like in the paper [1] you mentioned) are more restrictive. We do not fully comprehend how assuming a more structured feature space is helpful to further develop the theory as the current assumptions are less restrictive. **W4**: Thank you. As shown in our response to reviewer 7E4z, while the mentioned methods target different subpopulation shifts or have higher supervision, our method outperforms them in almost all experiments (the only exception is Waterbirds, where some have higher WGA but lower average accuracy). We will include the new baseline results in the final paper. As stated in Sec. 2.1 (L112-L123), There exist multiple types of subpopulation shifts (see response to reviewer 7E4z for class-imbalance). Refer to reference [4] in the paper for a categorization of subpopulation shifts and their formulation (Table 1 in their paper). Our focus in designing environment-based validation (EV) is on spurious correlation. The types of subpopulation shifts in CivilComments and MultiNLI are class-imbalance and attribute-imbalance, respectively (see the definition of these datasets in Appendix D.3, L714-725). Regarding your question, please refer to L316-324. As explained there, patterns distinguishing groups in CivilComments and MultiNLI are not predictive for the target class (see tables below). This reduces their visibility in the model’s final layers (reference [29] in the paper discusses the role of various layers of neural networks in different types of shifts). Since environment inference algorithms like EIIL (see Appendix E.2 for further investigation) depend on the last layers of a trained model, they cannot infer environments with notable group shifts (defined in L195-198) in CivilComments and MultiNLI. The group shifts in CivilComments and MultiNLI (L323-324) are significantly lower than those of the datasets that are reported in Table 3. Thus, the focus of environment-based validation is on datasets with spurious correlation. Nevertheless, loss-based sampling (LS) is effective for CivilComments and MultiNLI. Our EVaLS-GL, using ground-truth group labels for model selection and loss-based sampling for retraining, outperforms all other methods on CivilComments and those with similar group supervision on MultiNLI. Only GDRO, with full group annotations during training, performs better on MultiNLI. **CivilComments and MultiNLI training sets statistics** CivilComments: |Group |Class|Attribute |# Train Data| |-|:-:|:-:|:-:| |$G_1$|0|No Identities|148186 (55%)| |$G_2$|0|Has Identities|90337 (33%)| |$G_3$|1|No Identities|12731 (5%)| |$G_4$|1|Has Identities|17784 (7%)| MultiNLI: |Group|Class|Attribute|# Train Data| |-|:-:|:-:|:-:| |$G_1$|0|No Negations|57498 (28%)| |$G_2$|0|Has Negations|11158 (5%)| |$G_3$|1|No Negations|67376 (32%)| |$G_4$|1|Has Negations|1521 (1%)| |$G_5$|2|No Negations|66630 (32%)| |$G_6$|2|Has Negations|1992 (1%)|
Reviewer Response
Thanks for the explanation! I think we should clarify whether your loss-base sampling approach is class-wise or not. It seems to be so from our discussion, but your pseudo-code at the top in the general rebuttal did not consider the class, and I could not verify you are selecting $|\mathcal{Y}|*2k$ samples in total from the main paper.
Clarification Regarding the Algorithm
Dear reviewer RE7s, In the general author rebuttal, we aimed to keep the pseudocode minimal and straightforward to highlight the main components of the algorithm and framework. We greatly appreciate your attention to detail in noticing that the pseudocode in the general author rebuttal did not demonstrate the class-dependency of the algorithm. Details regarding this matter have been added to the pseudo-code below. We have clarified in the paper that loss-based sampling occurs within each class. In Section 3.1, where we introduce loss-based sampling, we clearly state: *“By combining these **$2k$ samples from each class**, we construct a balanced set $D^{balanced}$”, consisting of high-loss and low-loss samples (see Figure 1(c))* (L171-172). This matches what you state in the response. Moreover, in the caption of the main figure, Figure 1(c) in the paper, it is stated: *“We evaluate train split samples on the initial ERM classifier and sort high-loss and low-loss samples of **each class** for loss-based sampling.”* Additionally, in Figures 1(c) and 1(d) in the paper, you can see that in each class, samples with the highest and lowest loss are selected. It is worth mentioning that our environment-based validation also applies to each class separately, as stated in Sec. 3.2 Partitioning Validation Set into Environments (L194-195): *“Subsequently, each environment is further divided based on sample labels, resulting in $2\times|y|$ environments.”* You can also observe that all our reported group shifts in Table 3 are class-dependent, and other environment inference methods used are class-dependent as well (L201-203). We sincerely apologize for any confusion in this matter. Please let us know if anything further needs to be clarified. ------------- Class-dependency details for loss-based sampling **become** ***bold*** in the pseudo-code below. **Input:** Held-out dataset `D`, ERM-trained model `f_ERM`, maximum `k` value `maxK` **Output:** Optimal number of samples `k*`, Best model `f*` 1. Split the held-out dataset `D` into train and validation: - $D_{MS}$, $D_{LL}$ = `splitDataset(D)` 2. Infer environments from the validation split: - `envs = inferEnvs(D_MS)` 3. ***For each class $y\in\mathcal{Y}$*** - ***Sort $D_{LL}$ samples from class $y$ by their loss:*** - ***`sortedSamples[y] = sortByLoss(f_ERM, `$D_{LL}^y$`)`*** 4. Initialize `wea* = 0, k*= 0, f*= None` 5. For `k` from 1 to `maxK`: - ***For each class $y\in\mathcal{Y}$:*** - Select top-`k` high-loss samples: - ***`highLossSamples[y] = sortedSamples[y][:k]`*** - Select top-`k` low-loss samples: - ***`lowLossSamples[y] = sortedSamples[y][-k:]`*** - Combine samples: - ***`selectedSamples = {highLossSamples[y], lowLossSamples[y]} for all `$y\in\mathcal{Y}$*** - Retrain the last layer: - `f = retrainLastLayer(selectedSamples)` - Evaluate the retrained model: - `wea = evaluateWEA(f, envs)` - If `wea > wea*`: - `wea* = wea` - `f* = f` - `k* = k` 6. return `f*`
Discussion Summary
Thank you for the detailed clarification on having a class-wise loss-based sampling strategy. In this case, I agree that the theoretical analysis in proposition 3.1 aligns and supports the proposed approach very well. I also apologize for not realizing the sampling strategy applies to each class, and that's why I asked for pseudocode for clarification at first. I believe the paper can be improved after including the pseudocode and more highlights to avoid confusion, which the authors have already done well in the rebuttal. The major limitation still lies in the experimental results. The proposed algorithm EVaLS is only somewhat better than other baselines, and the results are quite arguable when compared to SELF[2], since SELF is more flexible and outperforms EVaLS-GL on Waterbirds and CelebA. When using EIIL for validation split instead of using the ground-truth group labels, it's also limited to the case where the spurious correlations are strong and not applicable to the two NLP datasets. The authors have correctly identified and discussed these limitations. After checking the comments from other reviewers, there are no other obvious major concerns. I have increased my score accordingly.
Dear author, Thank you for your reply and the additional experimental results. After reading the reply and the opinions of other reviewers, I maintain my score. Although I agree with Reviewer 7E4z's point that this work is of limited novelty, I think the author's research is complete and detailed with enough experiments to show the effectiveness of EVaLS. In addition, I appreciate the discussion on multiple (unknown) spurious features. Therefore, I keep my score. Best, Reviewer SN9B
Dear Reviewer SN9B, Thank you for taking the time to read our rebuttal. We are pleased to hear that you find our research complete and detailed, with sufficient experiments demonstrating its effectiveness. We must emphasize that the title of our work reflects its content. We propose a method to derive all necessary information from the model itself to enhance its robustness against spurious correlations. Section 3, which introduces EVaLS, presents solutions such as loss-based sampling for creating a balanced dataset and partitioning the validation set into environments for model selection. We have also conducted multiple ablations in Section E to evaluate different modules while maintaining the overall framework. Thus, our work has never proposed using and combining previous methods as a novelty (as Reviewer 7E4z has stated). Our work demonstrates the following points, as stated at the beginning of the general author rebuttal: - In contrast to what is considered a requirement for previous methods (see L528-549 in Appendix A-Related Work), we show that ideal group discovery is not required for model selection in robustness to spurious correlations. Instead, identifying environments with group shifts (Sec. 3.2) works effectively (see Table 1). - Relying on what a trained model learns, rather than auxiliary (even ground-truth) information (as in methods like AFR, DFR, etc.), could be more effective in the case of unknown spurious correlations (see Table 2 and Table 1-UrbanCars). - Loss-based sampling (Sec. 3.1) is not only an effective method for robustness to group shifts (compared to other methods such as loss-based weighting schemes like AFR or upweighting misclassified samples like JTT; see EVaLS and EVaLS-GL in Table 1), but it is also supported by a theory for data balancing with general assumptions (Sec. 3.3 - Theoretical Analysis). Best regards, Authors of Submission 19478
Decision
Reject