Group Robust Classification Without Any Group Information

Empirical risk minimization (ERM) is sensitive to spurious correlations in the training data, which poses a significant risk when deploying systems trained under this paradigm in high-stake applications. While the existing literature focuses on maximizing group-balanced or worst-group accuracy, estimating these accuracies is hindered by costly bias annotations. This study contends that current bias-unsupervised approaches to group robustness continue to rely on group information to achieve optimal performance. Firstly, these methods implicitly assume that all group combinations are represented during training. To illustrate this, we introduce a systematic generalization task on the MPI3D dataset and discover that current algorithms fail to improve the ERM baseline when combinations of observed attribute values are missing. Secondly, bias labels are still crucial for effective model selection, restricting the practicality of these methods in real-world scenarios. To address these limitations, we propose a revised methodology for training and validating debiased models in an entirely bias-unsupervised manner. We achieve this by employing pretrained self-supervised models to reliably extract bias information, which enables the integration of a logit adjustment training loss with our validation criterion. Our empirical analysis on synthetic and real-world tasks provides evidence that our approach overcomes the identified challenges and consistently enhances robust accuracy, attaining performance which is competitive with or outperforms that of state-of-the-art methods, which, conversely, rely on bias labels for validation.

Paper

Similar papers

Peer review

Reviewer Tfdv6/10 · confidence 4/52023-06-26

Summary

The paper identifies that current methods tackling spurious correlations requires group annotation in either the training or validation stage. To address this limitation, the authors propose uLA, a bias-unsupervised method that achieves superior empirical performance without any group annotation.

Strengths

The paper studies an important problem: how to tackle spurious correlations when group annotation is unavailable. The proposed algorithm is simple yet effective, demonstrating superior empirical improvement. The performance on the systematic generalization is particularly promising as it is a much more realistic scenario.

Weaknesses

Although the authors show some conceptual and empirical advantage of SSL over pure supervised training, it remains unclear how crucial a role SSL has in the training pipeline. I hope the authors can provide clearer insights about the criterion of selecting the pretraining scheme for the shared backbone model. (Also see question 3) The robustness of the algorithm to different hyperparameter settings is questionable (also see question 2). Not only does the biased prediction matter, but the biased logits are also quite important in the training pipeline. However, the logits can take dramatically different values given different hyperparameters. I hope the authors can provide more insights in this regard and perhaps include more principled method for hyperparameter search.

Questions

1. For the reported baseline, how are the models trained? Do they also go through the MOCOV2+ pretraining and linear probing as uLA? 2. On line 225, it is mentioned that the model needs to be validated at every epoch, which is uncommon in practice. Could you provide some justification for this procedure? Does it mean that the algorithm is sensitive to hyperparameters? 3. Why use an Imagenet pretrained model on waterbird? Are there any scenarios where using SSL hurt the performance? I hope the authors can provide more detail about this design choice and more discussion about the selection of the training scheme of the shared backbone.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

4 excellent

Limitations

The authors have adequately addressed the limitation.

Reviewer R4tq6/10 · confidence 3/52023-07-06

Summary

This work aims to evaluate and introduce a method to make classifiers perform well across subgroups of the data, focusing in particular on “spurious correlations” where one attribute correlates with the output label in the training set, but need not at test time. While most past work doing this requires “bias annotations”, or labels of the potential spurious attribute, some work aims to avoid spurious correlations without group info. This work reveals shortcomings in these methods; they introduce a new dataset (sMPI3D) where the goal is to classify the shape of an object, which is spuriously correlated with its color. They then introduce a new method, “bias-unsupervised logits”, which adjusts the output logits approximating the log probability of the label given the hidden attribute of x, without any access to the bias attribute. To approximate this quantity, the authors take representations trained with some self-supervised learning approach (they use MoCoV2+), then fine-tune these representations towards the target labels and use this as the prediction. The authors find that their method is comparable to existing methods that use bias annotations on Waterbirds, CelebA, cMNIST, and cCIFAR, and find that their method is the only one that reliably outperforms empirical risk minimization on sMPI3D.

Strengths

* The paper is well written, motivated, and referenced throughout. * The dataset sMPI3D seems like a good contribution; in particular, the authors do a good job explaining how the number of subgroups grows exponentially with the number of attributes you care about, that this means you can’t cover all groups in the training set reliably, and this dataset gives a great simple way to test how well models extrapolate to new groups (when the individual attributes are supported, but the direct product is not) * The authors setup---no bias annotations during both training and validation---seems realistic and important for subsequent work to follow * The results against other methods on existing benchmarks seem promising, though they’re sometimes worse than other methods.

Weaknesses

* There are places where the paper could use more exposition; for example, the current paper does not devote much time to how they extract bias variable observations, or time to how they construct the dataset (the two core contributions) * The method, while performing comparably to baselines, only offers significant improvement over previous methods on the sMPI3D, which the authors release (and thus have a lot more control over). This is somewhat ok because the task setup for sMPI3D is different (support over all individual attributes but not the direct product in training), but it would be nice to see additional experiments.

Questions

* Could you provide more intuition for why training on the actual labels is a good way to estimate p(y \mid z); in particular, what properties of the pretrained representation do you need for this to work?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

Addressed limitations.

Reviewer knxG6/10 · confidence 3/52023-07-07

Summary

The paper works on mitigating the impacts of spurious correlations during the risk minimization. The authors firstly introduce a systematic generalization task and illustrates existing methods implicitly made the assumption that all group combinations are represented within the training procedure. And then reveal that the importance of bias labels in model selection. Hence, the authors leverage pre-trained self-supervised models for bias information extraction, during the training and validating the debiased models.

Strengths

The proposed method uLA trains a linear classifier on top of an SSL pre-trained model. The predictions are hence biased, and could be made use of model debiasing. The key advantage is that uLA does not need any group labels, neither in the training nor in the validation.

Weaknesses

* (1) The presentation could be better , i.e., (a) in figure 2, it would be much better if authors could give more introduction about MPI3D. datasets; (b) Eqn (5) is a core component of uLA, it would be better if authors could explain more about the the insights of the estimate for $\hat{p}_{\text{data}}(Y, Z)$, for example, why this is a reliable estimate? Is it possible for authors to provide some empirical verification on the effectiveness of this estimation? * (2) Personally speaking, the proposed method is somehow related to a family of distributional robustness approach, that holds the viewpoint that "fine-tune last layer" [R1] or "post-hoc adjust the model prediction" [R2] can improve the distributional robustness substantially. Such methods also rely on a well-trained model, uLA can also be categorized in this setting since uLA leverages SSL pre-trained models. Thus, it would be better to either compare or discuss the connections between uLA and this family of robust solutions. * (3) Efficiency of uLA: SSL algorithms usually require longer training time and more computer resources, compared with classical training algorithms. References: [R1] Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations. [ICLR'23] [R2] Distributionally Robust Post-hoc Classifiers under Prior Shifts. [ICLR'23]

Questions

* (1) The choice of ssl algorithm: I am not asking authors to run additional experiments, just curious about how uLA behaves when different SSL algorithms are utilized, if the authors already have the result. * (2) Analysis of $\eta$: are there any analysis on the role of $\eta$ in eqn. (6), either theoretical or empirical analysis? * (3) How is the performance of uLA on waterbirds when a SSL pre-trained model is utilized?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

4 excellent

Limitations

As mentioned in weakness 3, one limitation is **Efficiency of uLA:** SSL algorithms usually require longer training time and more computer resources, compared with classical training algorithms.

Reviewer Y39n5/10 · confidence 4/52023-07-08

Summary

This paper introduces a new bias-unsupervised method for addressing spurious correlations, encompassing a debiasing training algorithm and a model selection paradigm. Specifically, the method employs a Self-Supervised Learning (SSL) pre-trained model as a bias proxy. The SSL model's fixed predictions are used to adjust the logit during the predictor's training phase. These predictions are also utilized in the model selection stage to adjust the weight of the sample, aiming to approximate group-balanced accuracy. Empirical evidence supports the method's efficacy, as demonstrated on several popular benchmarks, as well as a systematic generalization benchmark.

Strengths

- How to design a proper bias proxy is important for the bias-unsupervised methods. - The presentation of the method is clear and generally easy to follow. - A variety of benchmarks are considered in the experiments and an ablation study part is included.

Weaknesses

- Inconsistency. In the introduction part, one of the motivation of this work is to solve the failure of existing methods in systematic generalization. However, though empirically shown, it is not clearly introduced how the proposed method could achieve that improvement. - Some important related works are missing. - There are some other existing bias-unsupervised methods, including EIIL[1], PGI[2], and SCILL[3]. Similar to the method proposed in this work, all the 3 methods utilize a frozen reference model as a bias proxy. Especially, PGI is proposed for the systematic generalization task. - The logit adjustment strategy is similar to the strategies in some debiasing training methods, e.g. PoE[4] and DRiFt[5]. - A bias-unsupervised validation paradigm is also proposed in [2]. Therefore, this paper should discuss these related works and reposition its contribution. References: [1] Creager et al. "Environment inference for invariant learning." ICML, 2021. [2] Ahmed et al. "Systematic generalisation with group invariant predictions." ICML, 2021. [3] Chen et al. "When does group invariant learning survive spurious correlations?" NeurIPS, 2022. [4] Clark et al. "Don’t take the easy way out: Ensemble based methods for avoiding known dataset biases." EMNLP, 2019. [5] He He, Sheng Zha, and Haohan Wang. Unlearn dataset bias in natural language inference by fitting the residual. DeepLo workshop, 2019.

Questions

- Why the proposed method can benefit systematic generalization, theoretically or intuitively?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

Some theoretical justifications for the proposed method are lacking.

Authorsrebuttal2023-08-15

Request to participate in discussion phase

Dear reviewers and AC, We appreciate the reviewers for dedicating time and effort to thoroughly evaluate our work. We have taken care to meticulously address all of their concerns and questions, providing detailed comparisons to related work and extended empirical evidence reinforcing the claims of our work. As the discussion phase is about to end and we are still anticipating for feedback corresponding to it, we would like to kindly invite the reviewers to participate in this phase and engage in our responses. In the case of any follow-up questions, we will in turn provide with further clarifications. Kindly, The authors

Reviewer Tfdv2023-08-17

I want to thank the authors for the detailed response, which addresses most of my concern. I will keep the initial positive score.

Authorsrebuttal2023-08-18

Thank you for your response

Thank you for answering to our rebuttal and for the positive assessment of our work. To further understand the weaker and the stronger points of our paper, as well as to facilitate the decision process, we would like to know which of the concerns were addressed adequately and which were not. While it is not necessary, we would appreciate if the reviewer could specify them.

Reviewer Y39n2023-08-19

Response to the rebuttal

I appreciate the authors' efforts in addressing my concerns. However, some issues remain unresolved. Firstly, I'd like to highlight that a test set from the colored MNIST dataset as mentioned in [2] indeed exhibits systematic shifts, as evident from Figure 1 in [2]. In the test set $T_s$, the authors "colour the test set with the biasing colours, but such that no digit is coloured with its own biasing colour". This implies entirely novel combinations. Therefore, I contest the notion that [2] examines the systematic generalization of classifiers trained with group-robustness methods. Secondly, while the rebuttal acknowledges the logit adjustment methods proposed in [4] and [5], the distinctiveness of the logit adjustment method introduced in this paper remains ambiguous. Specifically, how is the new method "simpler and more generic"? Does it operate under less stringent assumptions? I look forward to clarifications for the above issues to better appreciate the authors' contributions.

Authorsrebuttal2023-08-20

Thank you for the response

We thank the reviewer for participating in the discussion phase. We proceed to address the remaining issues: ### Systematic Generalization in [2] Considering that we agree on that a systematic generalization benchmark needs to exhibit entirely novel combinations of attributes in the test set samples, we disagree that [2] examines such a case. This is already evident in Fig. 1. Specifically, Fig 1.a presents examples from the training set. Last row contains samples of digit 9, while it suggests that the biasing/dominant color for 9 is fuschia. Notice how a cyan 9 (which is not the dominant color) also appears in the training set. At the same time Fig 9.c presents samples from the systematic generalization test set. Observe that it also contains a cyan 9 in the last row. This is not a coincidence. The cited line from the paper in the comment just means that the test set contains only bias-conflicting samples; that is samples which are generated by combinations of (digit, not biasing color of that digit), like non-fuschia 9s. It does not claim, however, that such combinations are not exposed in the training set. In fact, "own biasing color" can just mean "highly correlated with the digit in the training set", as it is the case in [2] and for example in [8]. To underline this, we repeat here the training set generation procedure as it is presented in [2], and also mentioned in our rebuttal: > COLOURED MNIST: Consider an illustrative dataset with coloured MNIST digits. For the training set, Tr, MNIST digits are coloured with a set of digit-correlated “biasing” colours 80% of the time, and with ten random colours that are different from the biasing colours the remaining 20% of the time. We hope that this makes clear that **the training set of [2] is exposed to all possible combinations of digits and colors**, and hence under their training set construction there can be no possible test set containing samples from entirely novel combinations. Thus, in contrast to our work (see Fig 2.d and Appendix D.1 of our work), [2] does not actually examine a systematic generalization task, but an easier group-robustness case as it is further elaborated in our rebuttal. [8] Nam, Junhyun, et al. "Learning from failure: De-biasing classifier from biased classifier." NeurIPS. 2020. ### In relation to [4] and [5] [4] operates under the additional assumption that input data is available in pairs of (true context, bias source). In their case (VQA tasks) true context refers to an image and the biasing source is the textual question. In our work, true predictive features and bias features are not already "disentangled" like that in an input observation $x$. In that, our method is *simpler and more generic*. On the other hand, [5] does not have a mechanism which prevents the biased model from fitting exactly the biased training set. If the biased networks fits entirely the training set, consequent logit adjustment will not be effective as it no longer a proxy for $p(y|z)$, but on the training samples it is simply acts as look-up table for the training set. To prevent that, [5] depends on early-stopping hyperparameters for the biased network, which are tuned using unspecified data resources, possibly a balanced validation set. In contrast, our work tunes hyperparameters based on our proposed robust model selection criterion using iid data, as well as our pretraining approach based on SSL and linear probing yields decreased sensitivity to the number of training steps of the biased network (Left plot Fig. 2 of rebuttal material).

Reviewer Y39n2023-08-22

Response to the rebuttal

I would like to thank the authors for the response. Regarding the first point, I agree with the authors that the colored MNIST in [2] does not present wholly unique combinations of attributes in its test set samples. The crux, however, is to ascertain if the methodology from [2] can be adapted for systematic generalization scenarios. If the approach from [2] falls short in this aspect, whereas the new method presented in this paper excels, it would convince the novelty of the proposed method and its significance for systematic generalization. This distinction, however, remains ambiguous in the current discussion. For the second point, I understand that the design of the bias proxy in this paper, coupled with the model selection approach, undeniably simplifies the process and offers a more generalized solution. My primary concern pertains to the novelty of the logit adjustment step, specifically the one induced by equation (4) in this paper. It's essential to elucidate the theoretical disparities between this equation and those proposed in other works. In summation, I believe the paper can be further refined. I would like to encourage the authors to address these concerns to amplify the potential impact and utility of their work for the broader community.

Authorsrebuttal2023-08-22

Discussing further

We would like to thank the reviewer for maintaining engagement with us during the discussion phase. ### First point ***We are glad to find that the reviewer agrees with us on the original contested point: [2] does not examine a systematic generalization task***, in which the test set consists of samples generated from entirely novel combinations of attributes. We would have been happy to provide with the extra quantitative experiments, if that was the original request of the reviewer at the beginning of rebuttal period when we could have the time to respond. Nonetheless, current evidence in our paper suggests that methods which depend on reweighting per-sample losses, like the one presented at [2], cannot deal with systematic generalization tasks. The reason is that rare examples corresponding to the novel combinations cannot be found during training, thus these methods will not find useful sanples to reweight. For example, as we present in Table 3, reweighting methods such as GroupDRO and Just-train-Twice perform subpar in the sMPI3D task compared to our method, and in particular they perform close to the percentage of combinations found in the training set (out of the total possible ones). ### Second point ***We would like to thank the reviewer for highlighting that our method "undeniably simplifies the process and offers a more generalized solution."*** Furthemore, we would like to remind that, while [4] and [5] do not offer with any theoretical justification, our work is deriving the general form of logit adjustment for the spurious correlation problem from first principles, as we have already argued in the original rebuttal response (Proposition 2.1 and Appendix A). Finally, we would like to remind that the logit adjustment paradigm itself originates from the long-tail data learning literature [9], as we present in the first sentence of the Logit Adjustment paragraph in Sec. 2 , and our work effectively adapts it to the spurious correlation problem while providing with theoretical justifications. **In any case, our work does not claim novelty of the logit adjustment technique. What our work proposes is a training+validation methodology for group-robust classifiers without any group information**. For that we offer a favorable alternative (which uses logit adjustment in a well-justified manner) to methods which implicit rely on meta-information about existence of all groups during training (systematic gen. tasks) or explicitly to ood resources for successful model selection. [9] Menon, Aditya Krishna, et al. "Long-tail learning via logit adjustment." arXiv preprint arXiv:2007.07314 (2020). ### Final note During the rebuttal period, we believe that we have sufficiently addressed all points raised by the reviewer, as they have also reached agreement on those points. We will update the paper to reflect and clarify the questions raised by the reviewer and we will also discuss the extra related work, as we have meticulously done in our rebuttal.

Reviewer knxG2023-08-21

Thanks authors for the detailed responses, which have addressed most of my concerns. After reading the author's rebuttal and the other reviewers' comments, I would like to keep my score (6: Weak Accept).

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC