Spuriosity Didn't Kill the Classifier: Using Invariant Predictions to Harness Spurious Features

Summary

The authors present a method for training models that are robust to out-of-distribution data by focusing on stable features. Stable features are those whose conditional relationship with the label do not change across domains. The benefit of their approach is demonstrated 1 synthetic dataset and 3 "real-world" datasets.

Strengths

Spurious correlations is a significant problem in many applications, which erodes the power of predictive models and shortens their usable life. The method proposed by the authors helps to mitigate these issues. The paper is well written and easy to follow. The proposed algorithm appears easy to use and computationally scalable.

Weaknesses

This work is closely related to Puli et al. (2021). I think the authors should consider comparing against NURD as well. Puli, Aahlad, et al. "Out-of-distribution generalization in the presence of nuisance-induced spurious correlations." arXiv preprint arXiv:2107.00520 (2021). Does line 243 have a typo? What is \epsilon_1? Algorithm 1 (listing) could use some text describing each step. In Table 3, please bold the best non-oracle accuracy.

Questions

How can one know whether two feature sets are complementary? Are there any tests for that? What if the stable features cannot be fully isolated? For example, in Puli et al. the authors need only grab the border of images to define the spurious variables but that leaves the center of the image as a mixture of spurious and non-spurious content.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

The authors have adequately addressed limitations.

Summary

This work considers unsupervised domain adaptation, where a learner observes labeled data from multiple source domains and unlabeled data from a target domain, and needs to produce a classifier that works well for data from the target domain. In contrast to the domain generalization setting where no data is available from the target domain, here, the learner can be less conservative and use features that are spurious (unstable or domain-specific) but help improve performance on the target domain. The main challenge is to do this with only unlabeled examples from the target domain. The main result of this work (Theorem 4.4) specifies a sufficient condition under which it is possible to use domain-specific features to boost the performance of a conservative classifier that relies on stable features only. Given stable features $X_S$ (i.e., $Y | X_S$ is invariant across domains) and unstable features $X_U$, such that $X_U \perp X_S \mid Y$ (complementarity) and $X_S$ is informative about $Y$, they express $p_\text{target}(Y | X_S, X_U)$ in terms of $p_\text{target}(X_S, X_U)$ and $p(Y | X_S)$, both of which can be estimated in the unsupervised domain adaptation setting. Essentially, it reduces to predicting pseudo-labels $\widehat{Y} \sim p(Y|X_S)$ from the unstable features $X_U$ and combining it with $p(Y|X_S)$ to construct a classifier that uses both $X_S$ and $X_U$. Besides this the authors propose a representation learning method that separates stable and unstable features, such that they also satisfy the complementarity assumption. They evaluate their method on a few synthetic datasets, CMNIST, PACS, and Camelyon-17. The results indicate improvement compared to domain generalization methods (i.e., without an adaptation step).

Strengths

**Strength #1: Significance.** Unsupervised domain adaption is an important and relevant setting for the NeurIPS community. The main result (Theorem 4.4) is a solid contribution to the list of conditions under which some form of out-of-domain generalization (with or with adaptation) becomes possible. I like how the problem of label shift that is a challenge for many methods, is handled naturally in the proposed approach. **Strength #2: Originality.** As far as I know, the main result (Theorem 4.4) is novel. The fact that in population-level analysis even slight informativeness of $X_S$ for $Y$ allows recovering the true conditional distribution $P(Y|X_S,X_U)$ is not trivial.

Weaknesses

**Weakness #1: Clarity & Presentation.** The clarity of the presentation can be improved. The notation is not consistent in many cases. Many important discussions/details are presented in appendices, while the main text space is not used effectively. Please see more detailed comments below. **Weakness #2: Limitations.** 1. In Theorem 4.4, it is assumed that we have access to stable $X_S$ and unstable features $X_U$. This assumption is quite strong, as one can argue that finding stable and unstable features is one of the key steps in out-of-domain generalization. To my best knowledge there are no methods that extract and separate stable and unstable features well. 2. In Theorem 4.4, it is also assumed that we have access to the Bayes probabilities $P(Y|X_1)$, where $X_1$ are the stable features. In practice this is not easy to achieve. 3. The proposed algorithm to extract stable and unstable complementary features has many hyper-parameters. It is unclear to me how sensitive is the method to these hyper-parameters and how easy it is to tune them. 4. The proposed method requires enforcing stability and complementarity, which is challenging. Not only it is hard to enforce these conditions on source domains (e.g., limitations of conditional HSIC), but the same OOD generalization concern arises -- do these conditions enforced on source domains carry over to the target domain? For this reason, the work will benefit significantly from evaluations targeted to reveal how well the proposed representation learning approach achieves stability and complementarity (rather than evaluating only target domain performance after adaptation).

Questions

- Lines 112 - 116: It should specified what is meant by *relationship* between a feature $X$ and the label $Y$ (i.e., the conditional distribution of $Y|X$). I understand that stable and unstable features are defined later, but at this point of the text reader does not know it and can only guess. - Line 115: I think it should be more exact what $X_S$ and $X_U$ are. As I understand from eq. (3.1) they are not subsets of the components of $X$, but rather extracted features: i.e., $X_S = \Phi_S(X)$ and $X_U = \Phi_U(X)$. - Lines 126-127: This sentence is not rigorous enough. - Eq (3.1): It should be $f^e_U(X)$. Also, in (3.1) and (3.2) it should be $h^e_U$ instead of $h^e$. - Line 131: $f^s(X)$ -> $f_S(X)$. - Lines 137-143: A separate letter should be used to denote the new environment (using $e$ for everything is confusing). Also, the usage of $h_U$ or $h_U^e$ is inconsistent. Why is there a sum over training environments in eq. (3.3)? Shouldn't it be just optimizing for the domain-specific classifier on the target data given already trained $\Phi_S, \Phi_U,$ and $h_S$? - Line 163: It should be $\sigma \circ h_U \circ \Phi_U$. - Definition 4.3. It should be $X_S \perp Y \mid E$. - Theorem 4.4: It is worth to mention in the statement that $\epsilon_0$ and $\epsilon_1$ can be expressed in terms of $P(Y|X_1)$ and $P(X_1)$. - Lines 295-305 (recap and learning goals): I suggest to remove this part. Instead you can bring some parts from the appendix to the main text (e.g., Appendices A.2, H or I). - Line 309, objective function: Do I understand correctly that during this first phase you have one domain-specific classifier $h^e_U$ per domain? If so, is the point to make sure that you learn meaningful $\Phi_U$, rather than a constant function? In that case one can argue that if there is a new kind of spurious features available for the target domain, that might not be picked in this first phase of training. - Do the results in the main text use multiple rounds of pseudo-labeling? - Are there no suitable unsupervised domain adaptation methods to compare the proposed approach with?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

The authors listed assumption of complementarity in Theorem 4.4 as the main limitation of their work. To me this assumption itself is not a big limitation, as it is satisfied in many cases. The main limitation is finding a representation of data that separates stable and unstable features (see my comments on limitations in the "weaknesses" section).

Summary

Unlike the common trend in the out-of-distribution generalization literature in the presence of spurious correlation, this paper takes a different approach. They argue that the spurious feature could be utilized to enhance the performance of the classifier. Then they go on to show how unlabelled data in the test domain can be used to train a classifier that adaptively uses the spurious/unstable feature (along with the stable features) to get near optimal prediction performance.

Strengths

**Clarity**: This paper is very well-written and easy to follow with appropriate connections between theory and experiments. **Significance**: 1. The majority of current literature in the OOD generalization in the presence of the spurious correlation is focused on learning a predictor that only uses the stable feature and thus generalizes across different environments. In doing so they throw away (possibly) easy-to-learn and highly predictive features for the given task since their correlation is likely to change across domains. 2. This work is an important step in the direction of adaptively making use of the spurious feature without any need for extra labels in the test domains. **Theory**: 1. Gave sufficient conditions under which we can train an adaptive classifier that could use unlabeled data in the test domain to use the spurious/unstable feature. 2. Show how to estimate the prediction probability using an individual classifier that only uses stable or unstable features. **Experiment**: Demonstrate that their method achieves near-optimal performance on synthetic datasets where the assumption of their method holds.

Weaknesses

**Experiment**: 1. QRM baseline for the syn, Camelyon, and PACS datasets are not available. 2. Possible experiment on other real-world datasets where we expect the assumption made in the paper to hold: otherwise it is performing almost equivalently on Camelyon17 and PACS.

Questions

**Minor Typos**: 1. Line 210: X_{S} \not\perp Y |E 2. Line 651: 4th line of the equation: Extra bracket after $\epsilon_{1}$ and $Pr[Y=1|X_2]$ instead of $Pr[Y=1|X_2]$.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

NA

Summary

This paper introduces a method to utilize spurious features when they are assumed to carry complementary information for predictions. A main contribution is to show that with unlabeled test samples, it is possible to learn how to use spurious features in the test domain. Based on the assumption that both stable feature $X_S$ and spurious feature $X_U$ are known, they derived a provably consistent algorithm for unsupervised test-domain adaptation. They further propose a new training objective function to learn $X_S$ and $X_U$ simultaneously when we have multiple training domains, based on the complementarity assumption ($X_S$ ⊥ $X_U$|$Y$). The full algorithm, including training and test domain adaptation, is named Stable Feature Boosting (SFB). The effectiveness of the test adaptation part of SFB is theoretically proved to be consistent. The whole SFB is empirically shown effective on both real and synthetic datasets.

Strengths

- Originality: good. It is a new exploration on how to harness spurious features. - This paper shows a comprehensive theoretical analysis of the adaptation problem. - This paper is well written and easy to follow. Experiment settings are well introduced.

Weaknesses

- Incomplete related works and baselines. As this work assumes the access to a set of unlabeled samples from the test domain, its setting is domain adaptation but not domain generalization. As a result, related works on domain adaption should be discussed and compared so that we can judge the contribution of this work. However, only domain generalization methods are compared, which could be unfair as different knowledge are assumed. - IID accuracies are not shown. It is unclear how the SFB affect the IID performance. - Missing ablation studies for the new training objective function of SFB.

Questions

- How about the IID performance of SFB? - What stability penalty is used to define $P_{Stability}$ in the experiments? How the quality of learning $X_S$ may effect the OOD performance of SFB w. adapt? - How this work can be distinguished from existing domain adaptation methods?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors have underlined the complementarity assumption as a limitation. Besides, the estimation of $P_{X_S, X_U}$ is challenging in practice when the two variables are high dimensional. Furthermore, this adaptation method is theoretically useful only when $X_S$ is partially predictive.

References (continued)

[10] Dong, J., Fang, Z., Liu, A., Sun, G., & Liu, T. (2021). Confident anchor-induced multi-source free domain adaptation. In _Advances in Neural Information Processing Systems_, 34, 2848-2860. [11] Shen, M., Bu, Y., & Wornell, G. (2022). On the benefits of selectivity in pseudo-labeling for unsupervised multi-source-free domain adaptation. _arXiv preprint arXiv:2202.00796_. [12] Kundu, J. N., Venkat, N., & Babu, R. V. (2021). Universal source-free domain adaptation. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ (pp. 4544-4553). [13] Liang, J., Hu, D., & Feng, J. (2021). Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In _International Conference on Machine Learning_ (pp. 6028-6039). PMLR. [14] Eastwood, C., Mason, I., Williams, C., & Schölkopf, B. (2022). Source-Free Adaptation to Measurement Shift via Bottom-Up Feature Restoration. In _International Conference on Learning Representations_. [15] Jiang, Y., & Veitch, V. (2022). Invariant and transportable representations for anti-causal domain shifts. In _Advances in Neural Information Processing Systems_, 35, 20782-20794. [16] Zhang, X., He, Y., Xu, R., Yu, H., Shen, Z., & Cui, P. (2023). Nico++: Towards better benchmarking for domain generalization. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ (pp. 16036-16047). [17] Wiles, O., Gowal, S., Stimberg, F., Rebuffi, S. A., Ktena, I., Dvijotham, K. D., & Cemgil, A. T. (2022). A Fine-Grained Analysis on Distribution Shift. In _International Conference on Learning Representations_. [18] Ye, N., Li, K., Bai, H., Yu, R., Hong, L., Zhou, F., ... & Zhu, J. (2022). OOD-Bench: Quantifying and understanding two dimensions of out-of-distribution generalization. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ (pp. 7947-7958). [19] Quinonero-Candela, J., Sugiyama, M., Schwaighofer, A., & Lawrence, N. D. (Eds.). (2008). Dataset shift in machine learning. _MIT Press_. [20] Storkey, A. (2009). When training and test sets are different: characterizing learning transfer. In _Dataset Shift in Machine Learning_, 30(3-28), 6. [21] Moreno-Torres, J. G., Raeder, T., Alaiz-Rodríguez, R., Chawla, N. V., & Herrera, F. (2012). A unifying view on dataset shift in classification. _Pattern Recognition_, 45(1), 521-530. [22] Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017, July). On the calibration of modern neural networks. In _International Conference on Machine Learning_ (pp. 1321-1330). PMLR. [23] Lakshminarayanan, B., Pritzel, A., & Blundell, C. (2017). Simple and scalable predictive uncertainty estimation using deep ensembles. In _Advances in Neural Information Processing Systems_, 30. [24] Kumar, A., Liang, P. S., & Ma, T. (2019). Verified uncertainty calibration. In _Advances in Neural Information Processing Systems_, 32. [25] Gupta, C., Podkopaev, A., & Ramdas, A. (2020). Distribution-free binary classification: prediction sets, confidence intervals and calibration. In _Advances in Neural Information Processing Systems_, 33, 3711-3723.

I have read the rebuttal

I've read the authors' rebuttal. I am content to let my score stand (at this time).

Post-rebuttal comment

I appreciate the authors' response. The discussions and new results address my concerns well, which move the scores upward. I trust the authors will incorporate these elements and the further ablation results in the final version, as indicated.

Response to Authors

I thank the authors for their response. I stand with my current assessment. Please incorporate the abovementioned changes in the final version of the paper.

Reviewer response

Thank you for the detailed response. > b. Reporting the magnitude of the conditions/penalties on source and target domains: This is generally not done in DG (e.g., for [1–3]) as these values are not interpretable/informative. > c. Something else: Perhaps the reviewer had some other evaluation in mind? Given a reliable way of estimating $p(y | \Phi_S)$, one can estimate any distance or divergence between $p(y | \Phi_S)$ of all available domains (source and target) and test whether the average source-source distance is smaller than the average source-target distance.