DeNetDM: Debiasing by Network Depth Modulation

Neural networks trained on biased datasets tend to inadvertently learn spurious correlations, hindering generalization. We formally prove that (1) samples that exhibit spurious correlations lie on a lower rank manifold relative to the ones that do not; and (2) the depth of a network acts as an implicit regularizer on the rank of the attribute subspace that is encoded in its representations. Leveraging these insights, we present DeNetDM, a novel debiasing method that uses network depth modulation as a way of developing robustness to spurious correlations. Using a training paradigm derived from Product of Experts, we create both biased and debiased branches with deep and shallow architectures and then distill knowledge to produce the target debiased model. Our method requires no bias annotations or explicit data augmentation while performing on par with approaches that require either or both. We demonstrate that DeNetDM outperforms existing debiasing techniques on both synthetic and real-world datasets by 5\%. The project page is available at https://vssilpa.github.io/denetdm/.

Paper

References (51)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer Bkbf6/10 · confidence 4/52024-07-12

Summary

This paper presents a useful theoretical framework that shows that samples that exhibit spurious correlations lie on a lower rank manifold and that the depth of a network acts as an implicit regularizer for the rank of the attribute subspace. Building upon this, the paper proposes a method *DeNetDM* that creates a biased strong encoder (deep network) and a debiased weak encoder (shallow network) and then leverages this to train a final strong encoder that is debiased.

Strengths

1. The theoretical characterization provided in this paper is extremely intuitive and useful. Confirming the intuitions that examples with spurious attributes lie on a lower-dimension manifold formally is the most important contribution of this paper in my opinion. Supplementing this with the explanation that the depth of the network acts as implicit regularizer makes the method of this paper theoretically sound. 2. The confirmation of theoretical findings using synthetic experiments helps validate the theoretical claims. 3. Putting the theoretical findings together into a method that does outperform prior work illustrates the effectivness of this approach.

Weaknesses

1. Empirical Evaluation: The datasets chosen by the authors to evaluate their method are not standard in this literature. Common datasets such as Waterbirds [1] and CelebA [2] are missing. Moreover, including newer more challenging datasets such as UrbanCars [3], SpuCoAnimals [4] and SpuCoMNIST [4] could further improve this paper. 2. Insufficient discussion of similarity to related work: Two related works seem very similar in method to the proposed method. 1) Overcoming simplicity bias in deep networks using a feature sieve and 2) Learning from failure: De-biasing Classifier from biased classifier. The paper would greatly benefit from discussing the similarities and differences between the proposed approach and these 2 closely related approaches. Moreover, Identifying Spurious Biases Early in Training through the Lens of Simplicity Bias is another relevant theoretical / empirical study of spurious correlations which would be useful to compare & contrast with to better understand the contributions of the authors' work. [1] https://arxiv.org/abs/1911.08731 [2] https://arxiv.org/abs/1411.7766v3 [3] https://arxiv.org/abs/2212.04825 [4] https://arxiv.org/abs/2306.11957

Questions

See above in weaknesses.

Rating

6

Confidence

4

Soundness

4

Presentation

4

Contribution

3

Limitations

Empirical evaluation of method: discussed in greater depth in weaknesses.

Reviewer CcWa6/10 · confidence 3/52024-07-12

Summary

This paper proposes the unsupervised debiasing strategy via modulating network layers depth. It proves that the network with deep layers exploits bias attributes more than that with shallower layers, and shows that training on such less-biased network with shallow layers exhibit debiased learning. The proposed method outperforms other baselines in several datasets including CMNIST, C-CIFAR10, BAR, and BFFHQ.

Strengths

- It is novel contribution that identifies how the rank of attributes, specifically regarding dataset bias, is related to network depth. - DeNetDM shows superior debiasing performances against existing baselines across different benchmark datasets and correlation severities.

Weaknesses

1. Empirical analysis on networks' depth to learning of different ranks (bias-aligned / bias-conflicting) is limited to CMNIST. Additional experiments on other benchmark datasets, e.g., C-CIFAR10, BAR, and BFFHQ, are required for validity. 2. As shown in Tables 8 and 10, different pairs of networks result in significantly different results (about 20% differences in conflicting accuracy in Table 8). How sensitive are the bias-aligned (bias-conflicting) attributes with regard to depth of deep (shallow) networks, and the following performances of DeNetDM in stage 2? 3. Although authors focus on evaluation on biased dataset, it is unclear whether DeNetDM maintains the accuracy when applied to unbiased dataset.

Questions

See weaknesses.

Rating

6

Confidence

3

Soundness

3

Presentation

4

Contribution

3

Limitations

See weaknesses.

Reviewer 9dFt5/10 · confidence 3/52024-07-12

Summary

The submission proposes that deeper networks are more likely to use biased features than shallower ones. Using this idea, they develop a training algorithm to encourage reliance upon non-spurious attributes. This is done by training a deep and shallow network as a product-of-experts, then distilling the shallow network into a target network. Analytic experiments on synthetic datasets indicate that deeper networks do tend to fixate on simpler features as training progresses. A theoretical development of the key intuitions are provided, along with experimental results suggesting that the proposed algorithm can improve over baselines.

Strengths

Originality: The core idea in the submission about depth modulation is original to my knowledge. Some existing work (e.g. Gradient Starvation: A Learning Proclivity in Neural Networks) has discussed how being able to learn some target-correlating features can impede the learning of others, so it seems plausible that a network that learns a simpler feature quicker is less likely to also learn other features. Quality: There is a good set of comparisons to related methods, on some of the standard datasets in the literature. The ideas presented and the intuitions appear sound. Limitations such as anticipated difficulties in scaling beyond single-biases are acknowledged. Clarity: The submission is generally intuitively clear and easy to follow. Significance: The ML community continues to consider ways to approach bias in datasets, since specific domains that are data-scarce and unique enough that foundation models may not be available continue to motivate explorations along these lines.

Weaknesses

One key weakness in approaches of this flavour is the reliance upon use of a “debiased’ validation set to pick crucial hyper-parameters. This presumes knowledge of the bias, and if one knows it exactly, one can likely adopt other bias-informed approaches in practice. There is some work (e.g. Systematic Generalization with Group-Invariant Predictions) that have used a differently-biased validation set to pick hyperparameters relative to the test set, which can be a slightly better alternative than use of an “oracle” validation set. In practice, it is unclear if a model is always going to be operating in an OOD setting. In fact, it might be the case that OOD settings arise relatively infrequently in deployment. From this perspective, it might make more sense to consider both aligned and conflicting accuracies of all baselines. A competing model that takes less of a hit in-distribution might be preferable. The learning dynamics of decodability throughout training are interesting, but based only on the simpler datasets with a more drastic difference in the “complexity” of the core and biasing attributes. It is unclear how these trends play out for more realistic data. Minor: Is an equation with the full loss for training the target model missing? I’m assuming the objective used for training the target network is really L_t + \lambda*L_dist. A related baseline might be relevant for inclusion: Simple and Fast Group Robustness by Automatic Feature Reweighting, Qiu et al., ICML 2023. For the mathematical development, it might be better to include some of the key results from Huh et al., Wang and Jacot, etc. for sake of completeness, and also make it easier to gauge the additional contributions brought in this submission. Using \citep to might be better to enclose citations in parentheses/brackets.

Questions

The submission mentions use of a “small” validation set, could they clarify the exact size, relative to training and testing (I couldn’t find it easily upon a quick look)? It would be interesting to look at the decodability dynamics on the more complex datasets such as BFFHQ and BAR. Are there technical difficulties in showcasing these trends that I missed? Would it be possible to compare both aligned and conflicting accuracies for all methods?

Rating

5

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

The submission acknowledges limitations and broader impact in the Appendix.

Reviewer 2yWj7/10 · confidence 4/52024-07-24

Summary

This work makes several contributions in relation to network depth and dataset bias. It shows that bias-aligned samples lie on a lower rank manifold compared to bias-conflicting samples. This is linked with network depth by showing how deeper networks tend to prefer spurious correlations, which is demonstrated by the decodability of bias and core features from networks of different depths. Based on these insights, the authors propose DeNetDM, a debiasing approach to train deep (biased) and shallow (debiased) branches, with target debiased model corrected via knowledge distillation. Empirical results demonstrate effectiveness on both synthetic and real-world image classification datasets.

Strengths

[S1] The claims in the paper are backed both theoretically and empirically. Theorem 1 predicts lower-rank manifolds for bias-aligned samples and Theorem 2 shows how deeper networks prefer lower-rank features, thereby providing a solid theoretical backing for the argument that deeper networks prefer bias-aligned samples. The decodability vs depth plots support these claims. [S2] The proposed debiasing method does not rely on bias annotations or data augmentations, which is advantageous. The efficacy is demonstrated on evaluation benchmarks with varying ratios of biased/unbiased samples, which clearly shows benefits over existing debiasing methods.

Weaknesses

[W1] It is unclear why the biased branch needs to be deep. Clearly, both core and spurious features are available at shallower depths. Would it not be more beneficial to build a shallower bias detector instead? Other works including: [1, 2] have already explored leveraging shallower layers for bias correction, which is more efficient. [W2] DeNetDM is trained in two stages, but it is unclear why this is necessary. Is it not possible to train both biased branch and perform debiasing in parallel as done in [1]? [W3] The study does not take into account the impacts of loss functions/regularization. For instance, would the deeper networks exhibit similar proclivity to spurious features with spectral decoupling [3]? [W4] All the experiments are performed on image classification tasks, which raises questions on generalization to other tasks. [1] Shrestha, Robik, Kushal Kafle, and Christopher Kanan. "Occamnets: Mitigating dataset bias by favoring simpler hypotheses." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022. [2] Clark, Christopher, Mark Yatskar, and Luke Zettlemoyer. "Learning to model and ignore dataset bias with mixed capacity ensembles." arXiv preprint arXiv:2011.03856 (2020). [3] Pezeshki, Mohammad, et al. "Gradient starvation: A learning proclivity in neural networks." Advances in Neural Information Processing Systems 34 (2021): 1256-1272.

Questions

The questions correspond to the points listed in the weaknesses section: [Q1] Why not build a shallower biased branch? Is there any advantage a deeper biased branch offers compared to a shallower one? [Q2] Does the approach need to be multi-staged? Could the debiasing occur in parallel to the training of the biased branch?

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The limitations section should mention that is not tested on non-vision/non-classification tasks. Would the approach need any modifications for other tasks?

Reviewer 2yWj2024-08-11

I thank the authors for the response. One follow-up I have is: W1/Q1. Is it necessary to use explicit bias annotations to tell apart the core and spurious features? Existing works [1,4] use bias amplification to segregate the spurious features. Apart from that, I find the responses for Q2, W3, W4 to be satisfactory. I think the additional results will strengthen the paper. I [1] Shrestha, Robik, Kushal Kafle, and Christopher Kanan. "Occamnets: Mitigating dataset bias by favoring simpler hypotheses." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022. [4] Nam, Junhyun, et al. "Learning from failure: De-biasing classifier from biased classifier." Advances in Neural Information Processing Systems 33 (2020): 20673-20684.

Authorsrebuttal2024-08-11

Response

Thank you for your feedback. We are glad that we could address most of your concerns. We will add the additional results to the final revision of the paper. **W1/Q1**: As the reviewer rightly pointed out, bias amplification is a well-established method for addressing spurious features without needing explicit bias annotations, as shown in [1], and [4]. However, our approach leverages depth modulation as an alternative strategy for debiasing. Specifically, our deeper branch amplifies bias via depth modulation, a concept grounded in our theoretical insights. Our method also eliminates the need for explicit bias annotations to distinguish between core and spurious features, as the deep and shallow branches inherently act as biased and debiased models, respectively. Furthermore, we demonstrate that DeNetDM outperforms existing approaches like LfF, where bias amplification is achieved using GCE loss. We hope this clarifies your concern.

Reviewer 2yWj2024-08-11

Thank you for the response. I keep my original score of 7, recommending acceptance.

Reviewer CcWa2024-08-11

I thank the authors for the rebuttal regarding the concern of architecture sensitivity and unbiased dataset. My further responses are found below: 1. I appreciate the authors' response and the issue has been resolved. 2. I’m still concerned that DeNetDM’s sensitivity requires _significantly large amount of additional hyperparameter tuning_ for network architectures in both shallow and deep networks. Specifically, if there are $N$ and $M$ candidate of architectures for DeNetDM, it requires $N\times M$ validation for choosing the best model, which significantly increases the overall costs for validations when integrated with other hyperparameters. I believe it needs more systematic analysis on how sensitive the DeNetDM is, and therefore how consistently it outperforms other baselines that do not require such tuning, e.g., LfF, DFA. 3. Thanks for the clarification, and I believe such discussion regarding the limitation of DeNetDM when faced with unbiased test set should be included in the paper.

Authorsrebuttal2024-08-11

Response to Reviewer CcWa

We thank the reviewer for their valuable feedback. We provide additional clarifications with the hope of addressing the remaining concerns. 1,3 : We will include these results in the final revision of the paper. 2: We acknowledge the reviewer's point that if there are $N$ and $M$ candidate architectures for DeNetDM, selecting the optimal model could theoretically require up to $N \times M$ validations. However, the list of candidate architectures is manageable because we impose specific constraints on $N$ and $M$ based on our observations and assumptions. Specifically, we assume that the deep branch is either the same depth as the target network or one layer deeper, which limits the hyperparameter search space to $M=2$ across all cases. For the shallow network, we choose $N$ such that the depth of the shallow branch is significantly less than that of the deep branch, as our approach performs better when there is a substantial difference in depth between the shallow and deep networks. This further reduces the search space. For instance, in the case of CMNIST, where the target model is a 5-layer MLP, we consider the following pairs for hyperparameter search: (5,3), (5,4), (6,3), (6,4), and (6,5), which requires only five additional validations. Similarly in the case of C-CIFAR10, where the target model is a ResNet-18, we consider the following pairs for hyperparameter search: (ResNet-18, 3-layer CNN), (ResNet-18, 4-layer CNN), (ResNet-18, 5-layer CNN), (ResNet-20, 3-layer CNN), (ResNet-20, 4-layer CNN), (ResNet-20, 5-layer CNN) which requires only 6 additional validations. A similar strategy is applied for hyperparameter tuning on other datasets. The best-performing pair from this limited search space is used to report results in Tables 1 and 2. More extensive hyperparameter search might yield better solutions.But, even within the restricted search space, we demonstrate that our approach consistently outperforms all the baselines. Moreover, methods like LfF and DFA also introduce their own set of hyperparameters, such as $q$ in LfF (e.g., $q \in (0.7, 0.8)$) and $(\lambda_{dis}, \lambda_{swapb}, \lambda_{swap})$ in DFA, necessitating method-specific hyperparameter tuning to obtain the best model. The architectures used in Table 10 for C-CIFAR-10 were not originally included in the hyperparameter search space for C-CIFAR-10, as the deep network in question (ResNet-50) is significantly deeper than the target network (ResNet-18). The primary motivation for presenting the experiments in Table 10 is to demonstrate that DeNetDM can scale effectively with more advanced ResNet architectures. As discussed in the initial rebuttal, ResNet32 is more effective in capturing core features than ResNet8, which is a fundamental property of the architecture and not the depth. However, in both the cases, the inductive bias of depth modulation still holds since the deeper branch (ResNet-50) consistently captures bias irrespective of the depth of the shallow network. Hence, the inductive bias relying on depth modulation in DeNetDM is agnostic of such fundamental benefits / weaknesses stemming from architecture choices, and its sensitivity is a reflection of the best attainable accuracy from any given architecture.

Reviewer 9dFt2024-08-11

Thanks for the response! Having read the rebuttal and the other reviews, I am keeping my original rating.

Reviewer Bkbf2024-08-13

I find the author's rebuttal to be convincing and continue to recommend this paper for acceptance.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC