Mind the Graph When Balancing Data for Fairness or Robustness

Summary

This paper theoretically studies the applicability of data balancing in achieving fairness or robustness. The paper shows that data balancing may fail depending on the data generation mechanism. The paper introduces conditions for data balancing to produce invariant and optimal models. The paper also shows that data balancing can negatively impact fairness or robustness when combined with regularization strategies.

Strengths

Originality The paper extends the previous results on data balancing as mitigation for invariant models by considering different failure models and proposes strategies to distinguish between them. Qualify The paper offers some valuable insights into data balancing techniques, especially the failure modes of data balancing.

Weaknesses

This theoretical work on data balancing for fairness and robustness in machine learning presents some valuable insights, but falls short in several key areas. The paper's main contribution lies in demonstrating cases where data balancing techniques fail to achieve fairness. However, it does not effectively synthesize these findings into clear principles or rules, lacking a clear, overarching takeaway message. A more effective approach might have been to distill the various cases and propositions into a set of guiding principles or a unified framework for approaching fairness using data balancing. The authors present several propositions in Section 4, summarizing some of their results. While these propositions offer a formal structure to the paper, the conclusions they provide are relatively intuitive and do not significantly advance the field's understanding of data balancing. Another limitation of the paper is its focus on negative results. The authors primarily demonstrate scenarios where data balancing techniques are ineffective ("CANNOT results"), without offering constructive solutions to these identified problems. This approach, while valuable for highlighting potential pitfalls, reduces the paper's overall significance and applicability.

Questions

In Assumption 2.3, does “a function of X” mean “a subset of X”? Can the proposed method be applied to other types of distribution shift, like prior shift or concept shift?

Rating

4

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

N/A

Summary

The paper analyses training risk invariant models using data balancing. The authors consider the cases in which data balancing might help obtain risk-invariant models and the cases in which data balancing does not achieve the desired effect. The paper also considers the effect of regularization for robust model training, and how that compares to data balancing.

Strengths

The results presented are sound, and the problem considered in the paper is of practical relevance. I liked that the authors tried to analyse the problem on a simple semi-synthetic setup first, before moving onto the real-world data. This provides a nice test-bed to test and motivate the methodology presented.

Weaknesses

1. The paper only considers risk invariance w.r.t. correlation shift. Can this be generalised to other forms of distribution shifts? 2. The paper only considers the question of joint balancing of Y, Z. As the authors show in Proposition 5.1, joint balancing will not effectively remove the undesirable paths in G in all cases. Can the conditions be generalised to other forms of balancing? 3. Obtaining a DAG for a real-world setup involves the decomposition of $X$ (and positing causal relationships between these components). This seems like a non-trivial task in general. The authors don't touch upon the practical aspects of obtaining the DAG in the real world. 4. The paper does not provide a systematic methodology for determining whether data balancing is a good idea for a given real-world dataset. The experiment seems to be using a trial-and-error method to establish the failure modes, but this seems highly impractical.  5. Overall, I think the presentation can be improved. Certain parts of the paper are a bit vague. (See the questions below for more details.)

Questions

1. In line 161, does "performance" correspond to model accuracy? If so, how is the performance on $P^0$ lower than that on P? (where the latter has an overall accuracy of 0.717) 2. How do you try to decode $Z$ from the representation $\phi(X)$? This is an important piece of information that seems to be missing? 3. In Table 1, what is the test distribution on which the accuracy is measured? What is P(Z=1) for example, and is the test distribution of (X, Y, Z) fixed across the different tasks in Table 1? It seems that these test distributions differ across the different tasks. If so, the comparison of accuracy does not seem to be fair. 4. There is no motivation behind the choice of the 4 cases in Figure 1. These seem somewhat arbitrary. Why were these chosen specifically? Is there a systematic way of considering the different cases? 5. Does the model f need to be a neural network necessarily? If so, this should be made clear. 6. What do "penulatimate representation" and "intermediate representation" mean? Are these referring to the hidden layers of the model? 7. I found the paragraph starting on line 270 to be quite vague. For example, the authors talk about "regularization" without explicitly defining the regularization term. > "If we consider both the purely spurious correlation and the entangled case, we see that regularization and data balancing would have the same effects of blocking any dependence..." I am not sure I completely understand this statement either. The regularization is a methodology which would potentially modify the distribution of f(X), but the distribution of data (X, Y, Z) would still remain unchanged, so how does regularization change the DAG of data? In figure 3 and table 2, the accuracy on the original data distribution P should also be logged. In fact, in real-world settings, the practitioners would care about the accuracy on the original data distribution (not Q or P^0).

Rating

4

Confidence

3

Soundness

3

Presentation

2

Contribution

2

Limitations

The authors have addressed the limitations of their work in the final section.

Summary

The paper focuses on the topic of data balancing for fairness and robustness and uses a causal graph as a tool to analyze the effects of data balancing. In the paper, the paper tries to show both the positive impacts and potential pitfalls of data balancing. For the fairness aspect, the paper focuses on the independence between groups and labels in the data. For the robustness aspect, the paper focuses on the lowest risk across a family of target distributions. The paper uses several synthetic and benchmark datasets to support their theoretical analyses on data balancing.

Strengths

* S1. The paper empirically shows both the positive and negative impacts of data balancing, which possibly provides some insights into both the fairness and robustness field. * S2. The paper also suggests some causal graph-based analyses to understand the effects of data balancing.

Weaknesses

* W1. This paper examines both fairness and robustness, yet the relationship between these two aspects or the reason the paper focuses on these two remains somewhat obscure. The paper could benefit from giving a more in-depth discussion or intuition on why fairness and robustness are susceptible to similar issues. Potential discussions may include 1) how similar analysis can apply to these two different objectives and 2) why the paper focuses on fairness and robustness among various safety criteria. * W2. While the paper employs synthetic and benchmark datasets to illustrate various failure cases, a more concrete real-world motivating examples would further highlight the importance of addressing these failure cases in both fair and robust training. * Minor suggestion: It would be better to use full name of CBN in Figure 1.

Questions

The questions are included in the weakness section.

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

The paper has reasonable limitation, future work, and impact section.

Summary

The paper studies the role of data balancing e.g. on sensitive attributes or class labels on obtaining fair or robust models. It identifies various causal graphs and corresponding independence conditions under which data balancing is expected to succeed (or may fail) to provide recommendations on when to use (or not to rely on) it. The main contributions are in deriving sufficient conditions for data balancing to yield invariance under correlation shifts and relating the conditions to observations made in prior work through extensive experiments.

Strengths

1. Data balancing is a simple pre-processing method that one may occasionally resort to in practice to achieve fairness or robustness. Explaining when it may not work or when it works is significant. 2. Presentation and visualizations are clear, gives the required background, and explains the results in context of prior work. Jointly treating robustness and fairness in the exposition is a good way to generalize the results across the two problems. I like the organization of the results in successive sections 3-6 which are natural questions one may ask on data balancing. 3. The work discusses the implications when causal graph is not completely specified, features are learned, and tests the hypotheses on real, high-dimensional data (in contrast to causal ML work which typically only tests on tabular data). Experiments are carefully constructed.

Weaknesses

1. The writing can be clarified in places. For instance, I did not find a result which gives both necessary and sufficient condition contrary to the claim in Section 4 - please correct me if wrong. 2. (minor) Results can be summarized more concisely in a figure/table that shows the causal graphs or dependence structures where data balancing is fine and recommendations (e.g. on regularizers) for the other scenarios. 3. (minor) Some more recent works could be discussed. See comments below.

Questions

Please clarify which results are necessary, sufficient, and both. Minor comments which do not require a response from the authors: Although not necessary, please consider citing the original works for risk invariance under correlation shift for the claim in line 194, e.g. from Rojas-Carulla 2018 Invariant Models for Causal Transfer Learning https://arxiv.org/abs/1507.05333 or Peters et al. 2016 Causal inference using invariant prediction: identification and confidence intervals https://arxiv.org/abs/1501.01332. Consider relating the findings to more recent work such as Kaur et al. 2023 Modeling the Data-Generating Process is Necessary for Out-of-Distribution Generalization https://arxiv.org/abs/2206.07837. Please specify whether it is feasible and how to check if the sufficient conditions hold. Consider commenting on data augmentation that might achieve the same conditions as data balancing in some scenarios. I would suggest adding a footnote that the term sufficient statistic differs slightly from its traditional definition e.g. in line 198 it is more appropriate to say that E[Y|X_Z^\perp] is a sufficient statistic for the outcome risk function E[Y|X], not for Y. --- After the response Thanks for responding to the my concerns. I do not have any more questions and feel that the work is significant in that it presents a better understanding of data balancing (a commonly-used method in fairness or robustness problems). I have raised my score to 7 as a result.

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Limitations are acknowledged in good amount of detail.

Thanks for the response

The response adequately addresses all of my concerns. I particularly appreciate the table summarizing the work and updates to the text on sufficient and necessary conditions. I feel that the work is significant in that it presents a better understanding of data balancing (a commonly-used method in fairness or robustness problems). I have raised my score to 7 as a result.