Chasing Fairness Under Distribution Shift: A Model Weight Perturbation Approach

Summary

This paper tackles the issue of fairness in situations where there are distribution shifts, viewing it from the angle of model weight perturbation. The authors establish a theoretical link between distribution shift, data perturbation, and model weight perturbation, leading to the conclusion that distribution shift and model perturbation are essentially the same. They propose a sufficient condition to ensure fairness transference under distribution shift and introduce the Robust Fairness Regularization (RFR) method to meet these conditions and achieve robust fairness. Experimental results on both synthetic and real distribution shifts show the RFR method outperforms existing baselines in terms of fairness-accuracy tradeoff.

Strengths

1. The fairness under covariate shift is an important problem. The authors also manage to show that Data Perturbation equals Model Weight Perturbation and use this motivation to construct the following methodology descriptions and theoretical analysis. The motivation and writing is strong. 2. The method proposed by this work is novel about the insights from optimal transport to optimize the worst case within the model weight perturbation ball and Data Perturbation equals Model Weight Perturbation. This paper points out the limitations of previous works well. 3. The experiments on fairness accuracy tradeoff and hyperparameter study are sufficient. Although in the table it does not explicitly outperforms baselines, the tradeoff is clearly better.

Weaknesses

I do not find important weaknesses on this paper.

Questions

The hyperparameter tuning seems important. Do you record the experimental setting variables and preferably provide means to reimplement the result? ---------- I have seen other reviews and am a little confused and quite frankly appalled why we need to assume zero access to data from the target distribution. Is it even possible if we do not even know data from the target distribution (or the distribution variation) to do transfer learning? Although it would be interesting to know a good $\rho$ a priori by either prior knowledge or theoretical bounds, many papers out there also do not discuss hyperparameters like $\rho$, and they are all fine. I fail to see why this could be a MAJOR problem. In my view, it is not even a weakness, but something that would simply be good to know and good to ask the authors about. I would like to raise my confidence to 4 and advocate the paper if needed.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors do not talk about limitations of this paper.

Summary

The paper is in the context of fairness measures under distribution shift. There is first a theoretical connection between distribution shift, data perturbation and model perturbations. The proposed method Robust Fairness Regularizatin (RFR), proposes a regularization using 3 terms L_CLF-- classifier loss function, L_DP ensuring DP on source data and L_RFR= L_RFR_S0 + L_RFR_S1 which is the fairness transferability from training set. The experiments are in synthetic (PCA transformations) and real data (Adult and Folk)

Strengths

The paper is extremely well-written and clear. Enjoyable lecture. The method proposed is simple. The work seems extensive and thorough. Experiments are well-redacted and easy to understand. The work is one of the few (to the best of my knowledge) that are tackling distribution shift and fairness, without strong assumptions like causal graph availability.

Weaknesses

The method is designed/tested in Neural Network architectures in tabular datasets. In tabular data algorithms like gradient boosting decision trees achieve SOTA[1] and are restricted to Linear Regression in regulated environments. The paper calls fairness to just one metric --Demographic Parity-- Fairness is not just one metric. L_RFR is not performance based but group disparate treatment based. The work does not acknowledge the difference between algorithmic fairness and fairness in broader socio-technical systems. ## Minor L72 Citation [9] is a specific "fairness distribution shift" citation, not a general one about distribution shift, which is what is about the sentence. I wonder how different are actually Adult and ACS Income. There seem to be many citations coming from Arxiv. Perhaps it could be good to check if there are more recent proceedings versions. [1] https://proceedings.neurips.cc/paper_files/paper/2022/file/0378c7692da36807bdec87ab043cdadc-Paper-Datasets_and_Benchmarks.pdf

Questions

The work states that "any distribution shift can be equivalent to data perturbations." Does this approach (Sec2.2) considers pure concept shift? When $P(X_S)=P(X_T)$ and $P(Y_S)=P(Y_T)$, but the only thing that changes is the relationship between covariate and target? $X\Rightarrow Y$. Is the method strictly limited to NN? If so (since its tabular data) it may be worth stating it earlier and clearer in the paper. I wonder if the theoretical results from section 2 can be aligned with the previous empirical findings of https://arxiv.org/pdf/2302.05018.pdf

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

Fairness is just one metric. I wonder if this limitation should be stated early on the paper or even affect the title. The work does not acknowledge the difference between algorithmic fairness and fairness in broader socio-technical systems.

Summary

theoretical proof: prove that distribution shifting can be written as data perturbation, and the model trained with the perturbed data can be written as the perturbed model trained with original source data. methodology: motivated by the connection between distribution shifting and data weight perturbation, the author proposed RFR(robust fairness regularization), which is an upper bound of the difference of demographic parity between source and target.

Strengths

well written; easy to follow. well-structured and coherent framework that facilitates easy comprehension.

Weaknesses

- I think that the objective function which the author suggested did not reflect the true effect of $L_{RFR}$. Please add the experiments without $L_{DP}$ - It would be better if the notation $\delta_X, \delta_Y$ were changed. The author writes $\delta_X, \delta_Y$ to denote the perturbation for feature and label respectively, however, in equation (3), it seems like they depend on each random variable X and Y. It can make confusion for readers. - some typos: for example, line 495(appendix), $E_{\delta_Y}[\delta_X]$ → $E_{\delta_X}[\delta_X]$, line 152 $\|\cdot\|_2$ →$\| \cdot\|_p$

Questions

- I cannot understand the proof of theorem 2.3, lines 499-501; is that sufficient? Please give more details. - I think it would be better if the graph of EO was also included in appendix.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

refer weakness and questions.

Summary

In this work, the authors seek to learn neural networks that are fair not just w.r.t. the observed distribution of individuals, but also under shifts to this distribution. In order to achieve this, they propose robust fairness regularization (RFR) which uses adversarial weight-space perturbations at train time to create more robustly fair models (w.r.t. distribution shifts). RFR being a reliable regularizer for distribution shifts relies on a correspondence between distribution and point-wise data perturbations as well as a correspondence between point-wise data perturbations and model weight perturbations. Finally, the paper proposes that for each group, the difference between the predictive performances on the source and target distribution should be the same. By enforcing this over all groups, the authors have the RFR loss that they propose.

Strengths

The paper focuses on a very important topic for trustworthy ML. The paper is well-written and the notation is clear and easy to follow. The experimental results that are reported are strong and at the core evaluate the method well, though I have some suggestions below in the weaknesses for improvements here.

Weaknesses

Major weakness: The major weakness that I see with this paper is the underlying theoretical motivation. In particular Theorem 2.3 appears to have a critical flaw that is never addressed in the paper (as far as I can tell). I elaborate further at the bottom of this section, but the key point is this, though there exists weight perturbation that give us the same loss as any arbitrary input perturbations, there is no given relationship between the magnitude of the perturbations in either domain. Therefore, even given that you know the Wasserstein distance is bounded by some value $\epsilon$ there is no way to select the value of $\rho$ before you have observed the target distribution, rendering the method difficult to use in practice. Moreover, I find it troubling that this is (1) not addressed in the paper, and (2) the authors do not explain how they select $\rho$ either theoretically or experimentally. It is obvious to me that given access to the target distribution, one can optimize $\rho$ to give good results, but this is the same as training on your test distribution and claiming good results. Without a clear and theoretically grounded argument for selecting $\rho$ and an open source implementation showing that this works well across datasets, I do not think I can vote to accept this paper. Another weakness I see, is that the authors do not compare their weight perturbation method to the corresponding input perturbation method (as far as I can see). The ADV method used in the experiments refers to methods like LAFTR which are adversarial, but not in the same spirit as crafting worst-case perturbations. Elaborated major weakness: The first claim (Corollary 2.2) is that there exists a perturbation map that makes the classification loss between source, $P_S$, and target $P_T$ the same. This statement seems clear and intuitive; however, we do not, a priori know how the magnitude by which we will have to perturb each data point. Let us assume, for sake of argument, that we know a priori that the Wasserstein distance between $P_S$ and $P_T$ is bounded by a real value $\epsilon > 0$. Then for an empirical distribution consisting of $n$ atoms, we know that we won’t perturb any given point by more than $n\epsilon$, so this gives us some practical guidance on how to set the data perturbation value, assuming we know a bound on the Wasserstein distance. The second claim made by the paper, Theorem 2.3, is that there exists a weight perturbation such that the model has the same loss on the source distribution (after its weights have been perturbed) as it does on the target distribution. However, though we may have a bound on how large the input perturbation may be, there is absolutely no correspondence between the magnitude of input perturbations, $n\epsilon$ and the magnitude we should set for the weight space perturbations $\rho$.

Questions

Please address my key point on how one picks $\rho$ in a principled way.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

4 excellent

Contribution

2 fair

Limitations

See weaknesses.

Summary

The paper studies the problem of training a fair ML model in the presence of distribution shift. The proposed method aims to bound the biases on the target distribution and use it as a regularizer.

Strengths

S1 The problem of distribution shift is quite fundamental. S2 The idea of bounding the biases and using it as a regularization term is underexplored.

Weaknesses

W1 The approach of using the worst-case bound as a regularizer has been explored before for selection bias [1]. Please discuss the differences with this approach. W2 The experiments do not explain the observations in detail. Specifically, the variance is quite high for spatial shift. Are the results staistically significant? Why are the baselines doing worse? It would be worth adding an experiment where source and target are exact same. I would expect the baselines to perform much better in this case. More generally, only when the source and target are very different, I expect RFR to perform better than the rest. The current experiments do not seem to clarify any such aspect. W3 Please show how loose are the estimated bounds. Are the bounds tight? W4 If possible, consider adding proofs or sketch of theorems. [1] https://arxiv.org/pdf/2212.10839.pdf

Questions

Why does RFR not have the best fairness for spatial distribution shift? Is it because lambda is small?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The paper does not discuss the limitations of the approach. I think the loose bound of fairness estimates can adversely affect model accuracy. Authors should discuss the practical implications of the presented approach.

Thank you for your response

I would like to thank the authors for their response. I found their answers somewhat satisfactory, but felt that my question went largely unanswered. I understand that a hyper-parameter search can be done to optimize $\rho$ (which is how the authors came up with $\rho = 0.0005$, but this hyper-parameter search requires that one has access to the target distribution of interest (in order to validate the best range of $\rho$ values). Also $\rho$ is the critical parameter in Equation (8), (9), and (10) i.e., the core contribution of the work so selection of this parameter should not be swept under the rug. If the authors could clearly answer this I would be more satisfied: given $n$ samples from the source distribution for training and 0 samples from the target distribution, how do I empirically select $\rho$? Should I always set $\rho = 0.0005$ no matter what network architecture or dataset I am using? I find it hard to believe that even a simple fully connected network that has width 8 and a fully connected network that has width 4096 will have the same optimal $\rho$ value given the disparity in the weight magnitude distributions. Also have I misunderstood Figure 4? Where is the abalation on $\rho$? I only see variations on $\lambda$ (the x-axis on both plots) and no variation of $\rho$. Nonetheless, the problem studied is an interesting and important one and the paper reports encouraging experimental results which may be built upon by the community in future. I would like to increase my score, but need further clarification from the authors on these points.

elaborate quotation for author

for Q1) I updated my quotation before rebuttal period, but unfortunately it seems not visible for authors. Here is my updated quation: It seems to me that additional conditions, such as non-zero, might be necessary for this proof to be sufficient. Could you explain the reasons behind the sufficiency of this proof? If these questions are addressed satisfactorily, I would be more than willing to increase my score.