Demographic Parity Constrained Minimax Optimal Regression under Linear Model

Summary

This paper studies the linear regression problem under a new definition of $(\alpha, \delta)$-fairness consistency as a fairness constraint. In particular, the authors derived that under the constraint of $(\alpha, \delta)$-fairness, the minimax optimal error is $dM/n$ when $\alpha$ is less than 1/2. Moreover, they provided an estimator that achieves this optimal rate of convergence.

Strengths

This paper established matching upper and lower bounds on the minimax optimal error and proposed a regression algorithm that achieves this optimal error for the linear regression model under a specific fairness constraint. The theory and method extends the current literature to cover partial coefficients and non-sensitive features on the sensitive attribute. The paper is well presented and articulated. The authors also provided detailed proofs which hightlighted the technical difficulties in the minimax analysis, as well as a novel estimator for achieving the minimax optimal bound.

Weaknesses

It would be great if the authors could discuss and provide more understanding on how restrictive the assumption that $\alpha <= 1/2$ is. Also, how should one think about solving for the case that $\alpha > 1/2$? The paper is lacking empirical study/examples which I feel would be nice given the subject under study.

Questions

It would be great if the authors could discuss and provide more understanding on how restrictive the assumption that $\alpha <= 1/2$ is. Is it a necessary condition for achieving the minimax rate of dM/n? Also, how should one think about solving for the case that $\alpha > 1/2$? How does the fairness score proposed in the paper compare with other fairness scores in the existing literature? The authors mentioned that misapplication of the findings in the paper to other models might result in discriminatory treatment. Any discussion on the robustness of the proposed algorithms, especially under misspecified models?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors have pointed out the restriction of the proposed models and cautioned against misusing the algorithms for non-linear models.

Summary

This paper investigates the minimax error in regression under the constraint of demographic parity (DP). In contrast to earlier works, this paper studies both direct and indirect discrimination and their effects on the estimator. In addition to proposing an algorithm to construct a regressor achieving a desirable error, they provide extensive theoretical analysis characterizing upper and lower bounds under the DP constraint.

Strengths

1. The paper is nicely written, with a clear structure and a comprehensive explanation of a large number of notations. 2. It studies a simple but interesting, important problem, which indeed catches attention and makes a good contribution to the community. 3. The introduction of direct and indirect discrimination is novel and realistic, and the examples are helpful for readers to understand those notions. 4. The theoretical analysis is comprehensive. The discussion in Section 4 is interesting to read and may provide insights for future work in this area.

Weaknesses

I did not find particular weaknesses for this paper.

Questions

In your model, you mentioned that indirect discrimination mainly impacts $\mu_S$ while direct discrimination is reflected via $\beta_S^*$. My understanding is that, direct discrimination is for the “model” itself, i.e. $\beta$, while indirect discrimination has influence on the data set. Is my description accurate?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

The authors addressed them at the end of Section 8.

Summary

The authors study the problem of minimax optimality for the linear regression model under demographic parity constraints from the fairness literature. They provided matching upper and lower bounds for the model that is considered in the paper.

Strengths

Overall: The paper is well-motivated by presenting both direct and indirect discrimination that occurs in real-world settings and the proposed model accounts for it. The theory is well-laid out and the difficulty is elucidated in the main paper with details left to the appendices. However, there isn't any experimental work. Pros: (A) The model differs from the prior literature (Chzhen and Schreuder) and is well motivated in Section 1 and Table 1. (B) The technical difficulty of both the norm estimators and the direction estimators is presented in equation 7. (C) The key takeways that the optimal error is independent of mean but depends on the variance is intriguing.

Weaknesses

(1) How does this compare to the results of Chzen and Schreuder is unclear. There is no unified setting which captures both models. (2) Also, there is no experimental work which would help us know the tightness of the proposed upper bounds (constants). (3) Some of the writing can be sharpened and there are multiple typos.

Questions

Please check the cons. Slightly lower score because of lack of experimental work.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

3 good

Contribution

4 excellent

Limitations

N/A

Summary

This paper studies the statistical minimax error of building a linear regression model under the constraint of demographic parity. It generalizes the result in Chzhen and Schreuder [5] by allowing the slope coefficients to change with sensitive attribute rather than just the intercepts. The authors provide both upper and lower bounds of the minimax error, and show that they match in the dependency on $n$.

Strengths

Although this paper uses a framework similar to Chzhen and Schreuder [5], it covers a very important setting where there exists both direct and indirect discriminations. The analysis begins with Lemma 1, which can be easily derived from Chzhen and Schreuder [5]. However, to obtain the upper bound of the minimax error, the authors need to characterize the errors of approximating each component in (5), which is one of the main challenges in this paper. They develop new techniques in Theorems 4 and 5 to handle this challenge. I think it has made significant contribution on top of Chzhen and Schreuder [5].

Weaknesses

1. This paper assumes the covariance matrix of X is an identity matrix for each $s$. It will be better to allow a more general matrix $\Sigma_s$. 2. It only covers the case where $\alpha\leq 1/2$.

Questions

1. The statement made in line 136-137 should be explained more clearly. In particular, why does enforcing strict demographic parity result in a constant function? 2. The authors consider $(\alpha, \delta)$-consistently fair regressor while Chzhen and Schreuder [5] consider $(\alpha, t)$-valid estimators (see DEFINITION 5.2 in [5]). Are they equivalent? If not, why do the authors consider a different type of fair regressor in the definition of the minimax error. 3. Equation (4) needs to be explain, especially the second inequality. Also, I think $\beta$ here should be $\beta^*$.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

See weaknesses above.