Equal Opportunity of Coverage in Fair Regression

Summary

Authors propose a new uncertainty-aware fairness – Equal Opportunity of Coverage (EOC) since current definition do not guarantee equal coverage rates across more fine-grained groups (e.g., low-income females) conditioning on the true label and is biased in the assessment of uncertainty.

Strengths

- paper is technically sound - experimental results are promising

Weaknesses

- presentation is mostly verbose and unclear - novelty is limited

Questions

The paper is, in my opinion, technically sound with promisin gresults. Nevertheless I think that the approach is not completely novel. Many works in the literature of fairness focus on distribution matching (see also NeurIPS papers when they use MMD, Sinkhorn distance, or first order matching) and there in no discussion on the relation with these works and a comparison with the idea of this work. Authors should clarify and elaborate on this.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

Authors should better clarify the novelty of the work and its impact.

Summary

This paper studies fair machine learning under predictive uncertainty. They propose a new notion of fairness named Equal Opportunity of Coverage (EOC), building upon "equalized coverage." While EC ensures that every demographic group receives the same level of prediction certainty, EOC ensures that every demographic group with similar true labels receives the same level of prediction certainty. This extension is similar to the extension of equal opportunity from statistical parity. With EOC, they also ensure that the entire population's coverage can be maintained at a predetermined level. To achieve EOC, they propose a post-processing method called Binned Fair Quantile Regression (BFQR), a distribution-free post-processing method for a given trained model. They show the power of their method both on the simulated data and real-world datasets.

Strengths

This is a very interesting research direction. Fair regression is also a challenging problem, and little research extends equal opportunity (and its variants) to regression settings. This is a promising research direction.

Weaknesses

I have a few apprehensions regarding the notation and the inadequate introduction of certain mathematical objects. Please see my comments below.

Questions

### Major Comments: 1) Line 83: Why are the calibration and test sets exchangeable? 2) Line 111: How does this indicate "V \perp A"? 3) TV definition in Line 122 needs to be more rigorous. Please define what $p$ and $p'$, which space they live, etc. 4) Up to Line 135, I need clarification on $p$. Is the true data-generating distribution, or is it the empirical distribution? 5) In Lemma 3.2, what is the sum of independence measures? What is the exact reference of Lemma 3.2 in [30]? 6) Why do you choose to use TV? (I understand why not KS distance, but I do not understand why you choose TV over other metrics. An example could be Wasserstein distance, MMD, KL divergence etc.) 7) Line 164: The coverage number $\beta_m$ is the same for each group, right? It is different across subgroups. Then, I do not understand why $\beta$ is computed separately for each group. Isn't it computed separately for each bin? I do not understand that part. 8) The difference between BFQR and BFQR* is not discussed in the main text. What are the differences between the two? 9) What are the conditions that Assumption 1 is violated? How strong is this assumption? 10) Is it possible to include EOC constraints somehow during training? Do you think it would be a stronger method if this method could be an in-processing method? ### Minor Comments: 1) Typo Line 85: "Pr(" instead of "P(". 2) In (1), $min$ and $s.t.$ should be aligned, and "s.t." should be a math operator. 3) Please introduce each object rigorously. It is unclear to me what sort of mathematical objects you are referring to until I see them used. For example, quantile regression. What is $\hat q$? Is this a function? Is it a vector? What is $\alpha_{lo}$? This would help the readers a lot if you were to introduce each object appearing in the paper rigorously. 4) Line 252: "l" should be capitalized in lemma 5) Line 312: "t" should be capitalized in theorem 6) Please indicate the best results in the experiment with bold font. 7) Line 224: "W" should not be capitalized. 8) Line 360: "\beta_m" instead of $\beta m$ 9) Line 272: "Section" instead of "Sec." (also 290). 10) Line 311: "Figure" instead of "Fig." 11) What is the probability distribution $\mathbb P$ appeared suddenly in Theorem 4.1 and Theorem 4.2? I do not think the notation of this paper is unified. Please unify the notation during the rebuttal. Defining notation rigorously and sticking with it throughout the paper is important. Otherwise, the paper becomes unnecessarily hard to follow. 12) I think it would be safe to drop the "under no label distribution shift on test data" assumption from Theorem 4.2. In the beginning, if you were to define the true data-generating distribution and explain how the test data and the calibration data are created from iid samples from this true data-generating distribution, then there is no need to mention this shift. If you could cover the shifted distribution case, then I would suggest mentioning it.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

Yes.

Summary

The authors propose a method to achieve same coverage rates between different groups conditioned on similar targets in addition to satisfying a marginal coverage rate on the entire population. The proposed objective is denoted as Equal Opportunity of Coverage (EOC) and the authors propose a two step approach denoted as Binned Fair Quantile Regression (BFQR) to address it. This method partitions the target space into bins and for each bin finds a coverage rate such that EOC is satisfied (i.e., equal coverage across sensitive groups inside the bin and global coverage rate level is achieved.). At test time the prediction intervals are built as the union of the intersections of each bin and its associated conformal prediction interval taking into account the group membership of the test sample.

Strengths

I think the problem presented in this work is well motivated. Figure 1 is a nice graphical example to motivate the importance of considering EOC over Equalized coverage. The definition of EOC is clear and as far as I know it was not discussed in other works. Its associated theory is well linked to previous works on Equal Opportunity. Empirically, the proposed BFQR method is improving the maximum coverage gap between groups across the bins when compared to almost all the competing approaches. It also seems to achieve similar group coverage (average) across the groups and maintaining smaller interval widths.

Weaknesses

I think the clarity of the presentation should be improved. In particular, I found that the description of the proposed procedure in Section 4 needs better clarification. For instance, L161 to L164 mentions that to enforce EOC the coverage rate within each bin $\beta_m$ is the same across groups. The method to achieve this in practice is not clear to me when reading the main manuscript. Paragraph in L116 discusses that TV distance w.r.t. the closest EOC distribution (p’) is measured. However, it is not clear enough how p’ is obtained and how this distance is obtained. I understand the authors are citing the works from [39,40] where an ‘easy-to'-compute statistic T’ is being used but this is an important point in the proposed method and should be clarified here for reproducibility purposes. I found that Algorithm 1 in Appendix is a better summary of the description provided in last part of Section 4.1 and 4.2. However, this Algorithm builds on top of the previously estimated bin coverage rate $\beta_m$. I think the organization of the paper is a bit convoluted since Section 4.3 describes how to obtain $\beta_m$ but this was already discussed in Section 4.1 and Section 3. Moreover, Section 4.1 and Section 4.2 were relying on this. I think that the description on how $\beta_m$ is obtained could be improved, it could be made more concise and informative. Figure 1 provides a nice and intuitive illustration of the problem, it would be nice to see the same plot after the proposed post processing procedure is applied in comparison with the other competing methods. From Algorithm 1 in the Appendix and L172-173 it seems that during test time the sensitive attribute needs to be available. I can think of this as a weak point in the fairness setting since the sensitive attribute needs to be available at test time. However, this is also the case for other existing fairness post-processing approaches.

Questions

If the sensitive attribute needs to be disclosed for each test sample, I think a reasonable comparison would be against a model that performs a group conditional split conformal approach that conditions on the sensitive attribute and a binning based on the outcomes of the quantile regression model. Is any of the competing approaches doing this? CFQR seem to be the method achieving the best Max coverage gap in Table 1 and Table 3 by a large margin. In particular, in Table 1 the max coverage gap is 0.19 but group A=1 has an overall coverage of 71.38 versus the other two groups whose coverage are larger than 93. I wonder if this is because across the majority of bins group A=1 is the one with the worse coverage even though by a small amount per bin (i.e., less than 0.19). How many bins where considered to the synthetic example? I think to achieve that large difference between group coverages with small coverage gaps there has to be many bins.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

yes

Summary

This paper studies the problem of algorithmic fairness using uncertainty quantification methods in regression settings. The main novelty of this work over previous work is taking the notion of “equalized coverage” and strengthening it by considering finer groups by additionally conditioning on the target variable. The result is a conformal prediction based post-processing method that provably gives conditional coverage guarantees over sets defined by the demographic groups and target variable, where most prior work just considers sets on the group membership. Another upshot of this method is that the conformal-based post-processing method proposed (BFQR) explicitly focuses on also optimizing interval widths of predictions to improve “the amount of decision-making information available” to downstream decision-makers. The paper is organized by presenting the main fairness notion studied - equal opportunity of coverage (EOC), describing the conformal prediction based post-processing approach to improve EOC and optimize for prediction interval widths. The main theoretical results are marginal and conditional coverage guarantees of BFQR. Finally, there is experimental evaluation on one synthetic and two benchmark datasets with comparisons to a few other conformal prediction based regression procedures.

Strengths

This paper studies an important problem - giving conditional coverage guarantees, while conditioning on the true label of the data, using a general post-processing conformal method, with particular emphasis on minimizing interval width to give more utility to downstream decision makers. The presentation is clear and precise.

Weaknesses

A main upshot of BFQR as presented is the improvement on interval width, which seems to be modest on 2/3 (synthetic and Adult) datasets. It seems hard to compare this statistic with the other methods and evaluate how it compares to BFQR’s slight degradation in tightness of coverage rates.

Questions

There are a few things that would be helpful to clarify in your experimental setup. What are the groups / conditioning events for the comparison experiments run with GCQR and MVP? My understanding is that the sets they give conditional guarantees for do not need need to condition on the true label - but can use them if given the information. If tested in this way, could it take some of the bite out of the improvement we see in EOC with BFQR / is that already how the experiments were run?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

-