Machine learning (ML) models can underperform on certain population groups due to choices made during model development and bias inherent in the data. We categorize sources of discrimination in the ML pipeline into two classes: aleatoric discrimination, which is inherent in the data distribution, and epistemic discrimination, which is due to decisions made during model development. We quantify aleatoric discrimination by determining the performance limits of a model under fairness constraints, assuming perfect knowledge of the data distribution. We demonstrate how to characterize aleatoric discrimination by applying Blackwell's results on comparing statistical experiments. We then quantify epistemic discrimination as the gap between a model's accuracy when fairness constraints are applied and the limit posed by aleatoric discrimination. We apply this approach to benchmark existing fairness interventions and investigate fairness risks in data with missing values. Our results indicate that state-of-the-art fairness interventions are effective at removing epistemic discrimination on standard (overused) tabular datasets. However, when data has missing values, there is still significant room for improvement in handling aleatoric discrimination.
Paper
Similar papers
Peer review
Summary
The manuscript introduces FairFront i.e., an estimation for the upper bound on the Pareto Frontier for Fairness and Accuracy. The authors empirically show the tightness of this bound by showing how SOTA approaches perform close to the FairFront, while the gap may be attributed to the distributional variations.
Strengths
- The authors tackle the problem of finding the upper bound on the Pareto frontier in contrast to prior works which focus on the lower bound. - The approach presented in the manuscript is theoretically grounded (by the essence of using Blackwell's results). - SOTA approaches are used to show the tightness of the bound empirically.
Weaknesses
- The biggest weakness is that epistemic and aleatoric discrimination are terms introduced in the paper which have a link to the corresponding concepts in the uncertainty literature. However, the way these terms are used contradicts the definitions of the uncertainty counterparts e.g., Aleatoric uncertainty is linked to missing data (line 14-15, 65-66, 250-251). In prior literature, the lack of data falls in the domain of epistemic uncertainty [1][2][3]. - In the manuscript, Aleatoric discrimination is linked to distributional differences. However, prior work either differentiates it from both aleatoric and epistemic uncertainty[4] or considers it as a part of epistemic uncertainty [5 (Section 9.2 dedicated to this topic)]. - It is mentioned that distributional differences are largely a grey area in prior work. However, works such as FairBatch [6] and FairMixup [7], tackle such cases and are ideal for inclusion in the experiments. - The manuscript is a bit hard to follow. Even though it has content for describing which components are used formally, there is little intuitive background. Some links are hard to follow. e.g. it is not immediately obvious that $(S, Y) - X - \hat{Y})$ refers to the Markov chain in Definition 2. It is not evident why this is used as the Markov chain. [1] Swiler, Laura P., Thomas L. Paez, and Randall L. Mayes. "Epistemic uncertainty quantification tutorial." _Proceedings of the 27th International Modal Analysis Conference_. 2009. [2] Shaker, Mohammad Hossein, and Eyke Hüllermeier. "Aleatoric and epistemic uncertainty with random forests." _Advances in Intelligent Data Analysis XVIII: 18th International Symposium on Intelligent Data Analysis, IDA 2020, Konstanz, Germany, April 27–29, 2020, Proceedings 18_. Springer International Publishing, 2020. [3] https://docs.aws.amazon.com/prescriptive-guidance/latest/ml-quantifying-uncertainty/epistemic-uncertainty.html [4] Amini, Alexander, et al. "Deep evidential regression." _Advances in Neural Information Processing Systems_ 33 (2020): 14927-14937. [5] Varshney, Kush R. "Trustworthy Machine Learning." _Chappaqua, NY_ (2021). [6] Roh, Yuji, et al. "FairBatch: Batch Selection for Model Fairness." _International Conference on Learning Representations_. [7] Mroueh, Youssef. "Fair Mixup: Fairness via Interpolation." _International Conference on Learning Representations_. 2021.
Questions
It is hard to see intuitively why FairFront works e.g., why is the markov chain $(S, Y) - X- \hat(Y)$? How does it filter out valid classifiers? I acknowledge that I read the response and I feel overall the rebuttal was satisfying.
Rating
4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
Confidence
5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness
3 good
Presentation
2 fair
Contribution
2 fair
Limitations
No. - From Lemma 1, it appears that one of the limitations of this work is that the FairFront is only applicable to convex classifiers. - I would highly suggest renaming the terms 'epistemic' and 'aleatoric' discrimination since they do not portray the corresponding uncertainty counterparts accurately and may lead to confusion. Terms such as 'Distributional Discrimination' may be more fitting.
Summary
This paper splits discrimination in machine learning into aleatoric (which is that inherent to the data distribution), and epistemic (which is that due to choices in the model). They use Blackwell’s results to characterize the fairness Pareto frontier curve. Then, on 4 datasets with 5 fairness interventions, they characterize how close to their upper bound the algorithms are able to achieve.
Strengths
- Comprehensive set of experiments on a number of datasets and fairness interventions - Clear writing and presentation of work - Claims are backed up by theories and proofs
Weaknesses
- Given that the output from the algorithm (L236) is always the upper bound for FairFront, rather than exactly FairFront, I wish the paper would have been more upfront with this throughout, especially in the introduction and abstract, because I found this a bit misleading - Even though the names of aleatoric and epistemic discrimination are taken from prior work, I think they should be potentially rethought because for example, “epistemic discrimination” feels like a term that should be restricted to the philosophical foundations of the word “epistemology,” e.g., as it is used in epistemic injustice (https://academic.oup.com/book/32817), rather than a not-so related mathematical constraint - Given that a primary differentiation from prior work (L76) is that it works on multiclass classification as well as multiple protected groups, I would have liked to see empirical results on this rather than just claiming this is true - Given how similar the motivation of this work is to Chen, I. et al. (NeurIPS 2018)’s, I would have liked to see a deeper comparison of the findings of both works - “Discrimination” as used in this paper is only restricted to different forms of measurement disparities, e.g., statistical parity, equalized odds, and excludes many other forms of algorithmic discrimination that are more structural. So, sentences like L35 “We divide algorithmic discrimination into two categories” should be caveated that this is only a very specific form of algorithmic discrimination that is being divided into two categories (https://aclanthology.org/2020.acl-main.485/) - As further explanation, taxonomies of algorithmic discrimination are its own genre of normative research, for example the distinction between representational vs allocational harms (https://www.youtube.com/watch?v=fMym_BKWQzk), or a taxonomy of harms within representational harms (https://arxiv.org/abs/2305.01776)
Questions
Listed above in weaknesses.
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
4 excellent
Presentation
3 good
Contribution
2 fair
Limitations
I think that the authors could be more upfront about the limitation that their work maps out the upper bound of the FairFront rather than the FairFront exactly, and also that they have taken a rather narrow lens of fairness, neglecting broader societal concerns.
Summary
This paper proposes a decomposition of discrimination (in ML classifiers) into aleatoric (irreducible) and epistemic (reducible) components. The paper surveys related work in fairness. It then introduces and discusses the Fairness Pareto Frontier, which is essentially an upper bound on the accuracy of the best possible fairness constrained classifier. The paper proposes an algorithm for estimating aleatoric discrimination (this upper bound) which utilizes Blackwell’s results on comparing statistical experiments. It then demonstrates the applicability of the algorithm through experiments on 5 datasets. Finally, it identifies missing values in the dataset as a source of aleatoric discrimination, providing experimental evidence to support the discussion.
Strengths
Though I was admittedly not familiar with Blackwell’s results, the paper’s technical contributions are insightful and convincing. The paper makes a novel connection to statistical results that (as far as I am aware) had not been considered previously in fairness research. The theoretical framework and results, as well as the estimation method are applicable to a wide range of fairness-constrained predictive modelling problems (multi-class, multi-attribute), and thus are a substantial contribution to the field. The paper does a good job of surveying related work, and distinguishing its contributions. The writing is concise and of high quality.
Weaknesses
The paper is dense and would benefit considerably from an illustrative example, helping to build intuition about the terms and concepts introduced. Specifically to help digest the main theoretical result, Theorem 1, and the piece-wise linear approximation. The paper presents FairFront as an upper bound whose estimate depends on k, and T. However, for a given data distribution, it is unclear how the k-dependence is affected by the quantity of available data. As a practitioner, I would want to understand the robustness of this upper bound to data availability. Some experiments that subsample the dataset would be a helpful starting point. Perhaps a version of the experiment discussed in Appendix C.2 with varying levels of data availability (up to the infinite data regime). The region described as epistemic discrimination (if Figure 1) is effectively a gap between an upper bound (FairFront) and a lower bound (some fairness-constrained modelling approaches). As such, the true fairness Pareto Frontier is somewhere in between. It is unclear why the reader should interpret FairFront as the more accurate bound, and thus interpret the gap as epistemic discrimination (rather than FairFront estimation error). That is to say, the paper could elaborate on the tightness of this approximate upper bound based on choices of k and T. The use of transparency in Figure 2 makes it difficult to compare scenarios (missing probabilities), especially when printed.
Questions
When the fairness constraints are trivially satisfied, for example $\alpha_{\text{SP}}, \alpha_{\text{EO}}, \alpha_{\text{OAE}} = \inf$, how should one interpret the FairFront? Line 231: what is meant by “mostly violated”?
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
4 excellent
Limitations
The paper refers to fairness and discrimination, but makes no effort to connect these concepts to the harms or benefits experienced by real people. This is fairly common practice in the field and not unique to this paper. However, it is nonetheless a noteworthy limitation. That is to say, one cannot truly comment on the fairness (or discrimination) of a machine learning system without a broader understanding of the social impacts–beyond just its predictions. It is unclear whether the method can be applied in the numerous important situations whereby the prediction outcome is unobserved (or partially observed), and what effects estimating these counterfactuals would have on the method, e.g., in lending–where the outcomes of the rejected cohort are counterfactuals.
Summary
The paper casts the issue of fairness in machine learning from the perspective of two classes of discrimination: those due to aleatoric uncertainty, or where the inherent limitations of the data distribution, and epistemic discrimination, which is due to modeling choices. It then uses this framework to analyze a common set of fairness metrics, showing that epistemic discrimination can be reduced by optimizing for these fairness measures, but not necessarily aleatoric discrimination.
Strengths
- The choice to disentangle sources of discrimination due to modeling choices vs. data properties is sound and reasonable. - To the best of my knowledge, studying aleatoric discrimination via Blackwell's equations is novel (the reviewer is not familiar with this set of techniques, so can offer little here) - The choice to study missing data values is natural.
Weaknesses
My main concern with this paper was that its contribution feels quite limited both in terms of thoroughness of the experiments (it would have been interesting to study different sources of data properties beyond miss values, such as Khani & Liang, 2020) do with spurious features, as well as its relevance in modern machine learning (the only data studied is the well-known COMPAS dataset, and it could have been interesting to consider situations where more advanced models -- such as language or vision models -- are applied, as well as settings where the motivation to apply ML is well justified unlike risk assessment). I also think the authors could clarify quite early on they specific a particular subset of fairness interventions - group fairness with defined subgroup labels. This is not always possible when subgroups aren't clearly defined, challenging to measure, or are intractable. In a similar vein, the writing could be more precise to properly convey the scope of FairFront. For example, sentences like " For instance, if the training set contains few samples from a given population group, then increasing sample diversity is a more effective strategy than selecting a more complex model class or training strategy." --> there are training methods that try to account for unequal group size (e.g. Hashimoto, 2018)) when group labels are inaccessible, and increasing sample diversity is hard. These sentences could just be qualified more that we are in a particular setting where we know the groups we want to be fair over, and can measure them.
Questions
Are you able to run experiments that involve a bit more complex sources of data variability (e.g. a natural language task with different dialects, so more than 2 group labels, and using more state-of-the-art models like GPT series)?
Rating
4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
Confidence
2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
1 poor
Contribution
2 fair
Limitations
I think the paper could also be more clear on the importance of the overall result. Lacking in thorough experimentation, the paper largely presents a framework but does not argue for its utility. Do the authors intend their framework to be used as a rigorous benchmark? If so, they should focus on more comprehensive set of experiments. Otherwise, what is the core message of knowing that methods that are not data-specific do not address dataset-specific sources of discrimination?
Summary
In this manuscript, the authors make two main contributions to the technical study of algorithmic fairness. Firstly, on the conceptual level, they propose to distinguish between aleatoric and epistemic discrimination. By the former term, they refer to the notion that the optimal achievable performance level may differ between different (e.g., racial or gender-based) groups, corresponding to notions of differing task difficulty. No algorithmic bias mitigation approach can resolve aleatoric discrimination, which is a property of a dataset, not a model. Epistemic discrimination, on the other hand, refers to a model that performs sub-optimally on a given group, compared to the performance level it could achieve on this dataset. Secondly, the authors develop a new way to characterize an upper bound on the Fairness Pareto frontier, i.e., the frontier characterizing optimal trade-offs between overall model accuracy and various fairness constraints. The characterization is based on an old theoretical result by Blackwell and facilitates an efficient implementation by means of a greedy algorithm developed by the authors. This second contribution relates to the first in that it provides a lower bound on the level of aleatoric discrimination in a given dataset. Several numerical experiments in standard (low-dimensional, tabular) algorithmic fairness datasets confirm the validity of the bound and the relative tightness of some previously proposed bias mitigation techniques, emphasizing that significantly better trade-offs are unlikely to be achievable in these datasets. An experiment with artificially induced missing data shows how this increases aleatoric discrimination, as would be expected.
Strengths
The paper makes several highly original and important contributions to the algorithmic fairness field. The framing of different sources of algorithmic discrimination as separated into aleatoric and epistemic discrimination is novel, and it importantly helps in understanding and characterizing the limitations of bias mitigation techniques, which can only ever aim to reduce epistemic discrimination. While similar distinctions have been made before, I find the framing and formulation presented in the manuscript particularly clear and useful. To my knowledge, the manuscript provides the first characterization of an *upper* bound on the Pareto frontier, the latter being an essential object of study in algorithmic fairness. It also treats the general case of both multiple sensitive groups and multiple output labels, which is relatively rare in the field (which mostly focuses on the binary setting for both.) The manuscript introduces a novel theoretical tool, based on the old results of Blackwell, into the field of algorithmic fairness, which may spark further theoretical innovations beyond the present paper. I consider this alone to be an important creative contribution. The paper is very well written, the results are clearly presented, and prior work is comprehensively discussed and acknowledged.
Weaknesses
I do not see many weaknesses in the manuscript, but there is one issue that I think deserves a more comprehensive discussion. This is related to the approximation of g, which maps inputs X to P(S,Y | X), where Y is the output label and S the sensitive group. As I understand it, the correctness of the approximated upper bound depends crucially on the correctness of this approximation. In the simple low-dimensional tabular data cases considered in the manuscript, g can be computed efficiently and precisely, but in the case of more complex (continuous, image?) domains, I would imagine this to be a serious limitation. Do I understand correctly that if g is approximated poorly, the computed bound may in fact not be an upper bound? I would appreciate a more comprehensive discussion of the impact of this approximation on the correctness of the estimated upper bound. Also, can the authors provide some hints at important future extensions of the presented work, or important limitations of the approach? Section 5 is called "Final Remarks and Limitations", but it actually only summarizes the contributions of the present manuscript and lists no limitations.
Questions
1. As outlined in detail above: what are the implications of the approximation on g? How does this impact potential other application domains, such as image analysis? Do I understand correctly that in the tabular cases considered in the paper, g is not approximated but in fact computed exactly? 2. I was a little confused by the notation in some places. For instance, the authors write that they "define FairFront(αSP, αEO, αOAE) as the solution of the following optimization problem", where the following optimization problem optimizes over a classifier h. However, later on, the authors use notation such as "FairFront1(αSP, αEO, αOAE) ≥ FairFront2(αSP, αEO, αOAE)", prompting me to ask: what exactly *is* "FairFront"? Is it the model? Its achieved accuracy? Something else? This could be made clearer by a notation such as "FairFront = arg max ...", for instance. 3. How does the complexity of the algorithm scale with, e.g., the number of sensitive groups to be considered? The authors emphasize that their approach "can be easily extended to the setting where multiple subgroups overlap"; how exactly would this work? In the case of multiple non-binary sensitive attributes, the number of subgroups to consider grows combinatorially; will that present practical problems? 4. To give practitioners some sense of the complexity of Algorithm 1, could the authors provide run time measurements for the case studies?
Rating
8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
4 excellent
Presentation
4 excellent
Contribution
4 excellent
Limitations
1. Like outlined above, the limitations induced by approximating g are presently not obvious to me; I could imagine these to be quite consequential if they effectively limit the applicability of the presented work to low-dimensional tabular data. 2. More generally, the authors do not discuss any limitations of their approach. One limitation that comes to my mind is that it has been shown repeatedly that fairness-accuracy trade-offs may be illusory in the case of group-dependent label noise or selection biases, see, e.g., Blum and Stangl (2020), Wick et al. (2019), Dutta et al. (2020). Another limitation seems to be that the presented approach relies on the availability, correctness, and validity of sensitive attributes, which is often not given; see, e.g., Jacobs and Wallach (2021) and Tomasev et al. (2021). Blum and Stangl (FORC 2020): Recovering from Biased Data: Can Fairness Constraints Improve Accuracy? https://drops.dagstuhl.de/opus/volltexte/2020/12019/pdf/LIPIcs-FORC-2020-3.pdf Dutta et al. (ICML 2020): Is There a Trade-Off Between Fairness and Accuracy? A Perspective Using Mismatched Hypothesis Testing. https://proceedings.mlr.press/v119/dutta20a.html Jacobs and Wallach (FAccT 2021): Measurement and Fairness. https://dl.acm.org/doi/10.1145/3442188.3445901 Tomasev et al. (FAccT 2021): Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communities. https://dl.acm.org/doi/10.1145/3461702.3462540 Wick et al. (NeurIPS 2019): Unlocking Fairness: a Trade-off Revisited. Unlocking Fairness: a Trade-off Revisitedhttps://papers.nips.cc/paper/2019/file/373e4c5d8edfa8b74fd4b6791d0cf6dc-Paper.pdf
Thank you to the authors for responding to my comments. Upon reading the other reviews, I hope that the authors will be very explicit about the limitations of what kind of “fairness” they cover in their work, as you have stated in your rebuttal. I appreciate your detailed response for the comparison to Chen et al. (2018), and hope that some of it can be in the main paper as well. I will keep my rating as it originally is.
Thank you for your prompt response!
Thank you for your prompt response! Yes, we will make sure to explicitly state what kind of "fairness" measures covered in this paper and include a detailed discussion with Chen et al., 2018 in the revised paper. Finally, we would like to express our appreciation once more for the insightful and constructive comments you provided.
A1: Thanks for the clarification. The missing features makes sense in the case of aleatoric uncertainty. Can a link be defined in terms of this uncertainty and Fairfront mathematically? Or is the distribution used as a surrogate to link them? A2: Since you agree that distribution shift is a form of epistemic discrimination, how would aleatoric discrimination be linked to Fairfront in light of this? As per the rebuttal of A1, I am having trouble drawing a formal link between Fairfront and aleatoric uncertainty.
Thank you so much for your response! A1. Below, we revisit the mathematical definition of aleatoric uncertainty according to Section 2.2 on page 463 of [Hüllermeier and Waegeman, 2020]. We also recall the definition of FairFront and contrast the two based on their definitions. ---Given the probability distribution of the deployed data $P_{X,Y}$ and a loss function $\ell$, the Bayes optimal classifier is defined as $$ f^* := \arg\min_f E_{P_{X,Y}}[\ell(f(X)), Y] $$ where the minimization is over all measurable functions $f:\mathcal{X} \to \mathcal{Y}$. Then aleatoric uncertainty is defined as the uncertainty of applying $f^*$ to predict the outcome of a new test point $x_{test}$. ---The Bayes optimal *fair* classifier is a naturally extension of $f^*$ with discrimination control taken into account: $$ f_{fair}^* := \arg\min_{f} E_{P_{X,Y}}[\ell(f(X)), Y]\ s.t. \text{DiscVio}(f) \leq \alpha $$ where $\text{DiscVio}(f)$ measures the discrimination violation of the classifier $f$ (see Table 1 for some examples). The value of $f_{fair}^*$ will inherently depend on the fairness level $\alpha$, leading to a Pareto Frontier of fairness and accuracy, and this frontier is exactly FairFront. Both aleatoric uncertainty and FairFront, by definition, assume full knowledge about the data distribution $P_{X,Y}$. Hence, they are considered irreducible and cannot be diminished by collecting more data. However, incorporating more features into the model can enhance the performance of the Bayes optimal classifier, since you are inherently changing the data distribution by changing $X$. Similarly, if $X$ is observed through a noisy process (e.g., entries of $X$ are erased), then reducing the noise (e.g., number of erasures) – which would again change $P_{X,Y}$ – would affect aleatoric uncertainty. Similarly, it also impacts the fairness-accuracy curve of the Bayes optimal *fair* classifier. Hence, including more features in the model can help reduce both aleatoric discrimination (delineated by FairFront) and aleatoric uncertainty. Regarding missing values, consider the following example directly inspired by the HSLS dataset used in our paper. Data is collected from anonymous student questionnaire answers ($X$) in order to predict student performance and school dropout risk ($Y$). Each questionnaire is assumed to be independent and drawn from the same distribution $P_{X,Y}$. If there are a limited number of students to query, the limited sample size translates to imperfect knowledge of $P_{X,Y}$, and hence epistemic uncertainty/discrimination. This uncertainty could be reduced by querying more students, thus increasing sample size and rendering a more precise estimate of $P_{X,Y}$ and of the best prediction accuracy for a given fairness level. Now, assume that certain students may be reluctant to answer some questions in the questionnaire. For example, students whose parents did not go to college may leave the section on parents' education completely blank. In other words, for each questionnaire, some questions may be left blank, i.e. $X$ may include erased features. This cannot be overcome by querying more students, since missing features are built into the data generating distribution $P_{X,Y}$. In this case, missing values consist of *aleatoric* uncertainty. Of course, we could probe a student and ask them to complete the questionnaire but, since in our example answers are anonymous, we do not have the option. Collecting data from additional students would not resolve this issue, and the missing features are part of the aleatoric uncertainty/discrimination. In summary, it is important to distinguish the sources of missing data in terms of limited samples vs missing features: --- Lack of data due to *limited sample:* $P_{X,Y}$ is not known exactly, leading to epistemic uncertainty/discrimination. In this case, collecting more samples from $P_{X,Y}$ reduces epistemic uncertainty/discrimination since it leads to a more precise estimate of the distribution. –- Lack of data due to *missing features:* the features $X$ can be missing. In this case, since the data generating distribution $P_{X,Y}$ itself may yield missing features, this uncertainty cannot be reduced by drawing more samples from $P_{X,Y}$. Here, missing features are part of aleatoric uncertainty and discrimination. **(continued in the next comment ...)**
A2. In response to your question, we provide the following example to link distribution shift with aleatoric discrimination. Consider a bank that enters a new market and finds that its credit score system underperforms for customers aged over 60. This discrepancy may be due to the credit score model being trained on a dataset from a market with a different distribution, perhaps influenced by local income levels. To mitigate this bias, the bank could gather more data from the new market and apply post-processing fairness interventions to its credit scoring system. Since collecting more data from the new market improves accuracy and fairness, the distribution shift posed by the new market consists of an epistemic source of discrimination. However, collecting new data would not address aleatoric discrimination. Aleatoric discrimination presumes full knowledge about the data distribution in the deployment environment. For example, say that in the new market customers in group A always pay-off their loans, whereas customers in group B always default. In this case, achieving statistical parity in credit scoring across both groups would require compromising overall accuracy. This is due to the distribution of the data in the new market, and simply adding more samples from the new market wouldn't resolve this trade-off. Hence, it consists of aleatoric discrimination. We will add the above discussions to the paper and are happy to answer any follow-up questions.
The idea of finding an upper bound on the pareto frontier is novel and new. However, the way aleatoric and epistemic uncertainty are divided needs more attention and the link appears to be handwavy. For aleatoric uncertainty, while it is true, missing features can be a cause, there are also other factors, such as latent noise in the measurement. The overall idea is great but a bit more is needed to draw the connection between uncertainty types and Fairfront. Adding more details: Since the upper bound is theoretical, the uncertainties have to also be formally linked to them. The danger of using examples to link theoretical concepts is that they do not cover all use cases e.g. missing features are not the only source of aleatoric uncertainty. Is the epistemic uncertainty counterpart of the bound really epistemic and is the aleatoric one really aleatoric? The handwavy part is the missing solid formal link which would ideally be a parametric equation consisting of FairFront into its aleatoric and epistemic counterparts. How are the two disentangled? That is also an open problem which leads to more questions. Is the disentanglement problem also solved by this work? Final Verdict: borderline reject
**Disentangle aleatoric and epistemic discrimination via their mathematical definition** Thank you for your comments. We provided illustrative examples in our previous response to clarify the difference between aleatoric and epistemic discrimination – we misunderstood your point of being qualitative. We reiterate below the precise mathematical definition of aleatoric and epistemic discrimination to differentiate these two concepts. [**Aleatoric discrimination**]. For a given data distribution $P_{X,Y}$, (group) fairness metrics of choice (e.g., statistical parity and equalized odds), and discrimination violation threshold $\alpha \geq 0$, we quantify aleatoric discrimination by $1 – FairFront(P_{X,Y}, \alpha)$, which is $$ 1 - \max_{f} E_{P_{X,Y}}[\mathbb{I}(f(X) = Y)]\ s.t. \text{DiscVio}(f) \leq \alpha $$ Here the maximization is over all measurable functions $f:\mathcal{X} \to \mathcal{Y}$. In other words, aleatoric discrimination quantifies the minimal achievable loss under discrimination control over all possible classifiers. This only depends on the underlying distribution $P_{X,Y}$. As mentioned above, this aligns with the definition of aleatoric uncertainty in [Hüllermeier and Waegeman, 2020]. By its definition, aleatoric discrimination is irreducible by adding more data (as it already assumes perfect knowledge of the data distribution $P_{X,Y}$) or altering model class and learning algorithm (as it optimizes over all measurable functions). It is an inherent property of the distribution $P_{X,Y}$. [**Epistemic discrimination**]. For a fairness-intervention algorithm $\mathcal{A}$ that takes a training set $\mathcal{D} = \set{(x_i, y_i), i=1,...,n}$ as input and outputs a classifier $h$ from a hypothesis class $\mathcal{H}$, we quantify epistemic discrimination by comparing FairFront with the discrimination violation and accuracy of $h$: $$ FairFront(P_{X,Y}, \alpha) – Acc(h) $$ where $\alpha = \text{DiscVio}(h)$. In other words, epistemic discrimination measures the difference between the (accuracy, discrimination control) attained by the classifier from a fairness intervention and the optimal achievable one, delineated by FairFront. Note that training and deployed data may be drawn from different probability distributions (i.e., the generating distribution of $(x_i, y_i)$ is different from $P_{X,Y}$). Hence, distribution shift is a form of epistemic discrimination. Some other sources of epistemic discrimination include but are not limited to the choice of fairness-intervention algorithm, number of data, and the choice of model class. In general, epistemic discrimination arises from a lack of knowledge about the optimal “fair” model and can be eliminated in the limit of infinite data from the true (test) distribution $P_{X,Y}$ (see Figure 5 in Appendix). We will add this to the revised paper. ----- **Different sources of aleatoric discrimination** We can apply the aforementioned mathematical definitions to investigate various sources of aleatoric discrimination. In this context, we introduce a new theorem that sheds light on some of these sources. Theorem. Suppose that $X_{obs} = g(X)$ where $g$ can be any (potentially randomized) function. Then $FairFront(P_{X_{obs}, Y}, \alpha) \leq FairFront(P_{X,Y},\alpha)$ for all $\alpha \geq 0$. This theorem can be proved by the data processing inequality. This theorem shows that aleatoric discrimination can arise if the observed variable $X_{obs}$ is a noisy observation of $X$. The function $g$ in the above theorem can be an erasure channel that mask feature values according to a certainty probability (i.e., missing features or unobserved predictive variables) or introduce noise to the input features (i.e., noisy measurements as you mentioned). Above we only provide two sources of aleatoric discrimination. In general, aleatoric discrimination is influenced by the properties of the data distribution, so any action that might alter this distribution can have an impact on aleatoric discrimination.
I appreciate the detailed response. I have read the other reviews and associated rebuttals. I stand by my score.
Thank you so much for your response. We will make sure to include the promised changes in the revision (both in the main text and appendix).
I would like to thank the authors for their detailed and informative responses to my questions. In light of these, and also considering the other reviews, rebuttals, and discussions, I am updating my score to Strong Accept. One final remark concerning additional related work: it could also be interesting to mention the recent branch of the literature discussing omnipredictors. E.g., Globus-Harris et al. (2022) and Hu et al. (2023) discuss how and under which conditions Bayes-optimal fair classifiers can be derived using simple post-processing techniques from multicalibrated regressors. https://arxiv.org/pdf/2209.07312.pdf https://proceedings.mlr.press/v202/hu23b.html
Thank you so much for your response and providing further related literature – indeed, this line of research is very relevant! We will expand the first paragraph of our Related Work section to highlight the work on omnipredictors and multicalibration, including the ones you mentioned, and their connection with Bayes-optimal fair classifiers. In our final section, we will also point to the burgeoning literature on omnipredictors and multicalibration as a source of new strategies for producing fair classifiers with theoretically-backed performance guarantees.
Response
Thank you for your response. Q1. I appreciate the clarification regarding the strategic choice in choosing commonly used datasets (e.g. Adult) to highlight the limitation of studying these datasets given the FairFront bound. I don't think this messaging comes across in the current paper draft, however -- the focus of the paper seems more of a critique of broader approaches in the community to fairness interventions, rather than a specific critique of these simple datasets. If anything, I would thinking experiments with a more complex dataset and highlighting differences in results would drive home this point more. So I would encourage the authors to think more about the framing. Q3. The purpose of this comment was more to highlight that there exist proposed methods in the literature that are *training* time interventions, not just dataset-level operations -- the current text seems to imply addressing unequal dataset issues is impossible via the training method, but modifies loss functions can capture this. I have updated my score, but would strongly encourage the authors to explicitly state their choice to use their datasets in a revised version.
Thank you so much for your response and follow-up comments! Q1. Absolutely. In the revised paper, we will clearly state our main message: there are diminishing returns in benchmarking new fairness interventions on standard (overused) datasets as existing methods are approaching the information-theoretically optimal Pareto frontier delineated by FairFront. Additionally, we will articulate the rationale behind selecting commonly used datasets for our experiments. Q3. Thank you again for highlighting this line of work. We will clarify our scope more clearly in the revised paper.
Decision
Accept (spotlight)