Auditing Fairness by Betting

Summary

The paper proposes a safe anytime-valid inference (SAVI) testing procedure to audit the potential unfairness of a classifier or regression model. The paper posits group-conditioned fairness as a test of equality of means under some finite set of conditions, this definition includes several common fairness notions such as equality of odds and equality of opportunity. In this setting, the authors propose a technique allows for sequential testing of equality of means that can accommodate repeated testing, finite or infinite testing time horizons, batched testing, and importance weighted testing. The authors also provide expected stopping times for their testing procedure.

Strengths

The paper is well written and structured. All the necessary information to understand the paper's contributions is provided within the main text. Several practical considerations (time horizon, batched testing, importance weighting) are clearly outlined and addressed. The method itself is extremely simple to implement.

Weaknesses

Overall the paper is very solid. The main weakness I see is that there is no immediate way of extending these ideas to settings where equality of means may be too strict of a definition, and instead one would want to test that the absolute difference of means falls below some epsilon value.

Questions

See weaknesses above.

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

Yes

Summary

This work introduces practical and efficient methods for auditing the fairness of deployed classification and regression models. The methods are sequential, allowing for continuous monitoring of incoming data, and can handle distribution shifts resulting from changes in the model or the underlying population. The approach is based on recent advancements in anytime-valid inference and game-theoretic statistics. The methods are demonstrated to be effective through experiments on benchmark fairness datasets.

Strengths

Originality: The concept of developing a system that can continuously monitor and track the fairness of deployed real-world systems in a sequential evaluation context is a novel contribution. The idea of extending fairness evaluation beyond the development phase is an innovative approach. Quality: The paper demonstrates a high level of quality, with well-founded theoretical reasoning and rigorous analysis. The authors present a solid theoretical foundation that supports their proposed methods. Clarity: The paper is written in a clear and understandable manner. The research problem is effectively formulated, and the theoretical foundations are explained comprehensively. The clarity of the paper contributes to its accessibility and readability. Significance: Ensuring fairness in systems goes beyond the initial testing phase and requires continuous monitoring of deployed models. The significance of the paper lies in its contribution to the field by providing a way to quantify and evaluate the fairness of deployed models. This capability is becoming increasingly important as the need for ongoing monitoring and accountability in real-world systems grows. The paper's focus on continuous fairness monitoring and its impact on the research community make it a valuable contribution to the field.

Weaknesses

It would greatly enhance the paper's findings to include more than two datasets for result verification. Additionally, since only one model (Random Forest) was trained on the dataset, it would be intriguing to explore the effects of different fairness methods, such as in-processing, post-processing, and preprocessing approaches, by testing multiple models. By incorporating these suggestions, the paper's results would become more robust and comprehensive, providing a more thorough understanding of the proposed methods and their performance across various datasets and models.

Questions

I would like to discuss a thought experiment regarding a fair model that exhibits some degree of non-determinism. In this scenario, the model generally produces fair outputs for a given instance, but with a certain probability (e.g. 10%) the same instance would have an unfair output. My intention in raising this thought is to gain insights into the robustness and reliability of your framework. I am curious to understand whether such non-deterministic behavior would impact or challenge the effectiveness of your proposed framework.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

I acknowledge that cost considerations and reaching a sufficient amount of evidence to make a decision are factors that may lead to the discontinuation of data collection. However, it is crucial to recognize that the fairness of a deployed system can evolve over time, necessitating continuous evaluation to ensure ongoing fairness. In addition, the emergence of regulatory frameworks like the AI Act in the EU emphasizes the importance of continuous evaluation, as it may become a legal requirement. These factors highlight the need for continuous monitoring and evaluation to address fairness concerns and comply with legal obligations. So even if the null is rejected at some other time in the future this could not be seen any more. How does the framework could handle this? Is this a limitation?

Summary

This work proposes an approach based on recent results of super-martingales to detect when a certain predictor is demographically unfair. Studying different formulations of group fairness (such as equal opportunity, demographic parity, etc), the authors frame the problem as that of sequential hypothesis test, where a null corresponding to equalized metrics across groups. The problem is then to reject the null, if the alternative is true, as soon as possible while controlling for type-1 error. Because this is set up as a sequential hypothesis test, this implies strong conditions on the probability of false positive (i.e. for any future time). Relying on the idea of “betting”, the authors then design a pay-off function that allows the “payoff” to be a super-martingale under the null (and remain controlled), whereas the payoff grows exponentially under the alternative. While the authors derive their methodology a a “basic”/standard, they further comment on how the same approach can be minimally modified to accommodate for data collected at different rates from different demographic groups, with different data-collection policies (not iid), and under “distribution shifts”. They finally showcase their method in a couple of real world datasets, comparing with relatively simple alternative.

Strengths

- The studied problem is quite important and timely, given the increasing use of predictive algorithms in society. - The proposed solution is relatively simple and elegant, provably correct, and seems effective in practice. - This paper is beautifully written, and it was a pleasure to read. - The experiment results are very nice and easy to understand

Weaknesses

- The experiment results do not seem to address the time-varying data collection policies setting.

Questions

1. A potentially trivial question is how to employ the same ideas to not only equal opportunity but also to equalized odds (which requires not only equal TPR but also FPRs to be the same). Is it true that in this case one can simply generalize the null hypothesis as one where both rates being the same, and the results would naturally follow? A footnote on this might be useful. 2. While their approach seems to work quite nicely in practice, I wonder about the tightness of Proposition 1 (and its corollaries that follow). What are the typical constants associated with this bound? Moreover, do the authors think it would be possible to obtain a lower bound on $\mathbb E[\tau]$ as a function of $\Delta$? 3. As far as I understand, the experiments section did not showcase their results on being able to incorporate different data-collection policies (Sec 3.2). Could the authors comment on why this is the case? Even if the data they used involved static policies, could they simulate different policies a posteriori to showcase their results?

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

The results all hinge on the auditing process being conducted properly. The paper realizes this and appropriately mentions it in the limitations section.

Summary

This paper clarifies the issue of fairness auditing, wherein an auditor verifies the fairness of a system through sequential interaction with it. The goal is to develop a methodology that facilitates anytime-valid tests of system fairness. Essentially, this involves devising a sequence of test statistics for the null hypothesis (i.e., the system is fair), enabling the auditor to conduct statistical tests using these statistics at any given stopping time. To achieve this, the authors design a payoff for the invalidity of observed information against the null hypothesis. A sufficiently high value of this payoff can safely lead to the rejection of the null hypothesis. Both theoretical and experimental analyses of the proposed method substantiate the efficacy of this sequential fairness testing approach.

Strengths

This is a well-written and fascinating paper. Recently, the problem of fairness auditing has garnered attention, driven by societal demands for fair systems incorporating machine learning techniques. Sequential testing of fairness has become an essential tool for such audits. This paper delivers a robust technical solution for sequential fairness testing, offering impressive theoretical and empirical analyses of its performance. The principal technical innovation is the reduction of the fairness auditing problem into a two-sample testing problem. This transformation facilitates the application of the established and theoretically sound methodology of anytime-valid two-sample testing. The empirical experiments conducted in this paper effectively illustrate the potential for achieving higher power with fewer interactions, all while controlling the Type-I error in comparison to the permutation test. This evidence underscores the practicality of the proposed method.

Weaknesses

A notable weakness is the limited innovation in methodology. The construction of the testing algorithm is merely an application of the anytime-valid two-sample testing developed by Shekhar and Ramdas. Additionally, even though Shekhar and Ramdas did not account for adjustments to time-varying data collection and distributional shifts, such an extension might be relatively straightforward. In the context of time-varying data collection, the authors make the assumption that the density of the covariate distribution is readily accessible. This assumption, however, isn't practically feasible. Therefore, it would be advantageous to address the limitations of this assumption more thoroughly.

Questions

The existence of group-wise streams may not be feasible in practice. In a typical scenario, an auditor might receive a sequence of i.i.d. data drawn from a composite distribution that combines the group-wise distributions. Therefore, it would be more beneficial to illustrate the application in a situation where a single, mixed stream is present.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

There are no specific limitations or potential negative social impacts that need to be addressed in relation to this paper.

Questions and concerns addressed

I thank the authors for addressing my questions and comments.