Unprocessing Seven Years of Algorithmic Fairness

Thank you for your encouraging review and valuable feedback. We address each question in the following paragraphs. **Q2** > Have you considered putting your approach into a popular package so that researchers can quickly and easily compare their models? Thank you for bringing this up. We've open-sourced our implementation in a standalone python package (link to [anonymized repository](https://anonymous.4open.science/r/error-parity-8550/README.md) in the paper). Our implementation is easy-to-use and compatible with any score-based classifier (examples [here](https://anonymous.4open.science/r/error-parity-8550/examples/relaxed-equalized-odds.usage-example-folktables.ipynb)). Additionally, to reach a potentially wider audience, we have open Pull Requests to include our implementation on a popular algorithmic fairness python package (will be linked in the paper after deanonymization period), although this will expectedly still take some weeks of work to be made compatible with that package's API and structure. **W1** Thank you for raising this concern. Achieving error-rate parity was detailed by Hardt et al. (2016). The relaxed solution is more nuanced (since we can no longer rely on the intersection of all group-specific ROC curves), but the key idea is similar: using randomized thresholds allows us to access the convex hull of each group-specific ROC curve (while deterministic thresholds only allow us to access specific discrete ROC points). This makes it so the optimization domain is convex (the group-specific ROC convex-hulls), and hence easy to optimize over. While this is an efficient solution to the problem, a simple brute-force approach would also be possible (brute force implementation example [here](https://anonymous.4open.science/r/error-parity-8550/examples/brute-force-example_equalized-odds-thresholding.ipynb)). As linear optimization has been widely studied in the literature, we cite reference works such as Boyd and Vandenberghe (2004) for details on how to solve the LP. According to reviewer feedback, we were in fact torn between keeping this section in the paper or in the appendix, but as per your feedback we will maintain it in the main paper body. **W2** Thank you for raising this point. Appendix B details the infrastructure we used to conduct our experiments. Note that the run-times in Fig. 5 correspond to model training times using a single CPU-core per job (to allow for high job parallelization and a more efficient use of CPU nodes). We do believe that the variety of datasets and models tested significantly contributes to the generalizability of our findings. For full transparency, we will further detail the compute usage of our experiments in Appendix B. **W3** We agree that our approach is definitely intuitive. At the same time, the algorithmic fairness literature contains a wide range of fairness-aware inprocessing and preprocessing methods that claim to dominate results achieved by a simple postprocessing baseline. In our work, we show that this outperformance can be due to (1) comparing models at different levels of constraint violation, or (2) postprocessing a lower-performance model; both are arguably instances of unfair evaluation standards. Acting on each stage of the ML pipeline undoubtedly has specific advantages (e.g., preprocessing is potentially compatible with any downstream task/model, and can be useful when you have to hand over the data to a third party). With this meta study, we hope to have made the advantage of postprocessing clear: given accurate risk-score estimates, postprocessing can retrieve any optimal fairness-accuracy trade-off.

Thank you for your encouraging review and valuable feedback. We address each question in the following paragraphs. **Q1** Thank you for bringing this up. The LP formulation is actually paramount to the usefulness of our method, as an exhaustive search over all threshold combinations scales exponentially with the number of groups, and scales quadratically with the number of thresholds that the underlying predictor accepts. To clarify, an exhaustive search would have to span all combinations of group-specific _randomized thresholds_ in order to access the interior of each group's ROC curve (by using deterministic single-value thresholds we can only access specific discrete points in the exterior of each group's ROC curve). That is, each group’s decision function is represented by 2 values, $\{(\underline{t}_s, \overline{t}_s) : (\underline{t}_s, \overline{t}_s) \in \mathcal{T}^2 \land \underline{t}_s \leq \overline{t}_s\}, \mathcal{T} \subseteq \mathbb{R}$. Samples of group $s$ whose score is lower than $\underline{t}_s$ are classified negatively ($\hat{Y}=0$), samples whose score is higher than $\overline{t}_s$ are classified positively ($\hat{Y}=1$), and samples whose score is in the range $\left[\underline{t}_s, \overline{t}_s\right]$ are classified randomly by a coin toss [Hardt et al., 2016, Section 3.2]. For example, a coarse search grid over deterministic thresholds of $\mathcal{T} = \{0, 0.1, 0.2, ..., 1.0\}$, includes randomized thresholds $\{ (0.0, 0.0), (0.0, 0.1), ..., (0.0, 1.0), (0.1, 0.1), (0.1, 0.2), ... \}$, a total of $\frac{\left|\mathcal{T}\right|(\left|\mathcal{T}\right|+1)}{2}$ combinations (scales quadratically with $\left|\mathcal{T}\right|$). Perhaps more importantly, if $\mathcal{A}$ is the set of randomized thresholds, the search space will span $\mathcal{A}^{|\mathcal{G}|}$ different threshold combinations, where $|\mathcal{G}|$ is the number of sensitive groups. We've implemented an exhaustive-search solver and **added an example notebook** to the examples folder in the anonymized repository linked in the paper ([link here](https://anonymous.4open.science/r/error-parity-8550/examples/brute-force-example_equalized-odds-thresholding.ipynb)). Running an extremely coarse grid of $|\mathcal{T}|=11$ thresholds on our main experiment ($|\mathcal{G}|=4$) leads to over $9\text{M}$ combinations. With our implementation, a small experiment using $|\mathcal{G}|=2$ and $|\mathcal{T}|=8$ takes 3 minutes to run over the $4356$ combinations with an exhaustive-search solver, while the LP solver takes 109ms to achieve a superior solution (because the search grid is finer). **W1** Thank you for this suggestion. We've **added these comparisons in a new Appendix A.6** to the latest paper revision. Namely, we compare fairness and accuracy results for postprocessing and fairness-constraining the same model class (e.g., FairGBM compared to postprocessed GBM), or unprocessing and unconstrained training of the same model class (e.g., unprocessed FairGBM compared to GBM). Please let us know of any further suggestions you'd find beneficial for the presentation of our work. **W2** We agree that our most significant contributions are empirical, and describing the LP is not particularly novel. However, as all our findings rely on the proposed relaxed postprocessing method, we believe that a clear definition of this optimization problem should be in the body of the paper; additionally clarifying how exactly relaxed postprocessing differs from strict postprocessing (whose LP solution is detailed by Hardt et al. (2016)). We admit that a balance between lack of context and unnecessarily detailed explanations is hard to strike, as other reviewers even suggest that this section should be lengthened. --- Thank you for your insightful recommendations. We believe the latest paper additions have definitely improved our work.

Thank you for your encouraging review. We are very glad to know that you appreciate the significance of our work.

Thank you for your review and valuable feedback. We address each question in the following paragraphs. --- > According to Equation 1, unprocessing starts from the postprocessed Ŷ and aims to find the unconstrained optimized predictor. How can we do that with obliviously postprocessed Ŷ? Thank you for raising this question. The postprocessing procedure we propose is analogous to that of Hardt et al. (2016), only with a partially relaxed fairness constraint. Specifically, both procedures operate on _the scores_ of a score-based predictor, by finding the optimal group-specific decision-boundary (minimal loss while fulfilling the constraint). As such, these procedures are "oblivious", in the sense that they are functions of the joint distribution of $(Y, A, R)$, and do not consider the features $X$ directly. > How to make sure the unprocessed predictor has a sensible mapping from input features to target variable? Unprocessing (or, more generally, postprocessing) does not consider the features, only the scores of the underlying predictor. If the underlying predictor does not sensibly map features to the target variable, then postprocessing that predictor will expectedly have very poor results. For this reason, to unconfound our results with the performance of the base model, we pick the predictor whose scores can achieve the highest accuracy ($m^*$). > (...) I am not sure how to understand the relation between unprocessing and postprocessing. > > (...) provide a clear characterization of the relation between unprocessing and such definition of postprocessing, so that readers can understand why unprocessing is a helpful analyzing tool to understand the importance of postprocessing. The proposed postprocessing method is solely a more general version of the method of Hardt et al. (2016) that is now compatible with relaxed fairness constraint fulfillment. We call _unprocessing_ to the specific case of postprocessing where the fairness constraint was infinitely relaxed, $r=+\infty$. That is, unprocessing boils down to minimizing each group's loss over each independent group threshold (finding which point in the group's ROC achieves minimal loss). Strict postprocessing will map an *unconstrained* score-based predictor to a *fairness-constrained* classifier. We use the term "unprocessing" to intuitevely capture the reverse procedure: mapping a *constrained* score-based predictor to a *fairness-unconstrained* classifier. Of course, either procedure can be applied to any predictor, constrained or unconstrained alike, but applying unprocessing to an unconstrained predictor will expectedly not significantly change its accuracy/fairness (discussed in the last paragraph of Sec. 2.2). To clarify with an example: given a standard unconstrained classifier (e.g., GBM), strict postprocessing will map it to the fairness-accuracy space of constrained classifiers (e.g., FairGBM); on the other hand, given a constrained classifier such as FairGBM, _unprocessing_ will do the inverse mapping, outputting a classifier that will approximately sit in the fairness-accuracy space of unconstrained GBM models. We have added a **new Appendix A.6** with detailed comparisons between unprocessed models that were trained in a fairness-constrained manner, and standard unconstrained models; as well as comparisons between postprocessed models that were trained in an unconstrained manner and inprocessing fairness-constrained models. Hopefully this will further clarify the motivation behind unprocessing and postprocessing. --- Thank you for your feedback. Please let us know if your questions were addressed, or of any further questions you may have.

Dear AC. We've just submitted detailed responses to all reviewers, together with an updated paper PDF. The updated appendices required running some extra results that took longer than expected to finish. We apologize for any inconvenience.