Recent discoveries have revealed that deep neural networks might behave in a biased manner in many real-world scenarios. For instance, deep networks trained on a large-scale face recognition dataset CelebA tend to predict blonde hair for females and black hair for males. Such biases not only jeopardize the robustness of models but also perpetuate and amplify social biases, which is especially concerning for automated decision-making processes in healthcare, recruitment, etc., as they could exacerbate unfair economic and social inequalities among different groups. Existing debiasing methods suffer from high costs in bias labeling or model re-training, while also exhibiting a deficiency in terms of elucidating the origins of biases within the model. To this respect, we propose a fast model debiasing framework (FMD) which offers an efficient approach to identify, evaluate and remove biases inherent in trained models. The FMD identifies biased attributes through an explicit counterfactual concept and quantifies the influence of data samples with influence functions. Moreover, we design a machine unlearning-based strategy to efficiently and effectively remove the bias in a trained model with a small counterfactual dataset. Experiments on the Colored MNIST, CelebA, and Adult Income datasets along with experiments with large language models demonstrate that our method achieves superior or competing accuracies compared with state-of-the-art methods while attaining significantly fewer biases and requiring much less debiasing cost. Notably, our method requires only a small external dataset and updating a minimal amount of model parameters, without the requirement of access to training data that may be too large or unavailable in practice.
Paper
Similar papers
Peer review
Summary
This paper proposes an approach to removing biases from trained models without necessitating full retraining and without necessitating access to a large dataset that has annotations w.r.t biases. The proposed method consists of 3 steps: firstly, identifying which attributes are ‘biased’ (according to a counterfactual definition of bias: the more the predictions differ between the factual and counterfactual examples, the more biased the model is). Second, identify which examples are those contributing to biased behavior, using influence functions. Finally, once the (top K) harmful samples are identified, they run an unlearning procedure (via a Newton step) to remove the influence of those examples.
Strengths
- This paper tackles an important problem and provides useful motivation and intuition for understanding the issues with biases in neural networks - The proposed approach makes fewer limiting assumptions than some of the previous methods, and the authors show that empirically, it outperforms certain baselines on certain metrics on some (simple) datasets / models - The paper is well-structured and mostly clear, with some exceptions (see weaknesses section below)
Weaknesses
A) Clarity can be significantly improved in some sections. Some particular examples: a) Some imprecise statements, e.g. what does it mean to be “closely tied” (line 136) b) In line 158, what is z_k? Should that have been z_i? (i is the variable that the sum is over, but it’s not used, unless I’m missing it?). Also, need to define n in that equation. c) In Equation 3 (which defines the counterfactual bias), how do we choose the value to modify a_i with? Does the equation need some expectation over which alteration to make to the affected attribute? d) in the “Biased-effect evaluation section”, the authors say that they investigate how the bias is learned. More precisely, though, what they do is estimate the influence of specific training examples, so a more appropriate description is to identify which examples lead to learning the bias, not *how* the bias is learned. e) What does “up” mean / stand for, in e.g. Equation 6. f) Generally, the Bias-effect evaluation paragraph is hard to understand without a background in influence functions. It would be great to give more intuition around equations. g) In the experiments section, the authors don’t describe the baselines they use, making it harder for someone unfamiliar with this literature to assess their appropriateness and the value of the associated comparisons B) [Terminology] Unlearning refers to removing the influence of a subset of training examples from a trained model. Therefore, doing “unlearning” on external samples (e.g. from a test set) doesn’t make sense according to that definition. Would be useful to clarify this point and perhaps use a different term instead. Debiasing sounds like a more appropriate word to me rather than unlearning (though inspiration may be drawn from unlearning methods). C) Issues with the experimental setup: a) there are no confidence intervals in the tables, making it hard to assess whether the proposed method’s improvement over the considered baselines in each case is statistically significant. Sometimes the results are quite similar. b) In Table 1, why is DRO excluded from comparisons? In Table 3, why don’t the authors also show the Unb. and Wor. columns, as in Table 2? Those seem like a more direct and appropriate way to assess the degree of biasing in a way that actually matters / affects the model’s behavior, rather than only looking at the Bias column. c) Overall, the experiments are conducted on simple datasets with relatively small neural networks, and it’s unclear how this affects the conclusions. For instance, it’s surprising that unlearning only the top layer is successful, as in general this is not a good unlearning method (examples typically influence the trained model throughout layers, in a highly non-convex manner) d) It would also be nice to compare against the oracle for unlearning, to get an upper bound. Specifically, once the harmful examples are identified, the oracle for unlearning would simply remove them from the training set and retrain the model without them (this of course only applies to the case where unlearning is done using training examples, as is the setup in unlearning). How well does the proposed approach debias, compared to this oracle for the unlearning step? e) Looking at Table 5, counterintuitively, Eq. 9 (“unlearning” using external samples) leads to smaller bias than Eq. 7 (unlearning using training samples). This seems to contradict the narrative that the former should do worse than the latter but is adopted instead due to not wanting to assume access to the training set? Do the authors have an explanation for this? Or is it the case that all these results are statistically indistinguishable from each other? (again, there are no confidence intervals in the tables, making it hard to tell.)
Questions
- How did the authors pick the value of K (number of harmful samples to unlearn)? Is this different for each dataset / setting? - When computing the time (for quantifying the efficiency of the method), which steps are taken into account for the proposed method? Are all steps 1-3 counted towards this? If not, why not? Is this done fairly w.r.t how it’s counted for other methods? - Follow-up: what is the computational complexity of running the proposed method, in all 3 steps? Computing Hessians is expensive, and it sounds like the proposed method is to compute the influence of each example, in order to then rank them and find the top K. - How well does the method scale to deeper neural networks? All architectures used in the experiments are very small. - When using an external dataset for “unlearning”, how similar does that dataset need to be to the training set, in order for that to be beneficial? What if the external dataset is distributionally-shifted in different ways? - Could some of the baselines considered also be (easily) modified to use external datasets and / or fewer samples?
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
2 fair
Presentation
3 good
Contribution
2 fair
Limitations
- The proposed method requires datasets with access to attributes, so that counterfactuals can be created by e.g. changing the value of one attribute while keeping the values of other attributes fixed. But in practice, we typically don’t have attribute information for datasets of interest.
Summary
The paper addresses the issue of biases in trained models, which are often caused by imbalances in the dataset used for training. The conventional approach to mitigating biases involves finetuning the model, but this incurs additional training costs. The paper proposes an alternative method called Fast Model Debiasing (FMD) that does not require finetuning. The FMD method consists of three steps: identifying the bias attributes, identifying the top samples that contribute to the bias, and removing the bias. The key contribution of the paper is the use of Influence Functions (IF) and its extension to identify and eliminate biases. This approach is interesting and appears to be reasonable.
Strengths
The main contribution of the proposed method is to leverage the IF (Implicit Fairness) technique and its extension to identify and remove bias in models. This approach offers a practical solution for making models debiased. Compared to typical fine-tuning solutions, this method avoids the need for additional training effort, making it a more efficient procedure. By directly removing bias through the IF technique, the model can be made fairer without requiring extensive retraining. Furthermore, this method can be implemented as a separate API, allowing for easy integration into existing systems. This API can be used to remove bias from models, ensuring fairness in various applications.
Weaknesses
Here, the general idea is simple: to identify the samples, and further enhance the model to debias. More experiments should be added to verify this. refer to the following: limitation parts.
Questions
In the approach, several layers in the model are updated based on certain rules to remove the bias. To further extend this work, it would be interesting to explore which modules are sensitive to bias and contribute to the debiasing process. For example, when considering model adaptation during the inference stage, we can optimize the Batch Normalization (BN) module by re-estimating the mean and variance based on the fresh samples from new scenarios in just a few steps. HEre, BN is the sensitive for model adaptation. Back to the debias, which module is the key? There are some other questions: 1. Select K top harmful samples. what is the suggested value for K? 2. Such harmful samples highly depends on the testing samples. what is such testing samples from? -if testing samples is from one subset of the training/validation set, how does the proposed not involve new bias after removing the influence of K top harmful samples? 3. Not find the bassline accuracy of the model without the debias, and how to guarantee that the updated parameters would not impact the model accuracy? 4. Have you tried other complex or big model?
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
2 fair
Contribution
3 good
Limitations
1. In the proposed FMD, based on the Newton update, the model is updated. The model accuracy generally would be impacted. But, most of the given experiments show that both of accuracy and debias performance are best. Such results might not be convinced. All of the experiments have been conducted using small or simple models on easy tasks. It is important to verify whether such methods can be applied to other typical or complicated tasks/datasets, such as ImageNet/COCO or segmentation tasks. Please enable more experiments or cite more other reference. * Firstly, add one line or column to show the baseline performance, e.g., the accuracy without the debias. * Secondly, extend to more scenarios, e.g., imageNet dataset. * Thirdly, maybe one flow figure is helpful to understand the overall FMD procedure. * Model debias is partially related to other topics, e.g., OOD, model, domain adaptation or others, where some debias method are also introduced. if time and space are ok, survey and add more reference. 2. Implementing such a scheme requires building a specific dataset (even if it is small) and updating the model layers, which can be a time-consuming process. It might not be easier than performing a quick finetuning for debiasing. Therefore, it would be beneficial to compare the complexity between the proposed approach and quick finetuning, as well as evaluate their performance. * In general, quick finetuning does not necessarily require a larger dataset. A small specific dataset with data augmentation might be sufficient for finetuning to address bias.
Summary
In this paper, the authors propose Fast Model Debiasing (FMD), which targets solving biased prediction problems. The proposed FMD can effectively and efficiently remove model bias by leveraging only a small external dataset by altering the attribute, then evaluating the biased-effect of each sample, and finally using the most harmful samples to unlearn the model bias with the Newton update step. Experiments on Colored MNIST, CelebA, and Adult datasets demonstrate the effectiveness of the proposed method.
Strengths
The motivation and the idea proposed in this paper are reasonable. The unlearning progress is efficient since the model parameters only need to be updated in several steps. The execution time is greatly shorter than the compared methods. The experimental results are sufficient to support the effectiveness of the proposed method.
Weaknesses
1. Some writing issues. "Fast Model Debias" in the title, but "fast model debiasing" in the abstract. The full name of "FMD" is not explained when it first appeared in Sec 1. 2. In Eq 3, reduant ")". 3. Some usage of notation is not standard. For example, the function "B" accepts three kinds of input. 4. In line 231, the notation $C_i$ should be $c_i$. 5. The number of K when selecting harmful samples, and the bias threshold seems not explained in the experiments.
Questions
N/A
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
2 fair
Contribution
3 good
Limitations
N/A
Summary
This paper presents a novel method for model debiasing, based on the techniques of counterfactual biases and machine unlearning. The method designs an all-inclusive debiasing pipeline: bias identification, biased-effect evaluation, and bias removal. The proposed method can get rid of time-consuming human labeling or re-training, which is appealing to practical applications. Experimental results and ablation analysis suggest that the proposed method is effective and more efficient on multiple datasets for fairness evaluation. Overall, I appreciate the idea of using machine unlearning and counterfactual samples for efficient model debiasing, though some concerns still need to be addressed for improved quality.
Strengths
Strengths: 1. Model debiasing and fairness are critical for the deployment of large AI models in practice. The proposed method can identify and remove the bias in a cost-effective manner, which is more suitable for practical applications than previous retraining methods. 2. The idea is novel and interesting. Employing counterfactual samples to identify biases seems quite helpful. Leveraging the machine unlearning techniques with counterfactually constructed datasets for bias removal shows a new promising direction for efficient model debiasing. 3. The sanity check is clear, experiments in the main paper and appendix are quite comprehensive, and the performance in terms of accuracy, fairness and efficiency are promising.
Weaknesses
1. The post-processing baselines are included on the Colored MNIST. I suggest including them for other datasets, i.e., CelebA and Adult. 2. More discussions on how the proposed method can be generalized for large-scale networks or tasks. Existing fairness studies are mainly conducted on small datasets like C-MNIST and networks such as MLP/ResNet. It would be great if the authors can investigate how the proposed unlearning method can be extended to large-scale datasets or models, for examples, is it possible to extend this idea to large-scale language models. Even some preliminary results or discussions would be highly appreciated. 3. Besides quantitative results in Table 1-3, more straightforward visualizations of experimental results on how the proposed method can give rise to better fairness (less bias) are highly encouraged to better verify the effectiveness.
Questions
See weakness. Minor issues: 1. Writing need to be improved, e.g., typos and grammar errors: line 40 involves (involve), line 35 “or”, etc. 2. Paper title and manuscript title are not the same. Title should be "Fast model debiasing with machine unlearning"? 3. What are the definitions of Demographic parity bias (De.) [34], and Equal opportunity bias, and how they relate to the counterfactual bias? The Eq 10 and 11 need to be further clarified.
Rating
8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
Authors addressed the limitations.
Summary
This paper presents a novel approach to addressing bias in models by utilizing a small external dataset for de-biasing. The proposed method effectively identifies and mitigates inherent biases by analyzing the disparities between original images and counterfactual images. Initially, the method determines the specific type of bias present in the model by computing these differences. Subsequently, it identifies samples with the highest influence scores in terms of counterfactual bias when a particular bias type is detected. To mitigate this bias, the method updates the model by unlearning these influential samples, using the difference between the gradients of the original and counterfactual samples. The method is evaluated using one synthetic dataset and two real-world datasets. The results demonstrate that the proposed method achieves state-of-the-art performance in mitigating counterfactual bias, while also maintaining low computational costs.
Strengths
* The intuition of the proposed method seems clear and intuitive. Also, the proposed method is novel and well-designed two recent techniques in other domains, influence function and unlearning. * The proposed method can mitigate the bias of a model efficiently. It only needs a small amount of group-labeled data points. Furthermore, the method requires much less training time than the existing de-biasing methods based on re-training. * The method achieves good performance on CelebA and Colored MNIST.
Weaknesses
* Lack of some baselines. Given that the objective of this method is to achieve counterfactual fairness or individual fairness, it is important for the authors to compare their approach with existing methods [1,2] specifically designed for individual fairness. This would ensure a fair and comprehensive evaluation of their method's performance in terms of counterfactual bias. It should be noted that Group DRO and LfF, the authors adopted as baselines, were originally designed for group robustness, rather than individual-level fairness. * Challenges in obtaining counterfactual samples. One drawback of the proposed method is that it requires counterfactual samples for each instance in the external dataset. However, acquiring such counterfactual samples for image datasets is not a trivial task. For instance, assuming that images are annotated with 40 attributes, as done for the CelebA dataset, may not always be feasible. * There should be a more detailed explanation of why the experiment results turned out the way they were presented. Why does your method that archives the counterfactual bias also mitigate the worst-case group accuracy or any other group fairness metrics? * Improving the writing clarity. - It is suggested to provide clearer distinctions between the problems of model bias and fairness, emphasizing that while there is overlap between the two, they are not synonymous. - Additionally, the definition of counterfactual fairness in Equation (1) may be incorrect, compared to the definition introduced in [31]. - Some typos in equations. In lines 158 and 183, is it z_i instead of z_k? [1] Yurochkin and Sun. SENSEI: Sensitive set invariance for enforcing individual fairness, ICLR, 2021. [2] Yurochkin et al. Training individually fair ML models with sensitive subspace robustness, ICLR, 2020.
Questions
* Why is the worst-case accuracy better than the baselines in Table 2? Why does unlearning by Co. outperform unlearning by De. in terms of De? * Why does unlearning by De. fail to better demographic parity bias in Table 4? * In Table 7, unlearning more layers of neural networks does not lead to better performance. Why?
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness
2 fair
Presentation
3 good
Contribution
3 good
Limitations
The author has addressed their limitations.
Comments
Thanks for the response. The authors provided satisfactory experimental results regarding the large-scale models (Pretrained Bert and GPT-2), and the performance looks quite promising. They also include additional comparisons with baseline and clarifications of related works. I have checked other reviewer's comments and the author's responses. There are several common questions, e.g., adding more baselines, extension to large-scale models, computational complexity etc. The authors have provided additional experimental results and discussions. I agree with other reviewers that the paper could be better enhanced if these additional experimental results in the rebuttal could be incorporated into the revised manuscript, for example, how to extend to large-scale neural networks, how to leverage existing bias auditing/distribution-shifting datasets (e.g., BERT and GPT-2 with Crows-Pairs), and further extension to discover the key modules for debiasing. Overall, the rebuttal addressed most of my concerns, while the writing could be further improved. Based on the overall quality of the paper and rebuttal, I'd like to keep (maybe increase) my score.
Replying to Reviewer 2Mrc
Dear Reviewer 2Mrc, Thank you for your valuable feedback. We are grateful for your recognition of our response and the experiments in the rebuttal. We also highly appreciate your consideration of the comments given by other reviewers in your overall assessment. We agree with your suggestion that incorporating the additional experiments (extension to large models and bias auditing/distribution-shifting datasets) and discussions (insights from influence functions) into our revised manuscript is crucial. We are committed to refining our writing accordingly. Please accept our apologies for any confusion caused. Once again, we extend our sincere gratitude for your time and insights. Best wishes to you! Warm regards, The Authors
Thanks for your response, the rebuttal addresses my concerns. I would like to increase my score.
Dear Reviewer NRrd, We are very delighted with your recognition of our paper and rebuttal! Thanks very much for your time and comments! We will fix the bugs and clarify the parameters in our revised manuscript. Thank you very much! Best wishes to you! Warm regards, The Authors
Thanks for your response, the rebuttal mostly addresses my concerns. the additional experiments results are valuable. I support such work can be accepted.
Replying to Reviewer hEy4
Dear Reviewer hEy4, We are very delighted with your support! We appreciate your valuable comments on our paper. We are committed to incorporating the additional experiments (extension to large models & ablation studies) and clarifications (flow figure & related works) into our revised manuscript. Once again, thank you very much! Best wishes to you! Warm regards, The Authors
thank you for the responses
Hi authors, Thank you for the thorough responses in your rebuttal. The rebuttal clarifies some issues, e.g. about terminology, additional info about baselines, computational complexity. The authors also ran a number of additional experiments using additional baselines, larger models, imbalanced and distributionally-shifted datasets which strengthen the paper in my view. While some weaknesses remain (notably requiring access to datasets with attributes; which sounds like can be mitigated to some extent but is still an open research question), I believe this paper's contribution is valuable, and I'm increasing my score to reflect that.
Dear Reviewer hkdb, We extend our sincere gratitude for your thorough and insightful review of our manuscript, particularly in regards to aspects such as problem setting, writing clarity, and experimental analysis. Your feedback has been immensely valuable. We intend to incorporate the additional experiments and discussions that you have suggested into our revised manuscript. Furthermore, we are fully dedicated to enhancing the quality of our writing as per your recommendations. Your constructive feedback has been instrumental in shaping the improvement of our work. We greatly appreciate your acknowledgment of our response and the conducted experiments in our rebuttal. Thank you once again for your valuable comments. We hold high regard for your efforts in reviewing our manuscript. Wishing you all the best. Warm regards, The Authors
Reply to author's rebuttal
I thank you for your comprehensive rebuttal and apologize for the slight delay in my response. I truly appreciate the novel contribution regarding the proposition of a new post-processing method based on unlearning techniques. However, I still have a few remaining concerns about the current version of the paper that I believe could enhance its overall quality. 1. I understand from your response that individual and group fairness can sometimes align, yet they may also present conflicts, as mentioned in [A1]. To ensure a thorough and fair comparison, I recommend explicitly delineating the baseline methods in accordance with the fairness notion they target. Given that your method considers individual fairness, I suggest focusing the initial comparison on those baseline methods explicitly designed for individual fairness, such as SenSEI or SenSR, since the baseline methods used in the original paper, such as LDR, LfF, Rebias, and DRO, implicitly or explicitly target group fairness. Furthermore, considering post-processing methods like [A2] that target individual fairness could contribute to a more comprehensive analysis. 2. While the authors have included supplementary results in NLP domains, the applicability of the method remains somewhat constrained to diverse vision applications. because it is extremely hard to generate counterfactual images. In the paper, the considerations around Colored MNIST and CelebA datasets, do not fully represent real-world counterfactual scenarios. Colored MNIST considers just a synthetic bias we can control. Also, image pairs in CelebA are not real counterfactual images (in Figure 1(b), the backgrounds in the image pairs are different). However, the proposed method heavily relies on counterfactual samples in order to detect the bias and compute the influence score. Hence, it would greatly enhance the paper's rigor to address this limitation more extensively. 3. Building on the previous point, the use of potentially incomplete counterfactual samples in CelebA brings about an associated limitation in the bias metric's completeness. I believe it would be beneficial for the paper to delve deeper into this matter, outlining the potential implications and strategies for addressing such incompleteness. 4. If the estimation of the influence score is not accurate in deeper networks as shown in [8], how do we believe that the influence score for the last layer in DNN models is accurate? My concern regarding this point has been also highlighted in [A3], which shows that influence score estimation may be a poor match to leave-one-out retraining for nonlinear networks. As this concern could impact your method's performance, I encourage a more thorough discussion on how your approach mitigates or accounts for this challenge. 5. One important baseline [A4] is missing. [A4] also proposed a method that identifies negative fairness samples by estimating the influence scores and re-trains a model by re-weighting the training samples based on the scores. The difference from [A4] should be addressed in the paper and used as a baseline method. This inclusion could help provide a more comprehensive view of your method's advantages. Once again, I truly appreciate your efforts and the thought-provoking responses you've shared. [A1]. Dwork et at. Fairness Through Awareness, 2012. [A2]. Petersen et al. Post-processing for Individual Fairness, NeurIPS, 2022. [A3]. Bae et al. If influence Functions are the Answer, Then What is the Question?, NeurIPS, 2022. [A4]. Li and Liu. Achieving Fairness at No Utility Cost via Data Reweighting with Influence, ICML, 2022.
Reply to Reviewer AWfs (1/3)
We greatly value your expert feedback on our paper. We hope that our responses can address any concerns you may have. > 1. I understand from your response that individual and group fairness can sometimes align, yet they may also present conflicts, as mentioned in [A1]. To ensure a thorough and fair comparison, I recommend explicitly delineating the baseline methods in accordance with the fairness notion they target. Given that your method considers individual fairness, I suggest focusing the initial comparison on those baseline methods explicitly designed for individual fairness, such as SenSEI or SenSR, since the baseline methods used in the original paper, such as LDR, LfF, Rebias, and DRO, implicitly or explicitly target group fairness. Furthermore, considering post-processing methods like [A2] that target individual fairness could contribute to a more comprehensive analysis. We compare our method with one pre-processing baseline [A4], 6 in-processing debiasing baselines (LDR, LfF, Rebias, DRO, SenSEI and SenSR) and 4 post-processing baselines (EqOdd [B1], CEqOdd [B2], Reject [B3] and [A2]). [A4] utilizes influence function to reweight the training sample, in order to re-train a fair model targeting group fairness metrics (equal opportunity and demographic parity). Among in-processing baselines, LDR, LfF, Rebias, and DRO are designed explicitly targeting higher accuracy (on unbiased test set or worst-group test set) and implicitly targeting fairness, while SenSEI and SenSR are designed targeting individual fairness. EqOdd, CEqOdd and Reject are designed targeting different group fairness metrics (equal odd and demographic parity), while [A2] propose a post-processing algorithms for individual fairness. For fair comparisons, we include performance on both accuracy and fairness metrics. In our paper, we use counterfactual fairness, which provides explicit causal explanations for individual fairness. Furthermore, it's worth noting that, as discussed in [B4], counterfactual fairness also reflects group fairness. Results of additional baselines on Adult are provided below. We will further incorporate new baselines on other datasets. | Attribute | Method | Category | Fairness Target | Acc.(%) ↑ | Bias ↓ | Time(s) | | --- | --- | --- | --- | --- | --- | --- | | gender | Vanilla | - | - | 85.40 | 0.0195 | - | | | Reweigh [A4] | pre-processing | group fairness | 82.60 | 0.0051 | 36 | | | SenSR | in-processing | individual fairness | 84.09 | 0.0049 | 571 | | | SenSeI | in-processing | individual fairness | 83.91 | 0.0016 | 692 | | | EqOdd | post-processing | group fairness | 82.27 | 0.0157 | 0.01 | | | CEqOdd | post-processing | group fairness | 82.94 | 0.0039 | 0.6 | | | Reject | post-processing | group fairness | 73.37 | 0.0574 | 13 | | | PP-IF [A2] | post-processing | individual fairness | 81.96 | 0.0027 | 13 | | | Ours | post-processing | individual fairness | 81.89 | 0.0005 | 2.49 | | race | Vanilla | - | - | 84.57 | 0.0089 | - | | | Reweigh [A4] | pre-processing | group fairness | 82.97 | 0.0015 | 36 | | | SenSR | in-processing | individual fairness | 84.09 | 0.0036 | 571 | | | SenSeI | in-processing | individual fairness | 83.91 | 0.0015 | 692 | | | EqOdd | post-processing | group fairness | 83.72 | 0.0149 | 0.01 | | | CEqOdd | post-processing | group fairness | 83.06 | 0.0023 | 3.6 | | | Reject | post-processing | group fairness | 75.19 | 0.1038 | 14 | | | PP-IF [A2] | post-processing | individual fairness | 82.37 | 0.0015 | 13 | | | Ours | post-processing | individual fairness | 83.80 | 0.0013 | 2.54 | [B1] Hardt, Moritz, Eric Price, and Nati Srebro. "Equality of opportunity in supervised learning." *Advances in neural information processing systems* 29 (2016). [B2] Pleiss, Geoff, et al. "On fairness and calibration." *Advances in neural information processing systems* 30 (2017). [B3] Kamiran, Faisal, Asim Karim, and Xiangliang Zhang. "Decision theory for discrimination-aware classification." *2012 IEEE 12th international conference on data mining*. IEEE, 2012. [B4] Rosenblatt, Lucas, and R. Teal Witter. "Counterfactual Fairness Is Basically Demographic Parity." *Proceedings of the AAAI Conference on Artificial Intelligence*. Vol. 37. No. 12. 2023.
Reply to Reviewer AWfs (2/3)
> 2. While the authors have included supplementary results in NLP domains, the applicability of the method remains somewhat constrained to diverse **vision applications**. because it is extremely hard to generate counterfactual images. In the paper, the considerations around Colored MNIST and CelebA datasets, do not fully represent real-world counterfactual scenarios. Colored MNIST considers just a synthetic bias we can control. Also, image pairs in CelebA are not real counterfactual images (in Figure 1(b), the backgrounds in the image pairs are different). However, the proposed method heavily relies on counterfactual samples in order to detect the bias and compute the influence score. Hence, it would greatly enhance the paper's rigor to address this limitation more extensively. In our experiments, we utilize approximated counterfactual samples for CelebA due to the unavailability of strict counterfactual data. Based on attribute annotations, we select images with the same target attributes but opposite sensitive attributes, while maintaining other attributes as much as possible. Our method achieves the best results on the worst-case group, indicating that the approximated counterfactual samples can also effectively enhance fairness in predictions. Similar to our approach, [B5] proposes to select pairs of counterfactual images based on attribute annotations on the CUB dataset to produce counterfactual visual explanations. Their experiments also show that neural networks can discern major differences (such as gender in our work) between images without strict control (such as background). For real-world visual datasets (like facial dataset or ImageNet), the unavailability of strict counterfactual data is a common challenge. Existing methods propose to train a generative model to create counterfactual images with altered sensitive attributes [B6, B7, B8, B9], which seems to be a viable approach for obtaining counterfactual datasets for more diverse vision applications. Building upon these methods, we will extend our approach to more scenarios. > 3. Building on the previous point, the use of potentially incomplete counterfactual samples in CelebA brings about an associated limitation in the bias metric's completeness. I believe it would be beneficial for the paper to delve deeper into this matter, outlining the potential implications and strategies for addressing such incompleteness. Thank you for pointing this out. Our measured counterfactual bias on CelebA is not rigorous as the external dataset is not strictly counterfactual. The rigorous definition should be “for individuals in different demographic group (e.g. gender), a fair model should give similar predictions on similar individuals (w.r.t. other attributes)”, which reflects individual fairness requests. [B5] Goyal, Yash, et al. "Counterfactual visual explanations." *International Conference on Machine Learning*. PMLR, 2019. [B6] Dash, Saloni, Vineeth N. Balasubramanian, and Amit Sharma. "Evaluating and mitigating bias in image classifiers: A causal perspective using counterfactuals." *Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision*. 2022. [B7] Kim, Hyemi, et al. "Counterfactual fairness with disentangled causal effect variational autoencoder." *Proceedings of the AAAI Conference on Artificial Intelligence*. Vol. 35. No. 9. 2021. [B8] Joo, Jungseock, and Kimmo Kärkkäinen. "Gender slopes: Counterfactual fairness for computer vision models by attribute manipulation." *Proceedings of the 2nd international workshop on fairness, accountability, transparency and ethics in multimedia*. 2020. [B9] Cheong, Jiaee, Sinan Kalkan, and Hatice Gunes. "Counterfactual fairness for facial expression recognition." *European Conference on Computer Vision*. Cham: Springer Nature Switzerland, 2022.
Reply to Reviewer AWfs (3/3)
> 4. If the estimation of the influence score is not accurate in deeper networks as shown in [8], how do we believe that the influence score for the last layer in DNN models is accurate? My concern regarding this point has been also highlighted in [A3], which shows that influence score estimation may be a poor match to leave-one-out retraining for nonlinear networks. As this concern could impact your method's performance, I encourage a more thorough discussion on how your approach mitigates or accounts for this challenge. As verified in [B10], [8] and [A3], influence estimation matches closely to leave-one-out retraining for **logistic regression model**. As discussed in [B6], measuring influence score for the last layer can be regarded as calculating influence from a logistic regression model on the bottleneck features (Sec. 5.1). The same setup is followed by many influence function-based works [B11, B12] and proves to be effective. > 5. One important baseline [A4] is missing. [A4] also proposed a method that identifies negative fairness samples by estimating the influence scores and re-trains a model by re-weighting the training samples based on the scores. The difference from [A4] should be addressed in the paper and used as a baseline method. This inclusion could help provide a more comprehensive view of your method's advantages. The main difference between our approach and [A4] lies in the fact that [A4] necessitates the computation of two influence functions for each training sample, making it challenging to apply to large models and extensive training sets due to computational costs ([A4] performs experiments on three simple tabular datasets with two-layer neural networks). In contrast, our method circumvents this issue by post-hoc unlearning with an external dataset. Furthermore, [A4] is a pre-processing technique, requiring an additional re-training step after computing the influence for reweighting the training samples. We incorporate [A4] as our baseline, with results provided in the Table. On Adult, it can be observed that [A4] cannot achieve better fairness than our method, even with more procedures. Once again, we extend our heartfelt gratitude for your time and the invaluable suggestions you've provided! We would be grateful if you would inform us of any remaining concerns or questions. [B10] Koh, Pang Wei, and Percy Liang. "Understanding black-box predictions via influence functions." *International conference on machine learning*. PMLR, 2017. [B11] Pruthi, Garima, et al. "Estimating training data influence by tracing gradient descent." *Advances in Neural Information Processing Systems* 33 (2020): 19920-19930. [B12] Yeh, Chih-Kuan, et al. "Representer point selection for explaining deep neural networks." *Advances in neural information processing systems* 31 (2018).
Reply to Author's Response
I sincerely appreciate your patient response! While I am still a little skeptical about the assumption that counterfactual images can be collected or generated, your other responses mostly solved my concerns and helped improve my understanding of the authors' method. I will increase my score and agree that this paper is enough to be accepted. I wish the additional results and discussion will be included in the camera-ready version.
Thank you very much for your comprehensive comments and suggestions! We sincerely appreciate them and we will make revisions to our manuscript based on your comments and discussions in order to further improve the quality of our work. Thanks again and best wishes!
Decision
Accept (poster)