Fairness Without Harm: An Influence-Guided Active Sampling Approach

Summary

In this paper, the authors propose Fair Influential Sampling(FIS) to achieve a better Pareto Frontier of the fairness-accuracy tradeoff through sampling training data. To maintain fairness without diminishing the accuracy, training data examples are validated through a validation dataset. In this case, privacy is maintained by not annotating sensitive attributes in the large training dataset, but doing so for the validation dataset. An example data is selected as training data based on its influence on the accuracy and fairness using the validation dataset. This research also claims that by acquiring new data the proposed algorithm can decrease the group gap by utilizing the fairness components and can also prevent the negative impact of distribution shift by utilizing accuracy influence components, which in theory should improve fairness without harming the accuracy.

Strengths

The proposed sampling approach attempts to mitigate group fairness without compromising both accuracy and privacy. As the FIS does not require annotation of sensitive attributes for the large training dataset a substantial amount of cost and labor is negated. The work addresses potential distribution shift issues effectively to maintain fairness and accuracy. Even though the method is validated on some specific datasets the basic mechanism of the FIS has the potential to be applicable in different sectors where maintaining fairness is a crucial factor. The maintenance of privacy supports ethical AI application development practices.

Weaknesses

The success of the FIS is heavily dependent on the refinement level of the validation set. If the validation set is not a proper representation of the large training dataset then it may compromise the accuracy and fairness. The paper's sampling strategy mostly depends on particular influence measurements. The sensitivity of these matrices for different data distributions should be thoroughly explored. If there is bias present in the validation dataset, unfairness may be amplified by the FIS method. The proposed method relies substantially on a clean and informative validation dataset. Which might be hard to secure in certain cases. The paper does not mention several recent studies in the related work section, which could provide additional context or alternative methodologies for addressing fairness without demographic data. Here are a few examples: [1] Zhao, Tianxiang, et al. "Towards fair classifiers without sensitive attributes: Exploring biases in related features." Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. 2022. [2] Yan, Shen, Hsien-te Kao, and Emilio Ferrara. "Fair class balancing: Enhancing model fairness without observing sensitive attributes." Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020. [3] Chai, Junyi, Taeuk Jang, and Xiaoqian Wang. "Fairness without demographics through knowledge distillation." Advances in Neural Information Processing Systems 35 (2022): 19152-19164. The authors' method overlooks the presence of non-demographic features that could still relate to demographic information, posing a privacy issue even without direct inclusion of demographic information. Although in “Empirical Results” section states that 4 datasets were used, according to Appendix E, only 3 datasets(CelebA dataset, Adult dataset, COMPAS dataset) were used for experimental purposes. In 4.2, it is stated that line no 9 updates U and Soriginal, but according to the pseudo code this process is being executed in line no 11.

Questions

Please refer to the “Weaknesses” section.

Rating

3

Confidence

4

Soundness

2

Presentation

2

Contribution

2

Limitations

In this paper, the authors propose FIS to preserve the fairness of the model and the privacy of individuals. However, several limitations can be observed in the sampling process. Firstly, the authors should consider the computational expenses of the algorithm involving different scales of datasets. Moreover, as sampling relies on the validation dataset, its fairness may be jeopardized if the validation dataset is skewed. Additionally, it would be much more efficient to develop an algorithm to select a validation dataset that contains little to no bias. The sampling method also does not explain how fairness is maintained in the case of expanding datasets.

Summary

This paper addresses the challenge of achieving fairness in machine learning (ML) without compromising model accuracy. The authors propose a novel active data sampling algorithm that mitigates group fairness disparity by acquiring more data without requiring sensitive attribute annotations for training, thus protecting privacy. The algorithm scores each new example based on its influence on fairness and accuracy, evaluated on a small validation set, and selects examples for training accordingly. Theoretical analysis demonstrates how acquiring more data can improve fairness without degrading accuracy, and provides upper bounds for generalization error and risk disparity. Extensive experiments on real-world data validate the effectiveness of the proposed approach.

Strengths

Clear & meaningful motivation: The proposed fair influential sampling (FIS) method avoids the need for sensitive attribute annotations during the sampling or training phases, thereby protecting privacy. Sound Theoretical Contributions: The paper includes a thorough theoretical analysis, showing how the proposed algorithm can improve fairness without harming accuracy. It also provides upper bounds for generalization error and risk disparity, enhancing the method's credibility. Applicability: Based on the method design & setup, it seems it's not limited to single sensitive variable, rather multiple discrete ones. Therefore, this renders the method considerably better applicability compared to most other fairness methods.

Weaknesses

1) Insufficient discussion of fairness metrics: The choice and justification of the fairness metrics used in the evaluation could be more thoroughly discussed. A deeper explanation of why these particular metrics were selected and how they effectively measure fairness would provide a clearer understanding of how fairness is quantified in this study and the implications of these choices. For example, why choose risk disparity as the fairness definition, and how does it compare & contrast with other metrics, like equalized odds, or counterfactual fairness metrics. 2) As the authors point out, The proposed algorithm relies on a clean and informative validation set that contains sensitive attributes of data examples. This dependency may not always be practical or feasible in real-world scenarios, where obtaining such validation sets can be challenging due to privacy concerns and data availability. 3) Unclear experiment results presentation and significance: I find key result in table 1 confusing. For example, how is fairness_violation defined, and what is the highlighting scheme, and why is the row of DP of CelebA - Attractive not highlighted.

Questions

1) Could you provide more discussion on the choice of fairness definition used in your evaluation? I have skimmed section B of your appendix, but why is risk disparity selected as the fairness definition throughout, and how does risk disparity compare with other fairness metrics like equalized odds or counterfactual fairness metrics? 2) Your proposed algorithm relies on a clean and informative validation set that contains sensitive attributes. How do you address the potential challenges in obtaining such validation sets in practical scenarios, especially considering privacy concerns and data availability? What are the implications if the validation set is not fully representative or contains noisy labels, and how might this affect the performance and fairness of your algorithm? 3) Based on my understanding of equation 5, your method is compatible with multiple (discrete) sensitive variables. It that correct? 4) For table 1: How is fairness_violation defined? Why is the row of DP of CelebA - Attractive not highlighted? What if the highlighting rule? Writing suggestion: I am confused with the highlighting scheme of Table 1. It would be better if it's explained in the caption right above.

Rating

4

Confidence

3

Soundness

3

Presentation

2

Contribution

2

Limitations

The authors mentioned limitation that it needs validation set though did not attempt to address it.

Summary

Group/statistical notions of fairness are known to be at odds with the accuracy metric. The authors propose an active sampling strategy to improve the tradeoff between fairness and predictive performance. This sampling approach only requires the sensitive information to be known in a small validation set. They estimate the influence of a potential instance on improving the accuracy and fairness, and choose those that most positively influence both objectives.

Strengths

S1 - Improving the accuracy fairness trade-off is a very relevant problem. Besides, doing it in the scenario when only minimal sensitive information requires to be known extends its applicability to real-world scenarios. S2 - Figure 1 offers a nice overview to situate the problem and provide an intuition regarding the objective of the work. S3 - In general terms, the readability of the text is quite good. The ideas are presented clearly, with cohesive explanations throughout. The text is well divided into sections and paragraphs, which helps maintain a consistent flow and makes it easy to follow and understand. S4 - The paper demonstrates commendable attention to reproducibility by providing detailed information regarding the experimental setup.

Weaknesses

W1 - The related works section needs improvement, starting from the most general topics (active learning and fair classification), to more specific areas of research. Additionally, the claim made by the authors in lines 107-108 is not true. Those approaches do not strive to achieve uniform accuracy over all groups, but rather to improve the predictive performance of the classifier on the worst performing groups. Indeed, it is in the metrics considered by the authors where parity among groups is required. Besides, the acronyms DP and EOp are used before defining them. Moreover, when talking about [39] in line 92 it is not which problem the recent work aims to address. W2 - In section 4.1 you introduce the influence of the accuracy component but what you are looking into is the loss, and the loss need not be equal to accuracy. Maybe you could rename it and call it the loss/risk component, or change the definition otherwise. W3 - When active sampling is used the iid condition is not satisfied anymore, but the authors do not mention this in the work nor address this challenge. W4 - One of the main claims made in the work is that this strategy is able to improve the fairness guarantees of classifiers without sacrificing accuracy. The authors even claim the latter can be seen from Table 1. However, in the empirical validation there are no numerical results regarding the accuracy, only fairness-related metrics are shown. Consequently, this claim is impossible to validate from the results provided in the main paper. W5 - The comparison with respect to the JTT method is unfair. In fact, this procedure has a strategy to choose the optimal weight that needs to be given to the misclassified instances, and is learned in the training process. Thus determining the weight to be 20 is unfair. W6 - It is not very clear how much data/information is given to each of the methods, which makes it difficult to verify whether the empirical validation is fair. W7 - The main objective of the active sampling paradigm is to achieve superior performance (in this case better accuracy-fairness trade-off) with less datapoints, as they are chosen effectively. However, there is no empirical validation nor theoretical statement that supports the latter. Additional minor comments: W - In the paragraph labelling from section 4.2, the explanation that introduces the paragraph is not very clear, and it is in fact misleading… it would be nice to improve its clarity. W - In line 235 you talk about ‘most confident labels’, but maybe employing the word confidence is not the most appropriate. W - The authors do not explicitly state that the fairness loss is something to be minimized. W - The authors do not clearly state how the hyperparameters of the rest of the methods are tuned. W - In the paragraph *proposed algorithm* from section 4.2, the authors talk about tolerance $\epsilon$ but there is no such parameter in Algorithm 1. W - You are talking about the trade-off between accuracy and fairness but in line 89 you use the word generalization, which is not a synonym but a word with a completely different meaning. W - The text contains several typos. For instance, between lines 83-85, work(s) need to be plural, and train examples are better references as training examples. Also the authors employ ‘to us’ in line 113, and it would be better to replace it with a more formal phrase such as ‘to this work’. W - In line 86 when you talk about the trade-off it would be nice to include some references (as in the introduction). W - Check how the references are written, missing capital letters in [49] and [23]; no publish information in [57], etc. W - Regarding the titles of the sections, only the first letter of the first word needs to go in uppercase. W - $\G_p$ is not defined.

Questions

Q1 - What do > symbols refer to in equations (2) and (3)? Vectorial product? Q2 - According to the definitions of Lemma 4.1 and 4.2 the value of the influence depends on the size of the validation set. Why don’t you use an average so that this characteristic does not influence? Q3 - In the section *proposed algorithm* you describe an iterative procedure to find the top r instances whose fairness influence based on the true labels meet the conditions, but you are using proxy labels and not the true labels. Then, how do you know the latter? Q4 - Why do you need to make the assumption that the train/test distributions are drawn from a series of component distributions? Q5 - Regarding Theorem 5.2 you talk about the group gap, and claim that the more balanced the data is (...). But which gap/balance do you refer to? Is it in terms of representation or performance? Q6 - Regarding the tabular data, you resample the data to balance the class and group membership; but what is the reason behind this procedure? What happens if there is no such balance?

Rating

5

Confidence

3

Soundness

2

Presentation

2

Contribution

2

Limitations

Limitations are clearly stated by the authors in Section 7.

Summary

The authors propose a method called Fair Influential Sampling (FIS), which selects candidate data points for training based on their estimated improvement on both fairness and accuracy. This estimation considers the influence of a new data point as the average gain in accuracy and fairness achieved by performing a gradient descent step with the new data point. This average is empirically computed using a labeled validation dataset. The gain in loss (or fairness) is approximated as the inner product between the gradient of the loss (or fairness) with respect to the parameters evaluated on the new data point and each validation point. The proposed influence metric does not require access to the sensitive attributes of the candidate data points. Additionally, when labels for the target variable are unavailable for candidate data points, the authors suggest producing proxy labels by annotating the most confident class. This allows the estimation of the accuracy influence score. If a data point is selected for further training, a user must then perform the actual labeling. The selected data points are those that improve the fairness influence score without negatively impacting the average accuracy score on the validation set. Experimental results demonstrate that the proposed approach can select training data points that enhance fairness without reducing accuracy compared to a random sampling strategy. Furthermore, under the constraint of maintaining an accuracy equal to or better than a "random" model, FIS achieves superior fairness performance compared to other competing methods.

Strengths

The paper is well-written, addressing a relevant problem with clear formulation and proposed methods. The authors provide theoretical justification for the approximations used in computing their influence scores and offer formal analysis on the impact of adding more data on model generalization. Experimental results demonstrate that the proposed active sampling approach performs as expected. It does not harm accuracy compared to a base ERM model or random sampling. Moreover, the selected data points improve fairness under this constraint. Additionally, the proposed approach does not necessarily require labeled data, which is advantageous as only ground truth labels for the accepted data are needed. This can motivate the use of this method. That being said, I am not entirely familiar with the extensive state-of-the-art literature on active sampling to fully evaluate the novelty of the proposed approach.

Weaknesses

It seems to me that the data selection strategy based on the proposed influence scores is approximating the process of training with the validation set. I believe it would be beneficial to compare the performance of a model trained with this labeled validation set (without sampling and with random sampling) to demonstrate the advantage of using a data sampling strategy based on the validation set instead of simply including it as part of the training data.

Questions

Could the authors indicate if the results in Table 1 are computed with respect to a test dataset that does not include the one used for validation and sample selection? I ask this because in line 330, the paper states that "10% of the test data is randomly set aside as a hold-out validation set." I assume this portion is not used for evaluation. Typo in Eq 4, shouldn’t \ell be \phi?

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

yes.