On Measuring Fairness in Generative Models

Summary

The paper considers the problem of measuring fairness in generative models. In particular, the paper has two main contributions: (1) they have produced a dataset of hand labeled (sensitive attributes, SA) dataset for various SOTA generative models; and (2) they have proposed a method for estimating the expected sensitive attribute distribution which utilizes the error rates of the SA classifier. These two contributions are used to show that SA classifiers with low error can still cause high errors in previous methods of approximating the sensitive attribute distribution.

Strengths

- The paper presents a strong empirical study of measuring fairness in generative models, and outlines flaws in prior studies. - CLEAM seems to be an intuitive correction to the naive baseline method.

Weaknesses

1. One weakness of the paper is a hole in the narrative: how is $C_{u}$ obtained? In particular, it is unclear in the main-text on how one would generate a SA classifier. Furthermore, the assumption of knowing the underlying error rates should be discussed as a potential limitation. I assume that the SA classifier is trained on data which are not samples from the generative model one is trying to measure the fairness of (as otherwise we already have labels). In this case, the validity of error rates transferring might be questionable. 2. The notation in the paper is somewhat strange. In particular, as far as I can see, $ Pr(u \mid x )$ is not a probability … despite the notation. And it is further “aliased” as $ C_u(x)$ which is additionally confusing. I don’t see why one could not just define the “argmax classification” as $C_u(x)$ directly. 3. The notation of $\hat{p}$ and $p^{\*}$ seems consistent to me. In Section 2, it seems that $p^{\*}$ is the population statistic. However, in Section 3 $p^{\*}$ becomes the estimate generated from GenData. Yes, one could argue that GenData somewhat becomes the “new” population, but this change in perspective is not clear. I think it makes more sense to think of the $p^\*$ in this Section as a high quality estimate of the true statistic. It may be worth changing notation to reflect the 3 possible $p$’s: that generated by the SA-classifier, that generated by GenData, and the unknown true statistic which is being approximated by the former two. Typos: - Appendix Eq (3) LHS

Questions

1. Relating to “Weakness 1”, how exactly are the SA classifiers typically obtained? 2. Further relating to “Weakness 1”, how well do the error rates of the SA classifier transfer? That is, what is the difference between the SA classifier’s error rates with the dataset it was trained on versus the error rates on GenData (which I am assuming is that of Table 1 & 2)?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

I think the assumption of knowing the underlying error rate / accuracy needs to further discussed (see Questions)

Summary

This paper considers the fairness measurement for generative models. The contributions of this paper are three-fold. First, the authors reveal that the existing frameworks have significant measurement errors, even using sensitive attribute classifiers. Second, the authors propose a new framework namely CLEAM that uses a statistical model to evaluate inaccuracies of SA classifiers, thus reducing the measurement errors. Finally, the authors use the proposed CLEAM to measure fairness in important text-to-image generators and GANs, which show the effectiveness of the proposed framework. Experimental results with a manually labeled dataset show that the proposed CLEAM can achieve lower error as compared with some baseline schemes.

Strengths

1)Propose a novel framework to measure the fairness of generative models from both theoretical and experimental perspectives. 2)The fundamental statistic model is easy to follow, and the proposed method CLEAM is able to reduce the fairness measurement error. 3)The dataset created in this paper will benefit the research community.

Weaknesses

After reading this manuscript, this reviewer finds that there are some issues that need to be addressed, as 1)On page 6, the authors say that “the probability of the counts for each output cT in Eqn. 2 (denoted by Nc) can be modeled by a multinomial distribution.” Does this assumption hold in practical systems? 2)The presentation of the statistical model should be improved. Why do the authors assume a multivariate Gaussian distribution instead of other statistical distributions? 3)On page 6, what does "M" represent in equation (3)? 4)The description of equation (8) is not clear enough. The authors are suggested to explain it in detail. 5)The authors are suggested to provide more experiments with other datasets and generative models, in order to demonstrate the effectiveness of the proposed framework.

Questions

Please refer to my comments for details.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

Please refer to my comments for details.

Summary

The authors conduct a study on the fairness of generative models. They propose a CLassifier Error-Aware Measurement (CLEAM) framework which accounts for inaccuracies in classifiers involving sensitive attributes. The authors also create a new dataset of generated images from a text-to-image generator which are then used to evaluate the accuracy of existing fairness frameworks.

Strengths

Better measurement and quantification of bias is a very critical topic both in classification as well as generation tasks. The proposed CLEAM framework appears to produce results which are more balanced than the baseline methods. Comparisons are made to other approaches.

Weaknesses

The text/language/grammar could be improved in many places, especially the abstract. Even the opening of the intro is confusing- "fairness is defined as equal generative quality and equal representation w.r.t some sensitive attributes (SA). In this work, we focus on the more widely utilized definition – equal representation." Isn't this the same, or actually less demanding definition? Does generating samples from both classes, but one at a much lower quality level, actually constitute fairness? Doesn't there need to be some demand on the quality of generation? The jumping around between the language of SA classifiers and generative models is confusing throughout. The description/introduction of CLEAM (lines 59-68) is wordy and a bit confusing. Other than the fact that it is "a statistical model", it's unclear what it is after reading this section. Since this is the focus of the paper, it should be crystal clear what the core of this method is after reading this section. The authors state "we observe an intriguing property in Stable Diffusion Model"- however what they are observing is instability due to noise. Again, I believe this is more related to instability/adversarial attacks than "bias" as it is traditionally defined. I think disentangling these (related) concepts is important. The paper is very dense with many results as well as mathematics. It's clear the authors have much to tell with limited space. However, they need to walk the reader through it a bit more as it is hard to read through the tables and understand what are the key take-aways from each.

Questions

The abstract would benefit form fairly substantial rewriting IMO. "A ResNet18 for Gender with accuracy 97%" is both an awkward phrase, but also unclear what/why it is so specifically being referenced in the abstract. The abstract should seek to make a general observation or summary as to the goals and abstract of the work. Additionally generative models are mentioned first, but then the focus is on classifiers, before moving back to generative models. Cleaning up the abstract would be very beneficial. Are the results of Figure 2 "bias" or instability to adversaries? Traditionally bias is usually thought of when a difference in classes is introduced due to something like "doctor" being associated with men and "nurse" being associated with women. Here there is literally no difference between "a" and "one", yet the distribution shifts. This feels more similar to an adversarial attack where random noise is added as opposed to a biased classifier. Usually adversarial instability in image generation is addressed by adding noise either to the input or latent space. Here the issue is being caused by noise within the text prompt that is impacting the generation. It would be helpful to put the SE in the tables and not the supplemental only (makes it tough on the reader jumping back and forth).

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

This work is explicitly designed to overcome limitations of other frameworks which may have generative bias

Summary

The paper studies measuring fairness in generative models, which is defined as equal number of samples generated from different groups. The measurement needs a sensitive attribute (SA) classifier to predict group attribute to compute the fairness. The paper emprically finds out the error in SA classifier would largely impact the measurement performance of the fairness. They test it by manually labeling samples and comparing fairness measure. The paper then proposes a calibration trick to reduce the fairness measurement error from the SA classifer's error,

Strengths

1. Fairness in generative models is an important problem and open problem

Weaknesses

1. The paper has no insights on why the error in SA classifier would propagate to fairness measurement. The finding is entirely empirical, and seems independent of what kind of fairness definitions used. Would the SA classifier error impact all fairness definitions equally? How much would it impact? What determines the fairness error? The paper is mostly emprical in this finding without providing good insights. 2. The mitigation method seems to be simply computing the metric on different data subsets and then taking average and computing confidence interval. The technical contribution is a little too elementary. 3. The paper largely ignore the vast literature of noisy label, which I think is directly relevant to the problem, i.e. studying the label noise (i.e. imperfect SA prediction in this case) on the impact of fairness and models. For example: [1] Natarajan, Nagarajan, et al. "Learning with noisy labels." Advances in neural information processing systems 26 (2013). [2] Lukasik, Michal, et al. "Does label smoothing mitigate label noise?." International Conference on Machine Learning. PMLR, 2020. There also seems to be some connection between label smoothing and the mitigation. Can you point out any if it exists?

Questions

See Weakness 1 and 3.

Rating

3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

See Weakness.

Summary

This paper proposes a framework for fairness measurement. It first shows that existing framework has considerable measurement errors even when highly accurate sensitive attribute classifiers are used, then propose CLassifier Error-Aware Measurement (CLEAM), a new framework which uses a statistical model to account for inaccuracies in SA classifiers.

Strengths

1. Shows the significant measurement errors of existing frameworks by experiments. 2. Propose new datasets based on generated samples with manual labeling w.r.t. SA. 3. Proposes a simple statistical approximation method is proposed to obtain a stable and accurate estimation of the GT probabilities.

Weaknesses

1. The organization of the paper is kind of hard to follow. The first contribution takes too long and may confuse the readers. 2. The introduction to the proposed method is too short and not very solid. Some theoretical support may be better. Maybe you can talk about cases when using distributions other than Gaussian to approximate the distributions. 3. Only a public dataset CelebA-HQ is used. It's better to test methods on various datasets.

Questions

plz refer to the "weaknesses" part.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

The structure of paper is hard to follow and somehow boring for readers.

Summary

The objective of this paper is on measuring the fairness in generative models. There are three contributions. (i) Consideration of measurement errors of sensitive attribute (SA) classifiers in fairness measurement of generative models. (ii) A classification error aware measurement framework, called CLEAM, which based on statistical model accounts for the inaccuracies of SA classifier to reduce measurement error in generative models. (iii) As an application, the authors demonstrate that CLEAM can be applied to measure fairness in text-to-image generator and GANs.

Strengths

The paper is well written, with clear intuitions, illustrations, and experimental results.

Weaknesses

- Putting the application of gender bias in the introduction seems to be out of place, possibly undermining the main framework CLEAM. Similarly, table 1 is also out of place, throwing numbers to the readers without explaining the setup. - The authors make an assumption of ground-truth labels of sensitive attributes, which may not be available in practice. - Some of the human faces in Figure 2 and figure 3(a) are different, revealing the uncertainty of the generative models. I understand the intuition of the authors to provide a demonstration of the paper at an early part of the paper. Please clarify if I misunderstood anything.

Questions

- The presented method considers binary sensitive attributes. However, modern generative models easily generates images with multi-sensitive features. How do the authors extend to this scenario? Is naive enumeration the only possible way? - In Equation (2), are $p_i$ and $\alpha_i$ known? - In line 222, is $ s = 30 $ large enough? - Is there any future work based on this paper? Please also address the points in "Weakness" above.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

1: Your assessment is an educated guess. The submission is not in your area or the submission was difficult to understand. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

NA.

Re: Response

Thanks you for the detailed response. I currently have no further questions and will keep my scores for now.

Thank you for the very constructive feedback and increasing the rating

We sincerely thank the reviewer for the insightful comments. Thank you for the positive feedback and for increasing the rating. We will include all additional results in the revised version. Sincerely, Authors

Re: Response

Thank you for the detailed response. I don’t have any further questions at the moment and will keep my scores for now.

Thanking the Reviewers for their valuable feedback

We thank the Reviewers for all of their valuable feedback and for taking the time to thoroughly review our paper. Our proposed fairness measurement framework is grounded on a careful statistical analysis and can improve accuracy substantially. The framework can be used to study biases in SOTA generative models as demonstrated in our paper. We believe our work would make important contributions to advance fair generative modeling research. We have submitted all the codes of our experiments for reproducible research. Thank you for your valuable time in reviewing our submission. We hope Reviewers could consider increasing the ratings from initial decisions if our responses can resolve Reviewers' concerns. $ $ Sincerely, Authors

After rebuttal

Thanks for the response. It clarifies my questions. Best of luck.

Response to rebuttal

I have read the other reviews and authors' responses. I am willing to update my recommendation to weak accept. I think the question around bias vs. adversarial stability is an important point which is worth addressing. If the authors make the improvements to the text which they have indicated throughout the review, I think this paper will offer contribution to the community.