Fair Streaming Principal Component Analysis: Statistical and Algorithmic Viewpoint

Fair Principal Component Analysis (PCA) is a problem setting where we aim to perform PCA while making the resulting representation fair in that the projected distributions, conditional on the sensitive attributes, match one another. However, existing approaches to fair PCA have two main problems: theoretically, there has been no statistical foundation of fair PCA in terms of learnability; practically, limited memory prevents us from using existing approaches, as they explicitly rely on full access to the entire data. On the theoretical side, we rigorously formulate fair PCA using a new notion called \emph{probably approximately fair and optimal} (PAFO) learnability. On the practical side, motivated by recent advances in streaming algorithms for addressing memory limitation, we propose a new setting called \emph{fair streaming PCA} along with a memory-efficient algorithm, fair noisy power method (FNPM). We then provide its {\it statistical} guarantee in terms of PAFO-learnability, which is the first of its kind in fair PCA literature. Lastly, we verify the efficacy and memory efficiency of our algorithm on real-world datasets.

Paper

Similar papers

Peer review

Reviewer GuCQ6/10 · confidence 3/52023-07-06

Summary

This paper studies fair PCA problem. The authors provide a novel formulation of fair PCA based on the "Null it Out" approach and propose the corresponding criterion called PAFO-learnability. The authors also present a streaming algorithm for fair PCA, which has low memory complexity. Experimental results verify the scalability of the proposed method.

Strengths

1. The paper is well-written and mostly clear. 2. The studied problem is interesting and important. Streaming algorithms are very useful in the limited-memory setting.

Weaknesses

1. Assumption 6.1 is a bit strong to me. The paper may show that when D_s is a common distribution such as sub-Gaussian distribution, the generated data satisfies assumption 6.1. 2. I'm worried about the tightness of theorems in section 6. The theoretical results show that Alg 1 and Alg 2 may require very large block sizes, which reduces the contirbution of saving memory. 3. The experiments part may present some quantitative results rather than just show the images. The expriments should verify that proposed FNPM algorithm is PAFO-learnability.

Questions

1. Why this paper proposes a new formulation of fair PCA rather than use a existing one? What's the advantage of "Null it out" formulation? 2. Do existing fair PCA algorithms satisfy PAFO-Learnability?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The paper has no potential negative societal impact.

Reviewer 6AMc6/10 · confidence 4/52023-07-07

Summary

This paper proposes a new approach for Fair PCA algorithms that is scalable and fair at the same time. The main contributions of the paper are as follows: - A new formulation of fair PCA based on the "Null It Out" approach. The goal is to maximize explained variance while nullifying the subspace spanned by the mean difference and leading eigenvectors of the covariance difference between groups. This formulation leads to a closed-form solution and avoids infeasibility issues in previous covariance matching approaches. Their approach removes the unfair subspaces using a noisy power method. - A new notion of learnability for fair PCA called Probably Approximately Fair and Optimal (PAFO)-learnability. This provides a statistical framework to analyze some of the fair PCA algorithms. - A new setting called fair streaming PCA, which addresses practical memory limitations. The authors propose an algorithm called Fair Noisy Power Method (FNPM) which only requires O(dk) memory, where d is data dimension and k is the target PCA dimension. - The empirical study of this method using a vision task is intuitive.

Strengths

I believe the strengths of this paper can be summarized in three main key points: - Theoretical rigor. The paper provides the first statistical framework for analyzing fair PCA in terms of PAFO-learnability. This gives theoretical guarantees on the solution quality of algorithms like FNPM. Previous works mainly lacked such a framework. - They propose FNPM for this problem, which is quite simple to implement, building on standard tools like cumulative averaging and the noisy power method. This makes it easy to apply in practice. - Their approach is scalable and intuitive. They have validated that with a vision task to demonstrate the scalability. Also, the formulation based on nullifying the "unfair" subspace gives flexibility in how much fairness to impose by choosing m. This can be tuned based on the use case. The analysis provides insights into how properties like the singular value gaps and mean difference norm affect the sample complexity and solution quality.

Weaknesses

There are some main concerns I have regarding this paper: - The results are limited to binary sensitive attributes and two groups. Extending the approach to handle more complex, multi-group scenarios with sensitive feature interactions would strengthen the paper. Some of the main previous approaches can easily handle this problem as well [A,B]. - Certain assumptions made, like those in Assumptions 6.1 and 6.2, are quite strong and restrictive. Relaxing these assumptions, or testing how sensitive the approach is to their violation would improve the robustness. - The experiments focus on how much FNPM removes features visually related to sensitive attributes. Evaluating the fairness of solutions in a more quantitative, metric-based fashion would provide a more objective assessment. This quantity is defined differently in various approaches. - Lack of comparison with the other branch of fair PCA methods. Although the goal of equalizing losses is different than what presented here, it would be more beneficial to better understand how different methods would compare. - Comparing the effects of this fair PCA on downstream tasks like classification can be beneficial to better understand the effects of a fair PCA approach. [A] Morgenstern, Jamie, et al. "Fair dimensionality reduction and iterative rounding for sdps." arXiv 2019 (2019). [B] Kamani, Mohammad Mahdi, et al. "Efficient fair principal component analysis." Machine Learning (2022): 1-32.

Questions

See the previous part

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Some of the limitations I discussed in the weakness section are not clearly discussed in the paper.

Reviewer VcC67/10 · confidence 2/52023-07-10

Summary

This research paper focuses on the fair principal component analysis (PCA) problem using streaming data while requiring low memory. The authors introduce a new formulation for fair PCA, which involves optimizing the vanilla PCA objective with a linear "fair" constraint. In the oracle setting, where the true parameter is known, the problem has a closed-form solution. The authors define the concept of "PAFO-learnable" to quantify the sample complexity of learning a semi-orthogonal matrix V, which is an approximately optimal solution to the oracle problem. They present streaming algorithms and prove that such an algorithm scheme has finite sample complexity according to the proposed PAFO-learnable notion. The main idea is to utilize the noisy power method framework to estimate the unfair subspace (the linear constraint) and subsequently employ this estimation in the NPM (Noisy Power Method) to estimate fair PCA. The effectiveness of the proposed method is evaluated using real-world data.

Strengths

Overall, the contribution of the paper is well-motivated and aligns with the ongoing development of methods for problems involving fair constraints. The paper is well-written, and the investigation is quite extensive. However, I must admit that I am not familiar with recent developments in fairness in machine learning and cannot provide an assessment of the significance of the new formulation Equation (1) and the concept of learnability (Definition 4.2), although they appear reasonable and interesting.

Weaknesses

I have a few minor comments: * L68: It might be more appropriate to use the term "semi-orthogonal matrix" instead of "orthogonal" to distinguish between O(d) and St(d,k). * L69, L116, L171: QR decomposition typically yields two outputs, namely the (semi-)orthogonal part and the upper triangular part. * L164: Regarding F_d, it differs from the one defined in Definition 4.2, where learnability is defined for a different quantity denoted as F_d. * L262: Once again, F_{d,m,k} is inconsistent with the definition provided in Definition 4.2.

Questions

See above.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Yes.

Reviewer SSmx4/10 · confidence 4/52023-07-10

Summary

The paper defines a new notion for fair pca. It is assumed that the data comes from mixture of (two) distributions and the goal is to find a subspace for it such that the solution subspace is perpendicular to 1) the difference in the difference vector of the mean 2) the top m eigen vectors of the difference of the variance This problem can simply be computed by computing PCA on a projected space. This paper looks for an algorithm that is 1) with probability 1-delta, reports an approximate solution. 2) Works in the streaming setting where once gets samples from the mixture of the distributions and uses space which depends on O(kd). The algorithm is to use the samples to estimate the parameters of the two distributions and thus approximating the orthogonal constraint and then run the standard SVD algorithms.

Strengths

- The paper defines a new notion for fair PCA.

Weaknesses

- The paper does not discuss why this particular notion captures fairness. - The amount of technical novelty in the paper is limited.

Questions

NA

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

NA

Reviewer GuCQ2023-08-15

Response

Thank you for the rebuttal, which address part of my concerns. However, I still have some additional questions: 1. It is not clear to me whether Alg.1 is online or offline. 2. The iteration number U in Alg.1 does not seems to appear in the theorems in section 6. How does it affect the performance of the algorithm?

Authorsrebuttal2023-08-17

We thank the reviewer for their attention to our work and the insightful questions they have raised. Below, we provide responses to each of the questions. > **Question 1: It is not clear whether Alg. 1 is online or offline.** Our Alg.1 is an online algorithm that takes a data point one by one, and performs learning in $O(dm)$ space complexity to output $N$, an approximate unfair subspace (see Section 3.2). Although we have shortened our pseudocode due to space constraints, the main estimators used in Alg. 1 (see Eqn. (5) in our draft) are updated in an online manner using only vector-vector additions and vector-matrix multiplications. We included the complete pseudocode of Alg.1 in Appendix B, where the reviewer can find a more clarified version of our Alg. 1. In the offline setting, as we can fully compute $\mathbf{Q}$ and $\mathbf{f}$ from the given offline data, the unfair subspace $\mathbf{N}$ can be computed via SVD of $[\mathbf{Q} | \mathbf{f}]$, as discussed in Section 3.2. Depending on the problem setting, one can still transform an offline setting into an online setting by going through the data points one by one. This would help alleviate memory limitations or other issues, as we’ve done in our CelebA dataset experiment. > **Question 2: Effect of the iteration number $U$ on the theorems in Section 6 and the performance of Alg. 1** Thank you for pointing this out. Our final sample complexities (Theorem 6.3) are derived by multiplying the iteration number by the batch size for each phase, which is why there isn’t an explicit mention of the iteration number $U$ in the theorem statements; for the proof we have chosen a suitable $U$, and it is included in our Theorem 6.3. Precisely, Theorem 6.1 and 6.2 characterize the sufficient block size for ensuring small noise terms in the noisy power iterations, and Lemma 6.1 (which is taken from [8]) universally characterizes the iteration number that ensures a small final error, given that the iteration errors are small. Let us further elaborate on the effect of the block size $b$ and the iteration number $U$ on the convergence rate of the noisy power method. With a closer look at the convergence result by Hardt & Price [8], especially their Lemma 2.2 and Theorem 2.3 (Lemma 2.3 and Theorem 2.4 of their arXiv version, resp.), the distance between the noisy power method iterates and the ground truth decays roughly as $\varepsilon + C^U$, where $\varepsilon$ scales inversely with the square root of the block size $\sqrt{b}$, $U$ is the iteration number, and $C$ is a problem-dependent quantity that depends on the singular value gap and $\varepsilon$. Thus, for a fixed block size $b$, our choice of $U$ is a minimal choice (and thus “tight”) such that the second term becomes negligible compared to the first term, resulting in the error $\varepsilon \sim \frac{1}{\sqrt{b}}$. In other words, even though $U$ is increased far beyond our choice, the final error will still be $\varepsilon$, i.e., a much higher number of iterations does not lead to an error less than $\varepsilon$. Again, we emphasize that our choice of $U$ is *sufficient* to ensure the final error is small. In practice, a lesser iteration number may be sufficient for good performance. Indeed, for our CelebA dataset experiments and additional UCI/synthetic dataset experiments, we have observed that a moderate number of iterations (10~20) is enough. For completeness, lastly, we provide here the precise form for the iteration number in Alg. 1 (For the notation, please refer to Assumption 6.2.): $$ U = O\left(\frac{\nu_m}{\nu_m-\nu_{m+1}}\log\frac{d}{\epsilon\delta}\right) = O\left(\frac{K_{m,\nu}}{\Delta_{m,\nu}}\log\frac{d}{\epsilon\delta}\right) $$ We hope these responses resolve the reviewer’s concerns, and we are happy to answer any more questions or concerns that the reviewer may have.

Reviewer GuCQ2023-08-19

Thank you for the response which addresses my concerns. I have raised my score from 4 to 6. I think the complete pseudocode of Alg.1 should be presented in the main paper (at least the paper should indicate the full version is in appendix B). Also, I hope the author can add the discussion on the iteration number U to the revision.

Authorsrebuttal2023-08-19

Thank you for your constructive feedbacks and for revising the score. We will ensure that the discussion and clarification you suggested are reflected in our final manuscript.

Reviewer 6AMc2023-08-19

Thanks to the authors for the responses. I will keep my score.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC