Summary
Suppose we are given a distribution of data points $(x,z)$, where $x \in \mathbb{R}^d$ and we have one-hot class labels $z \in \mathbb{R}^k$. This paper shows how to construct an affine transformation $\phi(x) = Px + b$ of the data so that
* $\phi(x)$ has zero covariance with $z$
* $\\|\phi(x) - x\\|$ is minimized, subject to the above condition
The paper argues that this is the optimal affine transformation that ``erases'' the concept $z$ from the representation $x$. Namely, linear regression on $x$ with any convex loss function will be unable to reconstruct $z$ better than random guessing. The paper empirically shows that the proposed method outperforms previous methods (RLACE, INLP) for erasing concepts from a representation, in terms of computational efficiency/collateral damage of the concept erasure.
Applications are demonstrated in: 1) fairness, and 2) probing language models to measure how much a certain concept matters to their performance.
Strengths
The paper has excellent exposition: the proofs and definitions are clear and simple to follow, and the experiments are generally well explained. The applications to language models are compelling, and the paper demonstrates an improvement over previous approaches. The result is of broad interest to the community.
Weaknesses
I am not an expert in the field of concept erasure, so I do not know if this method was previously known. However, my feeling is that the paper proposes a "trivial" solution: orthogonal projection of the random vector $x$ to the subspace of random vectors uncorrelated with $z$. So I find it surprising that this method was not previously known (maybe in a different field and under a different name).
Of course, this is not actually a weakness of the paper (but rather of the reviewer).
Questions
1. The authors state that "prior work has focused solely on preventing linear models from leveraging Z", whereas their work applies also to deep neural networks. How does this claim relate to the cited papers [22, 5, 3, 37], which also intervene on the representations of deep neural networks?
2. Can the authors clarify the paragraph at the top of page 7, entitled "Results"? Why is 77.3% profession-prediction accuracy reported, and then 78.1% accuracy reported? What is the difference between these settings?
Typos:
* In Definition 2.1, \mathfrak{L} is said to have been defined in Section 2.1, but it is not.
* In Section 2.2, the equation between lines 56 and 57 is an argmax over X' but X' does not appear in the expression
* Figure 3, "ramdom" -> "random"
* Appendix E.1, "a forteriori" --> "a fortiori"
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.