Robust Concept Erasure via Kernelized Rate-Distortion Maximization

Summary

This paper considers the concept erasure task removing an attribute from distributed representations while retaining other information from the original representation space as much as possible. Specifically, it proposes a Kernelized Rate-Distortion Maximizer (KRaM) framework and provides a theoretical analysis for the proposed algorithm. Finally, various experiments are conducted to show the effectiveness of KRaM.

Strengths

The topic considered in this work is important/interesting and the structure of the paper is well-organized.

Weaknesses

1. The contributions of this paper are unclear, and the novelty is also insufficient. From my understanding, the major weakness is that the KRaM is just a simple application of the kernels to the rate-distortion maximizer framework. Meanwhile, the strengths/differences between the proposed KRaM and the current concept erasure methods such as [1,2,3]. The authors should make more discussions in the rebuttal phase. 2. The theoretical results are also limited because it merely applies to the RBF kernel. Thus, it is better to show the theoretical results for other kernels. 3. The experiments are insufficient to support the effectiveness of the proposed KRaM. Because 1) there are a few competitive methods [1,2] that are not considered in this work; 2) the improvements of the KRaM against the current methods are limited. In particular, in Tab.2, compared with RLACE, the performance gap is sharp such as Acc.(y) and A_k. 4. Why do we need to train an extra concept function f? What will happen to the performance if we only use the outputs of a pre-trained model as f? Ref: [1] Learning Diverse and Discriminative Representations via the Principle of Maximal Coding Rate Reduction. [2] Kernelized Concept Erasure. [3] Probing Classifiers are Unreliable for Concept Removal and Detection. ================================================== Post-rebuttal: My concerns are basically solved. So I'd like to raise my score to BA.

Questions

See weakness.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

None

Summary

The KRaM framework is proposed in this paper to perform concept erasure. The approach aims to differentiate similar concept labels in the learned representation space while preserving other information. The paper demonstrates the effectiveness of KRaM in erasing various concepts across different forms of representation (i.e., categorical, continuous, and vector-value variables) through theoretical analysis and empirical setups. Additionally, the paper proposes a clear measurement to evaluate the alignment of representations before and after concept erasure.

Strengths

This paper presents several strengths, including a clear and well-written introduction that guides the reader to understand the problem and the motivations behind it, as well as providing helpful background study. They provide clear setups to study and offer results empirically and theoretically. This paper proposes a new approach to address the limitation of prior works, which did not evaluate the amount of information preserved in the representation. The proposed method uses a clever and heuristic idea to measure alignment by comparing overlapping neighborhoods of the original representation.

Weaknesses

Here are some potential improvements to the original content: 1. In line 273, it is stated that a more effective method of erasing concepts would result in low accuracy and high mean squared error (MSE) for the deleted concept. This statement may seem counterintuitive, since KRaM claims to retain some original information. However, as shown in Figure 3, the performance of downstream task drops relatively significantly when concepts such as Christian, Jewish, and Muslim are erased. The relationship between utility and representation preservation needs more explanation. 2. The author has presented a heuristic approach to determine the amount of information that is retained in a representation. However, it is unclear how this approach can guarantee that the representation is altered according to the given concept, particularly in the continuous form of representation.

Questions

Is it possible to apply KRaM to image-based datasets? Doing so could potentially enhance the proposed method's ability to generalize across various modalities.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

The paper presents a novel framework that should potentially be applied to various modalities. Finding and understanding the exact information which is altered by concept erasure is a future work and has positive societal impact to the field.

Summary

The paper proposes a novel method to erase concepts from a distributed/latent data representation. The method is based on a kernelized rate-distortion function that can be maximized to achieve concept erasure. The kernel encodes the "concept similarity" between two samples, and it is therefore supervised. That is, it needs a concept label associated with the samples. From a set of experiments, the authors show that in general they manage to achieve a more effective concept erasure without loss of performance in downstream tasks.

Strengths

- (Significance) Concept erasure is a timely topic for AI safety, and an interesting topic per se in the broader interpretability field. - (Novelty) Kernel-trick, coupled with rate-distortion theory seems like a reasonable novel contribution. - (Soundness) Experiments seem convincing to support authors claims

Weaknesses

- (Soundness) Assumptions and drawbacks should be further clarified. See section below. - (Theoretical results) I am unsure how useful is the proven lemma in the overall narrative of the paper. The only mentioned conclusion is the one in line L.202. However, this conclusion can be reached by simply looking at equation 2. And actually the conclusion is wrong if we were to only look at the inequality: since R(Z) is the *lower* bound, increasing R(Z/K) does not necessarily imply an increase in R(Z).

Questions

- Gaussianity assumption: have the authors tested with different formulations of the rate distortion function? would this be feasible? - Determinant scalability: the determinant seems to badly scale with the number of samples in Z. Could the authors comment on the computational bottleneck of their method, and possible ways to solve it? - Additional assumptions: I believe there should also be an assumption about the concepts being uncorrelated. I would imagine that if different concepts are correlated in the dataset, then the authors method could accidentally erase more concept, e.g. using one of the experiments, erasing religion could potentially erase also ethnicity information. Could the authors comment on this, potentially also from the perspective of societal impacts. - As a sanity check, have the authors tested their method on the bottleneck layer of a concept bottleneck neural network [Koh, 2020]? I would imagine that if the method worked properly, then the authors could clearly see that one of the dimension in the bottleneck layer would essentially become constant.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Only few limitations are discussed from a techincal perspective. Societal impacts are also very briefly mentioned.

Summary

This paper studies concept erasure, a task proposed by prior work [1,2] which aims to remove information from a chosen attribute while retaining other information from the original input as much as possible. This paper achieves this by maximizing the kernelized rate-distortion maximizer (KRaM), an objective from information theory measuring how well lossy compression methods obtain the original information—specifically, maximizing the objective results in increased dissimilarity between instance pairs with similar concept labels, leading to the distances in the learned representation unrelated to the concept labels, achieving concept erasure. To preserve the information from the original input, a constraint is proposed in this paper by maintaining the rate-distortion function to a constant size using a Lagrangian function. The authors propose to use k-nearest neighbors to measure the alignment: how much the learned representation preserves the information from the original input after concept erasure. Bounds for the objective and expectation of the alignment are provided. Finally, the authors provide empirical results to demonstrate that the proposed method can erase concepts (categorical, continuous, and vector-valued) in the DIAL, UCI Crimes, and Jigsaw toxicity datasets. [1] Ravfogel, Shauli, et al. "Kernelized Concept Erasure." arXiv preprint arXiv:2201.12191 (2022). [2] Ravfogel, Shauli, et al. "Linear adversarial concept erasure." International Conference on Machine Learning. PMLR, 2022.

Strengths

Originality: The research problem is already defined and studied, but the method of using kernelized rate-distortion function for concept erasure is new. Quality: The quality of this submission is relatively high. The related publications are cited and discussed. The empirical analysis is comprehensive, and the theoretical results are also provided. Notably, the method can erase concepts from categorical attributes and continuous or vector-based concepts (e.g., word vectors). A synthetic experiment verifies the proposed measuring alignment and outperforms other alignment measurements. The empirical results on vector-valued concept erasure are convincing. Clarity: The paper is well-written. Key terms, such as concepts, the rate-distortion function, and the alignment score measurement, are clearly defined. Significance: The authors make a great effort to use the proposed KraM on distributed word vector representations such as GPT-3.5 to erase concepts, making this work very relevant and significant to the recent trends of large language models and valuable for many real-world tasks requiring removing certain concepts or attributes.

Weaknesses

> Instead, objectives for concept erasure use no supervision (apart from the labels of the concept to remove). This makes using adversarial learning or mutual information estimation methods for concept erasure difficult. Unfortunately, this may not be the case. Although the specific problem of concept erasure is not studied in mutual information estimation methods, works such as [1-3] remove information from a particular attribute without any downstream task known using mutual information estimation or invariant learning. It would be helpful for the reviewer to give final recommendations if the authors could further compare these works. Also, the proposed method has similar or worse results than baselines, such as some entries in Table 1. [3] Song, Jiaming, et al. "Learning controllable fair representations." The 22nd International Conference on Artificial Intelligence and Statistics. PMLR, 2019. [4] Tsai, Yao-Hung Hubert, et al. "Conditional contrastive learning: Removing undesirable information in self-supervised representations." arXiv e-prints (2021): arXiv-2106. [5] Moyer, Daniel, et al. "Invariant representations without adversarial training." Advances in Neural Information Processing Systems 31 (2018).

Questions

In Figure 4, the accuracy and alignment score trend differs from Section 4, where the authors claim that the alignment score is well correlated with the downstream performance. The reviewer is unsure about this and gently asks the authors to clarify.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The authors have discussed the limitations of the work as well as future directions.

Thank you

This reviewer's question and concern have been addressed. Therefore, the reviewer raises the rating.

Thanks for the rebuttal

I thank the authors for the insightful rebuttal. Most of my concerns were addressed, however I am unsure about the additional experiment provided. The table provided does not seem to directly address my concern since, if I am understanding correctly, the additional experiment is essentially the same as the ones already provided in the submission (E.g. figure 3). What I would have like to see was more something similar to Figure 5, but for the very specific case of concept bottleneck models, or similar models where we know that the concepts are aligned to the canonical base, and therefore I find it to be a very nice sanity check to see that only the dimension of the erased concept is affected. I would find this to be better evidence than Figure 5 since an additional step (UMAP) is needed for the visualization, which makes it more difficult to fully give credit to Kram for the concept erasure. Actually thinking about it, your method does not need to be applied to a latent representation from a neural network, right? Theoretically speaking, for any tabular dataset, the original feature of the input data could be regarded as concepts. In other words, if you were to apply your method to, say, the Iris dataset to delete the "concept" "petal width", we should see an effect only for that feature, right? Would you be able to run such an experiment? Should be quick I believe.