Summary
This paper hypothesizes that the latent representations learned by text-guided diffusion models contain structured subspaces corresponding to semantic concepts. Concepts are formalized as latent variables, and the concepts associated with a (text,image) pair are formalized as a bag of these latent variables, such that the distribution over image outputs is conditionally independent of text inputs given these concepts.
Necessary and sufficient conditions (causal separability; Proposition 3.3) are given for concepts to be arithmetically composable; arithmetic composability in turn admits algebraic manipulation of concepts. A proof-of-concept algorithm is given for identifying subspaces of latent representations corresponding to concepts (based on "spanning prompts") and examples are provided of identifying and manipulating these subspaces using the Stable Diffusion model.
Strengths
The conceptual framework constructed in this paper is though-provoking. The exposition is quite clear, striking a good balance between helpful exposition and mathematical precision. Definition 2.5 (arithmetic composability) is interesting, as is Proposition 3.3 (characterizing arithmetic composability in terms of "causal separability).
The algorithms proposed in Section 4 for identifying and manipulating concepts are well-motivated. I also appreciate the discussion in Section 5 of the structure/character of concept subspaces. These analyses could be a good starting point for guiding empirical study of the score representation.
Depending on how the effective the proposed algebraic interventions prove to be in practice (see Weaknesses) I think that the abstractions and definitions introduced in this paper have the potential to stimulate an entirely new direction of work on control and interpretability of generative models. Control and interpretability are broadly relevant to the NeurIPS community.
Weaknesses
The experiments in this paper are very rudimentary. There are no quantitative results, and the qualitative results consist of a small number of examples provided in Figures 1-5. It is unclear to me how broadly effective, reliable, or robust this method is. There is just not enough information to evaluate whether these ideas work (or could be made to work) in practice; see the Limitations section of my review for additional thoughts about practicality.
Minor: "Causal Separability" is strong language, and I'm not convinced that the word "causality" accurately describes the relationship defined by this term. At the very least, some argument is needed to justify calling this a causal relationship.
Questions
Manipulating the non-parametric representation space via evaluated values along the diffusion of a score-based model is clever, and my understanding is that this is the key to making concept algebra tractable. Could these methods be applied to, e.g., the Parti model? If not, it might be helpful to point this out and be more explicit about how and where these method depend upon the structure of a score-based model.
This work seems closely related to interpretability research. The concept editing algorithm could be seen as a sort of causal intervention (in the sense of "do calculus") on a hypothetical interpretation of a latent representation. I'm curious how this work connects or complements recent interpretability research, e.g., ongoing work on mechanistic interpretability.
Is the definition of "causal separability" given here related/consistent with established definitions of "causally separable processes"?
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Limitations
It is unclear to me how restrictive the causal separability condition might be (Definition 3.2). Especially for more abstract, complicated concepts, I could imagine that causal separability might never be satisfied (but I am open to being convinced otherwise). Even for simple concepts, it seems like causal separability can be violated in ways that might not be obvious: for example, the inseparability of (male,female) and (deer,human) surprised me (although it became very clear once the reason for inseparability was explained to me). Understanding the how broadly causal separability holds seems essential to the ultimate practicality of this framework.