A theoretical design of concept sets: improving the predictability of concept bottleneck models

Summary

This paper provides a theoretical analysis of properties of concept sets in CLIP-based CBMs. Specifically, the authors focus on the effect of concept sets on the empirical performance of CBMs in the low-resource regime and under distribution shifts. Towards this, the authors identify two characteristics of concept sets that help improve CBM performance: the size of CBMs and the degree of misspecification. The authors validate their findings on two datasets: CIFAR-10 (for evaluating image classifier performance) and MetaShift (for evaluating distribution shift).

Strengths

__Significance__: A lot of recent work has provided empirical efficacy of using CLIP-based interpretable models for few shot classification [1-4]. In practice, the concepts that achieve a high CLIP activation are often nonsensical. A theoretical understanding of the underlying mechanisms behind CLIP based CBMs are extremely desirable and relevant to the NeurIPS community. `[1]`: https://arxiv.org/abs/2211.11158 `[2]`: https://arxiv.org/abs/2308.03685 `[3]`: https://arxiv.org/abs/2404.09941 `[4]`: https://arxiv.org/abs/2210.07183

Weaknesses

_(This work is outside my area of expertise; I have slight empirical familiarity with CBMs, but I cannot comment on the mathematical correctness of the proofs. Most of my comments are in the questions section)._ __Lacking experimental evaluation__: We have much better datasets than CIFAR-10 to measure the efficacy of an image classifier. I recommend the authors look at LaBO's `[1]` experimental evaluation section for a more comprehensive set of experiments. Overall, I'm recommending a __Weak Accept__. Theoretical research in the properties of CLIP based CBMs is deeply desired and relevant to the NeurIPS community. I haven't given a higher score because it's hard to make informed opinions based on the current evaluation setup.

Questions

Questions while reading the paper: - L23: I think CBMs can be motivated with a better example. Here is another example: A “Tesla model X” did not exist when CLIP was trained, yet we can make a reasonably accurate classifier with a concept bottleneck: {SUV, T logo, gull wing doors, (lack of) exhaust pipe}. - L99: What is P here? - L106: (Definition 1) What is a non-trivial instantiation of a set that is rejected by this definition (ie: a set which is not a concept set)? - L154: I understand that the theoretical justification necessitates linearity, but empirical studies suggest that the final CLIP activation is based on a hierarchical composition of neuron activations `[5]`. Can such relationships be captured by the linear model? - L248: I agree that removing redundant concepts is important. However, in practice, redundant concepts seem to help improve performance (`[4]` identifies almost 3000 concepts for some datasets). - L267: I recommend the authors try out more datasets than just CIFAR10 `[6]`. I think following the experimental section for LaBO `[1]`, one of the paper's cited, would be a good idea here.I recommend testing in domain generalization on CINIC-10 `[7]` as well. - L275: The appendix mentions using ViT B-32 while this section mentions using ViT L-14. The provided code shows references to both ViT B-32 (for CLIP embedding generation) and ViT L-14 (for querying DINO features; which I'm assuming isn't part of this paper). A clarification of which model was used in which context would be beneficial! `[5]`: https://distill.pub/2021/multimodal-neurons/ `[6]`: https://arxiv.org/pdf/1806.00451 `[7]`: https://arxiv.org/abs/1810.03505 ----- Increasing my score to __Accept__ after discussions with authors.

Rating

7

Confidence

1

Soundness

3

Presentation

2

Contribution

3

Limitations

The authors have adequately addressed limitations in the Appendix. I appreciate the inclusion of a broader impacts section!

Summary

This paper presents theoretical contributions related to CBM, which delves into the impact that the choice of concept set has on CBM performance. It identifies advantageous conditions for CBMs, offering an orthogonal and meaningful perspective compared to most other works on CBMs.

Strengths

- Overall, the theoretical findings are insightful. The authors demonstrate the situations in which the performance of CBMs can be enhanced by well-choosing the concept sets. - The framework of the theories is clear. The assumptions are appropriate and not overly restrictive. The definitions are comprehensive and primarily clarify the expressiveness and model-aware inductive-bias, which are two key properties affecting the performance. - The empirical results show that the CBMs surpass the foundational embedding models. These results also provide a unique perspective compared to existing works.

Weaknesses

- The multi-step approach introduced in Sec.5.2 seems to be difficult and brings uncertainty from LLMs, which to some extent limits the practicality and generality. - It would be better if more baselines and datasets were provided in the experiment part. Additionally, due to the costly training procedure, this paper only incorporates the concept representations from foundational multimodal models. This is reasonable but complicates comparisons with other CBMs.

Questions

None.

Rating

6

Confidence

3

Soundness

3

Presentation

2

Contribution

3

Limitations

Yes.

Summary

This paper addresses an important research question in CBM — understanding the properties of concept sets and their connection to the performance of CBMs. A theoretical framework for concept sets is proposed, focusing on two desiderata for concepts: expressiveness and model-aware inductive bias. Their theoretical analysis covers under what conditions the concept representation-based predictions outperform raw feature representation-based predictions. Empirical evaluation with two datasets confirms their theoretical insights that well-chosen concept sets improve sample efficiency and robustness under distribution shifts.

Strengths

Studying the properties of concept sets and how to effectively find a good concept set satisfying the properties, ultimately improving the utility of CBMs, is a very interesting and important research question. The two proposed properties of concept expressiveness and model-aware inductive-bias are intuitive and well-defined. Especially, their observation on connecting them to improved data efficiency is interesting. The proposed theoretical framework naturally reveals the regimes on which it is undesirable to have a concept bottleneck and leads to a principled solution based on the theoretical insights.

Weaknesses

Notations are very loosely defined: * Notations in introduction are used without being defined; for instance, $f, g, \theta, \mathcal{H}$.
 * in line 104, the definition of function f as a concatenation of a mapping $\hat{g}_m$ and a set $[c_1, \dots, c_m]$ is awkward. * what is $d$ in Theorem 1? * in line 154, please clarify that $x$ is not a raw input (e.g., image), but a feature representation output from a backbone foundation model. Limited experiments: * Evaluated only with two datasets. * No performance comparison with other CBM methods, given the same number of concepts and training set size. * No demonstration and anlysis on the actual results of generated concept sets Please increase the fonts in the figures for visibility.

Questions

* in lines 135-136, `This flexibility means we can replace the underlying ... while keeping the output model unchanged.` Could you please elaborate on this? I reckon the output model should also be changed when the coupled concept identification model is changed. * in lines 137-139, `The concept set serves as a stable communication interface ... potentially harmonizing datasets and modalities`. Please extend these statements and explain the broader potential impact of this perspective using concept representations. * how do you determine the thresholds in concept generation? (i.e., $\Omega_m$, $R_m$).

Rating

5

Confidence

4

Soundness

3

Presentation

2

Contribution

3

Limitations

Limitations are described in Appendix C.