Stochastic Concept Bottleneck Models

Summary

Focusing on concept bottleneck models, this paper extracts concept dependencies with a multivariate normal distribution and derives an intervention strategy based on the confidence region of the normal distribution that incorporates concept correlations for better interventional effectiveness.

Strengths

1. This paper is organized and written well. 2. Concept bottleneck model is an important direction for xAI and even generalizability.

Weaknesses

1. Experiment results show marginal improvement over existing methods. 2. The novelty and contribution of this paper is limited.

Questions

1. Intervention by setting values can be difficult in some case, the example given by the paper: "intervene on the CBM by setting its value to 1" is an extreme case. Humans are not good at estimating probabilities. 2. "do not use the intervened-on concepts to update their remaining concept predictions" may not necessarily be a bad thing. As in many real-world cases, there are exceptions to rules and patterns. Nothing is absolute. Universally "extend the concept predictions with the modeling of their dependencies" may be problematic. Are these exceptions handled by confidence region? 3. What is the novelty comparing with E. Kim et al. (2023)? just relax the assumption of diagonal covariance matrix? 4. Will the proposed method suffer leakage?

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

Limitations are adequately addressed.

Summary

This paper introduces a novel concept dependency modelling scheme via an explicit distributional parameterization based on multivariate Gaussian distributions. This allows for capturing the dependencies between different concepts, while giving rise to an effective intervention strategy. The experimental results vouch for the efficacy of the proposed approach, which allows for end-to-end training, and exhibits on-par or improved performance before and after interventions.

Strengths

This constitutes a well-motivated approach based on some probabilistic arguments for capturing and examining concept dependency. The overall idea is simple and easy to follow.

Weaknesses

Even though the proposed approach uses either an amortized or global formulation, the complexity of learning the covariance matrix either way surely introduces a lot of complexity. This is especially true in cases where the number of concepts are large, a common occurrence in complex datasets with many classes such as CIFAR-100 and ImageNet (not explored in this work). This complexity is further burdened by the Monte Carlo sampling scheme; commonly, 1-10 samples are enough, but the authors report the usage of 100 MC samples which would greatly slow down the training process. In this context: 1) What is the complexity compared to a standard diagonal approach? 2) It would be important to assess the complexity and performance in the case of larger datasets like ImageNet. The authors use the CLIP generated data of [1] for CIFAR-10, so applying the approach to CIFAR-100 and ImageNet should be quite easy. 3) In my experience with MC-based methods, I find Table 2 a bit hard to believe, especially with 100 MC samples that the authors report. Can the authors provide wall time measurements for the per-epoch time for each method? Even for just Hard CBM and Amortized SCBM. 4) Having access to the code, so the experiments could be validated would also be important. 5) How does the number of MC samples affect the complexity and performance? I find an ablation study to be necessary here. 6) How many GPUs did the utilized cluster have? The considered Bernoulli formulation is very similar to the work in [2], which they call concept discovery. It constitutes a data-driven Bernoulli posterior for concept selection and the formulation is very similar. I find this method more appropriate compared to other approaches mentioned in the related work and the experimental evaluation, and some discussion/results should be included in the main text. In Table 1, the authors report the concept accuracy which I assume is based on a binary accuracy. Recent works have suggested that maybe this is not a good metric in the context of interpretability, since most concepts are sparse and this can lead to misleading results [3]. Can the authors report the Jaccard similarity between the ground truth and the obtained concepts for all the methods? [1] Oikarinen, T., et al., Label-free concept bottleneck models, ICLR 2023 [2] Panousis, K. P., et al., Sparse Linear Concept Discovery Models, ICCV CLVL 2023 [3] Panousis, K. P., et al., Coarse-to-Fine Concept Bottleneck Models, Arxiv 2024

Questions

Please see the Weaknesses section.

Rating

5

Confidence

4

Soundness

2

Presentation

3

Contribution

3

Limitations

The authors mention the limitations in the dedicated section. The main limitation of the proposed approach is complexity.

Summary

This paper presents a method of performing interventions on Concept Bottleneck Models. The method parametrizes the space of concepts with a generative model of Bernoulli distribution and concept logits with a normal distribution, whose mean and variance depends on the input data distribution. The method also compares amortized covariance, which is instance-based, and a global covariance, marginalized over all samples. Rather than doing the interventions one-by-one as in previous works, this method also modifies a single concept by hand, which in turns modifies other related concepts using the covariance learned. Experiments demonstrate that the method performs competitively against related CBM works that also allow for intervention, and is much faster during inference. During intervention, the model performs competitively against other baselines.

Strengths

The problem presented in this work is well-motivated and is of great importance to the current field of interpretable machine learning. While there are existing works that also formulate the concepts as a known distribution, this method has demonstrated well its advantages from the lens of intervention. Hence, this work has met the bar in terms of novelty and contributions in this area of research. The experiments of this work are also solid and adequate in addressing the claims made. While the performance is sometimes worse than related methods, they are still comparable and is not the main focal point of this work.

Weaknesses

While the overall writing of the paper is clear, Section 3.3 was a bit challenging to follow, especially the part about the confidence region. It is not entirely clear to me what the confidence region is of. As an example, L183-184 “the likelihood-based confidence region provides a natural way of capturing the region of possible $\mathbf{\eta}’_{\mathcal{S}}$ that fulfil our desiderata”. The confidence of what? What desiderata? I suggesting either adding a diagram, a bit more background, a reminder, or an example, on what the intervention is trying to achieve so it more clearly demonstrates the process.

Questions

Often times we don’t have many labeled samples of fine-grain annotations (like CUB) or the annotation can be noisy (like CLIP annotations). To what extend do these two factors affect the learning of the distributions in your method? Is it true that if the global dependency structure is of poor quality (correlation matrix), then this can severely impact the downstream performance and interpretability?

Rating

7

Confidence

3

Soundness

4

Presentation

3

Contribution

4

Limitations

A key part of this method is learning a good dependency matrix for each sample or globally. I would expect this requires a lot of samples and can severely impact the performance if the matrix is not representative of the relationships between concepts. This is clearly stated in the work, hence the limitations are sufficiently stated.