Towards the Characterization of Representations Learned via Capsule-based Network Architectures

Experiments on the small-scale dataset

In the literature, MNIST, CIFAR10, affNIST, or SVHN are the datasets frequently used by the community for conducting research on CapsNets-based architectures. Therefore, we opted to conduct our analysis on a variety of these datasets so that our method and reported results could compared against by future methods. Beyond these standard datasets, we conducted experiments on CelebA and PASCAL datasets which are considered much complex in comparison with MNIST and CIFAR10. At this point, conducting experiments on ImageNet or similar large-scale datasets will only contribute to indicate whether the observations made in the paper remain on such big data settings. Due to the large scale of these datasets, the density of our analysis, our reduced computational resources and the limited discussion period, conducting an experiment on ImageNet might not be feasible (even partially). We will attempt to conduct an experiment on the CIFAR100 dataset, which possesses a significantly larger number of classes when compared to the datasets considered in the paper and do our best to provide results during the discussion period. We believe this should provide insights on the observations made in our work under a larger scale dataset.

Clarity on the Perturbation Analysis (Sec. 4.1)

Regarding the clarity of presentation of Section 4.1, We checked the section and we agree with you regarding the following; $A\prime$ where indeed refers to the flattened activation obtained from all examples of a given layer. Where its dimension $S\prime$ refers to the number of examples corresponding to a class $c$ and $A\prime$ refers to the total number of flattened activations. This matrix represents all the activations of all the units in layer l across all the examples of class c. We agree that to preserve the flow the superindex l should be in place ($A_c^l$) in order to preserve its connection to its corresponding layer. From there we compute a matrix $\eta_c^l = [min(A^l_c) ; max(A^l_c) ; mean(A^l_c); std(A^l_c)] \in\mathbb{R}^{[4 {\times} A\prime]}$ is computed which represents the first-order statistics for the activations of every unit within layer l for class $c$. A similar approach is conducted with a larger matrix $A^l_{all}$ containing activations from examples of all classes in the dataset. It is from these $A$ that the empirical activation range ($\alpha$) is computed at the unit level. In the text of the paper have introduced an additional matrix $\eta$ to refer to the first-order statistics computed from their corresponding $A$ matrices. We are considering droping these $\eta$ matrices for clarity. In the revised version of the paper, we have addressed these notation unclarities plus other typos highlighted by the reviewer. We thank the reviewer for the meticulous revision of this part of the manuscript we believe the provided feedback improves the clarity and flow of the content.

Magnitude of the applied perturbation

We thank the reviewer for highlighting this very important aspect, we are currently conducting some experiments to further elaborate on this aspect. We hope to provide more insights on this before the end of the discussion period.

Clarity/Intelligibility of the presentation of some figures and tables.

Thanks for bringing the issue with the missing color bar (Fig.2/Sec. 6.1) to our attention, we already updated the color bar where the yellow indicates to higher classification accuracy. Similarly, we updated Fig.4 where $D0$ ${−}D9$ represents the classes as you stated. For clarity, we also updated the title of both Fig.4 and Fig.6.

Assessing Part-Whole Relationships via Relevance Mass Accuracy (RMA)

The description that the reviewer provided for the operation of RMA is accurate. It indeed measures the spatial coverage between a heatmap (matrix of continuous values) and a matrix of positive values. (reference) In our work, we estimate the level to which features encoded in a trained CapsNet encode part-whole relationships by measuring the spatial overlap between the responses of two units from different levels in the architecture that have a strong connection between them. The main intuition is that given the hierarchical representation that a CapsNet is expected to encode across layers, it is expected that more granular elements (parts) are encoded at earlier layers of the architecture and coarser elements (whole) are expected to be encoded at latter layers as a composition of the part elements. Therefore, spatially-speaking, features coming from a part element is expected to lie within the extent the whole element it contributes to compose. Hence, we measure the level to which this type of encoding is present by measuring the spatial coverage that responses at earlier capsule layers in the architecture have wrt. latter layers. Following the formulation of RMA, the $2D$ positive matrix is obtained by thresholding the response of the latter layers (whole) and the continuous matrix is defined by the response of the earlier layer (parts). In order to obtain the $2D$ response of a given layer, we apply the methods from [Simoyan et al., 2014] which, given a unit of interest, an input $x$ and a predicted class label $y$, generates a heatmap highlighting the spatial regions in the input $x$ which determine the prediction $y$.

Expected results from the Relevance Mass Accuracy (RMA) analysis (Sec. 6.3)

The output of the RMA metric is a value in the range $[0,1]$ indicating the coverage between the two input matrices with $1$ indicating a high coverage. The results reported in our analysis (Table 3) indicate that, when a threshold of $0.5$ is used to produce the positive reference matrix, the RMA score is on average $~0.04$ among the considered datasets. We notice this score increases as we use less strictive threshold values (e.g. $0.1$), however at this level the spatial extent of the candidate part unit is so coarse that hinders a local analysis. The lower layer did encoded parts of the elements encoded at deeper layers we would have expected a layer spatial coverage/overlap.

Experiments on ConvNets/ Transformer based models

Regarding conducting experiments based on other types of networks, while our methodology is applicable to other architectures, the aim of this paper lies on the interpretability of CapsNets-based architectures.

Input size of the considered dataset

Regarding the input size of the datasets used in our analysis, on the one hand, given our limited computational resources, we opted to conduct our analysis on a smaller input size. On the other hand, we believe that using larger input sizes could have helped us improve the level of detail of the provided qualitative results.

Experiments on the small-scale dataset

In the literature, MNIST, CIFAR10, affNIST, or SVHN are the datasets frequently used by the community for conducting research on CapsNets-based architectures. Therefore, we opted to conduct our analysis on a variety of these datasets so that our method and reported results could compared against by future methods. Beyond these standard datasets, we conducted experiments on CelebA and PASCAL datasets which are considered much complex in comparison with MNIST and CIFAR10. At this point, conducting experiments on ImageNet or similar large-scale datasets will only contribute to indicate whether the observations made in the paper remain on such big data settings. Due to the large scale of these datasets, the density of our analysis, our reduced computational resources and the limited discussion period, conducting an experiment on ImageNet might not be feasible (even partially). We will attempt to conduct an experiment on the CIFAR100 dataset, which possesses a significantly larger number of classes when compared to the datasets considered in the paper and do our best to provide results during the discussion period. We believe this should provide insights on the observations made in our work under a larger scale dataset.

Regarding related recent efforts

In [1], interpretation capabilities of CapsNets were investigated by analyzing coupling coefficients that link the connections between capsule layers. This analysis helps understand how CapsNet predicts amyotrophic lateral sclerosis (ALS) or Healthy, it reveals which primary capsules were activated when the model predicts ALS, similar to how we interpret predictions in our methodology. In addition, coupling coefficients were averaged, and the results are presented in heatmaps/histograms, highlighting the differences in capsule activation between both classes. In our case, we produce heatmaps that highlight the spatial regions in the input which determine the prediction. Similar to our work, [2] conducted a perturbation analysis of the predicted vector which was used to link learned representation to visual attributes such as thickness, orientation, etc. by reconstructing the perturbed vector using handwritten-digit images. Similar to us, this work was also conducted using a perturbation analysis ([-0.2,0.2] with steps of 0.06). Different from our work, where a methodology is put forward for the extraction of this perturbation range, the range considered in [2] seems to have been selected in a rather arbitrary manner. [3] proposed an interpretable multimodal fusion method (IMCF) aiming at getting insights on how modalities interact and which ones are significant for predictions made by the model. Similar to our work, routing coefficients are considered to guide the interpretation process. In contrast, [3] introduces this interpretation capability by the modification of the CapsNet architecture via the integration of BiLSTM/LSTM components. [4] proposes a CapsNet for the classification of imagined phonemes and words in EEG signals. This effort differs from ours on that it uses the learned representation of the output vectors in the class capsule layer to construct activity maps (similar to heatmaps) of brain activity based on different categories. These maps explain the most active parts of the brain during the process and they are produced based on statistical features obtained from EEG speech imagery signals. However, in our methodology, we produce gradient-based heatmaps that explain the predictions made by the model. [1] Predicting the prevalence of complex genetic diseases from individual genotype profiles using capsule networks. "Nature Machine Intelligence" (2023) [2] Quantum Capsule Networks. Liu, Zidu, et al. "Quantum Science and Technology" (2022) [3] Interpretable Multimodal Capsule Fusion. Wu, Jianfeng, Sijie Mai, and Haifeng Hu. "IEEE/ACM Transactions on Audio, Speech, and Language Processing" (2022) [4] Interpretation of a deep analysis of speech imagery features extracted by a capsule neural network. Macías-Macías, José M., et al. "Computers in Biology and Medicine" (2023)