Neural Sculpting: Uncovering hierarchically modular task structure in neural networks through pruning and network analysis

Summary

When conventionally trained, neural networks do not demonstrate structural properties like input separable functions or reusability of sub-modules. The authors investigated this phenomenon and proposed iterative pruning to enhance structural properties of neural networks. They demonstrate the effectiveness of their method on boolean functions and a modified MNIST dataset.

Strengths

* The paper is well-written with nice motivations and clear logical flow. * The paper investigates this interesting scientific question "whether conventionally trained neural networks display structural properties", which is an important question for network interpretability * The paper proposes iterative pruning to enhance structural properties, which is a technical contribution.

Weaknesses

* The scope is a bit limited. The paper only discusses two properties, input separable functions and reuse of sub-modules. The examples are a bit too simple. Said so, maybe this is not too much of a problem for a scientific paper whose goal is to understand something. Still, I'd love to see your method applied to larger-scale experiments. * The novelty is not very clear to me. I'm glad to see that iterative pruning works pretty well for module reuse, but it seems there is not too many technical contribution. I suggest authors should highlight the technical contributions/comparisons to previous works. * Regarding the method, I feel encouraging modularity directly in training (e.g., adding Eq. (2) as a penalty in training) may further enhance structural properties. I'd love to see how this trick changes the outcomes (especially for the failed cases).

Questions

* Figure 3 and Figure 4 seem to have the exact same titles and captions. Should distinguish them more clearly. * I'm not very sure about your definition of reusability (your definition sounds more like "shared features" to me). An example I would count as reuse of submodule is: consider input (x1, x2), output ((x1-x2)^2, (x1+x2)^2), the squared function should be learned twice (reused). However, there is probably no shared feature in a trained network. A trained network (e.g., a fully-connected network) can only learn the squared function twice and independently, even with your pruning strategy I guess. * Do you expect your method to generalize to larger models, e.g., large language models?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

None

Summary

This paper conducts an investigation of hierarchical modularity in neural networks by studying boolean networks. The paper studies simple hierarchically modular boolean functions and learns them using MLPs. It then propose metrics to discover this modularity by examining 1) input separability and 2) reusability of sub-functions. Finally, the paper provides a clustering based method to identify the modules in neural networks applied to arbitrary tasks.

Strengths

1) Clear and scientifically principled investigation into an important yet understudied facet of neural networks 2) Interesting results showing that hierarchical modularity in neural networks often doesn't emerge with standard training but with pruning (both edge and neuron), the sparsity forces the networks to become hierarchically modular. 3) Novel method to find with some empirical backing

Weaknesses

1) The experiments on more realistic datasets e.g. MNIST can be expanded to link the findings of the paper more concretely to what is observed in practice.

Questions

Currently, the experiments on MNIST to verify the method to discover modularity do serve as a proof of concept but more extensive experiments would be needed to confirm the effectiveness of the proposed approach to identify modules in neural networks in general. In particular, extensions to CNNs on more realistic image datasets might be very interesting. It might be useful to conduct experiments with a subset of classes from a dataset like CIFAR100 or ImageNet where the superclasses and subclasses often offer some natural opportunities for hierarchical modularity.

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

N/A

Summary

The paper proposes a methodology for uncovering hierarchical modularity in neural networks (NNs). It combines iterative pruning and network analysis to reveal the underlying hierarchy of sub-functions in tasks. The paper demonstrates the effectiveness of the method on both Boolean functions and vision tasks using the MNIST dataset. The main contribution of the paper lies in providing a novel approach to uncover hierarchical modularity without prior knowledge of the task's hierarchy. The methodology offers insights into efficient and interpretable learning systems and showcases the potential of pruning and network analysis methods in revealing and utilizing structural properties in NNs.

Strengths

The paper demonstrates a significant strength through its comprehensive experimental evaluation, encompassing modular and hierarchical Boolean function graphs, as well as tasks utilizing the MNIST digits dataset. The authors meticulously conduct numerous trials, systematically varying network parameters like depth, width, and seed values to thoroughly validate the efficacy of their proposed methodology. The experimental results substantiate the approach's ability to precisely uncover the hierarchical and modular structures within the tasks, serving as empirical evidence of the methodology's robustness and broad applicability.

Weaknesses

One potential weakness of the paper is the limited discussion and analysis of the results regarding the failures or limitations of the proposed methodology. While the experiments highlight the success rates in detecting modules and uncovering hierarchical structures, there is less exploration of cases where the methodology might not perform as well or situations where the detected modules do not align perfectly with the expected sub-functions. A more in-depth analysis of the challenges and limitations of the approach could provide valuable insights for further improvement and understanding.

Questions

1. How does the proposed iterative pruning process affect the overall performance of the neural networks in terms of learning efficiency and accuracy? 2. Are there any limitations or challenges encountered when applying the proposed methodology to more complex tasks beyond Boolean functions and MNIST digit classification? 3. Could the approach be extended to tasks with dynamic or evolving hierarchical structures, where the sub-functions change over time? 4. How does the proposed methodology compare to existing approaches in terms of accuracy, efficiency, and scalability when uncovering hierarchical modularity in neural networks?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

Minor comments: - Figure 5B1 -> the text is hard to read on a printed paper, due to small fonts in the figure. This is a minor comment, because if one is reading the PDF paperlessly, one can zoom in.