Summary
This paper considers to adaptively determine inference time steps of spiking neural networks to improve the tradeoff between accuracy and time. Two methods are proposed. The first one uses confidence score thresholding. The second one introduces an additional policy network to predict the number of timesteps by reinforcement learning. Experiments show the effectiveness of the proposed methods for direct SNN training and ANN-SNN conversion methods.
Strengths
1. This paper considers early-exit of spiking neural networks, which can effectively improve the efficiency of SNNs. The idea of dynamically determining inference time steps based on the difficulty of inputs is interesting.
2. Extensive experiments on static and neuromorphic datasets as well as qualitative assessment are conducted.
Weaknesses
1. Dynamic inference time steps of SNNs, especially based on confidence scores, were also explored by (possibly concurrent) recent works [1,2], which can be discussed. The idea of confidence score is very simple and straight-forward.
2. For SEENN-II, an additional policy network is required for inference, and it is unclear how this can be deployed. From the descriptions (and codes), the policy network is an ANN rather than SNN, so it is not compatible with the main SNN for hardware deployment. This poses challenges and also arouses questions of why considering such hybrid architectures. Additionally, SEENN-II seems not to be flexible, i.e. after training it cannot make tradeoff between accuracy and time/energy.
3. From the descriptions, it is not clear enough whether the energy consumption estimation of SEENN-II consider policy networks. This should be included and more detailedly discussed. For example, on ImageNet, is the policy network an ANN ResNet-34 (as shown in code)? Then it may consume more than the SNN part. The energy result on ImageNet is missing.
[1] Wu, D., Jin, G., Yu, H., Yi, X., & Huang, X. (2023). Optimising Event-Driven Spiking Neural Network with Regularisation and Cutoff. arXiv preprint arXiv:2301.09522.
[2] Li, C., Jones, E., & Furber, S. (2023). Unleashing the Potential of Spiking Neural Networks by Dynamic Confidence. arXiv preprint arXiv:2303.10276.
Questions
Can the authors discuss more about conceptually how to determine the inference of (deep) SNNs (in small time steps) with confidence signals from the top layer? Note that the common time steps of SNNs simply refer to time steps for each layer and do not consider propagation across layers. However, if we consider asynchronous neurons, e.g. on neuromorphic hardware, it also takes time to propagate signals across layers, and the actual delay may be layer number plus time steps per layer. For deep SNNs (e.g. >18 layers in this paper), it takes much time to obtain the confidence score from the top layer, which may be longer than the small time steps per layer. Then how can it effectively control such small time steps?
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Limitations
The authors do not discuss limitations and societal impact.