Enhancing Adaptive History Reserving by Spiking Convolutional Block Attention Module in Recurrent Neural Networks

Summary

The present study introduces a novel model of spiking recurrent neural networks, which incorporates a spike convolutional block attention mechanism. This model is referred to as SRNN-SCBAM. The primary objective is to effectively incorporate historical information into the spatial and temporal characteristics of spatiotemporal patterns, resulting in improved memory retrieval and elimination of redundant historical data. The efficacy of the proposed model in leveraging historical information and attaining high precision has been validated through experiments conducted on DVS128-Gesture datasets.

Strengths

The practical issue of invoking adaptive memory in spiking recurrent neural networks has been addressed in this paper. The primary objective is to incorporate a Spiking Convolutional Block Attention Module into the gating computation of Spiking ConvLSTM networks. This would enable the selective activation of historical information and the elimination of superfluous history during the training phase. This will provide the iterative calculation with a favorable initial state. The motivation is interesting and the idea of utilizing a spiking convolutional block attention mechanism to acquire historical data in gating computation appears intriguing.

Weaknesses

1.More ablations of the different gates being open or closed in Table 1 are required because the influence of ‘i’ is unclear. And if there exist the adaptive parameters to control the possibility of gate open, could it be better for RSNN to achieve better performance? 2.There is a typo in Figure 4.

Questions

1.When encounter a new dataset, how the model works and how the architecture should be designed? I don't see discussions of this part in the paper. 2.More ablations of the different gates being open or closed in Table 1 are required because the influence of ‘i’ is unclear. And if there exist the adaptive parameters to control the possibility of gate open, could it be better for RSNN to achieve better performance?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The limitations are the same as the weakness.

Summary

This study proposed the spiking recurrent neural networks model with a spiking convolutional block attention module component (SRNN-SCBAM). The proposed model invokes the historic information in spatial and temporal channels adaptively through spiking CBAM, which brings the advantages of efficient memory calling and history redundancy elimination. The experimental results show that the proposed SRNN-SCBAM model makes better use of the historic information in spatial and temporal dimensions with less memory space, and achieves higher accuracy compared to other models.

Strengths

1. The learning algorithm for recurrent SNNs is simple and easy-to-understand. 2. Well-thought-through reuse of standard modeling and algorithmic components to solve the memory calling and history redundancy elimination in recurrent SNNs.

Weaknesses

Although the motivation is reasonable and the proposed methods are simple and easy to implement, the experimental results could be supplemented to indicate the advantages of the proposed model.

Questions

1. Table 2: What is the network structure in [23]? The network structures should be presented more clearly to show the components in the network models. 2. Why the performance of Spiking ConvLSTM with Spiking CBAM is quite higher than that with CBAM in Table 1? 3. What is the parameter of alpha used in Equation (7)? 4. It seems the proposed Spiking ConvLSTM is suitable for long-term time-series datasets due to its adaptive memory maintenance. Could the proposed model be applied to other time-series datasets?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Some details of training the Recurrent SNNs are missing in the paper (only mentioned by a few sentences). It would be better to also include those settings in the appendix.

Summary

The authors proposed a Recurrent spiking neural network (RSNN). The essential component of the proposed RSNN is a spiking conv block attention module (SCBAM), which contains channel and spatial attention blocks. The proposed method is validated with classification tasks on CIFAR10-DVS and DVS128 gesture datasets.

Strengths

N/A

Weaknesses

-1. The writing of the submission should be dramatically improved. Many things are hard to follow. In lines 242-243: "Although the ...is lower than LIAF-Net, ...". In Table 2, [23] provides lower accuracy than the proposed method. Therefore, I cannot really trust the provided experimental results. -2. lines 114-115: "Spiking Convlstm is more suitable for removing facial expression data...." Why? -3. SNN is different from CNN, which can process temporal information without any treatment, especially since many works have proved that, such as [R-1], [R-2]. The process of membrane potential release, accumulation, and triggering spiking is a natural temporal extractor. I do not think using RNN-based architecture is a meaningful way to go in the SNN domain. At the minimum, the authors should provide experimental results of the comparison with these methods, such as [R-1] and [R-2]. -4. How does the potential threshold of LIF impact the performance of the proposed methods? -5. How does the number of LSTM steps impact the performance of the proposed methods? [R-1]: Biologically Inspired Dynamic Thresholds for Spiking neural networks. [R-2]: Spiking Transformers for Event-based Single object Tracking

Questions

Please see the weaknesses section.

Rating

3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

2 fair

Presentation

1 poor

Contribution

1 poor

Limitations

No limitation is provided. From my perspective, the work follows the thoughts of ANN without an insightful understanding of SNN, especially the temporal processing power of SNNs. SNNs offer compelling temporal feature extraction capability without any special treatment, evidenced by many works already. In addition, I did not see the novelty of the proposed approach from either ANN or SNN. Therefore, I think the submission is way below the bar of NeurIPS.

Summary

This article proposes a spike recurrent neural network model with a spiking convolutional block attention mechanism, called SRNN-SCBAM. Its main idea is to adaptively call historical information in the spatio-temporal features of the spatio-temporal pattern, which has advantages in efficient memory calls and eliminating historical redundancy. The experiment on the DVS128 gesture dataset is conducted to verify the effectiveness of the model in utilizing adaptive historical information. The idea of the article is clear, the textual description is clear, and examples are provided to illustrate results.

Strengths

1.The paper is easy to read, and generally well written. 2.The proposed model of RSNN is simple but effective, the motivation is clear. 3.Experimental evaluations were conducted on two different neuromorphic datasets to demonstrate the performance improvement with the proposed SCBAM under different settings.

Weaknesses

There is a lack of explanation for some parameter settings. In addition, there are some shortcomings in the experimental aspect of the paper. The experimental results mentioned in the article on the Cifar10-DVS generally can be supplemented with appropriate visualization results.

Questions

1.The parameter setting about the time constant in the first paragraph of section 2.4 should be explained in detail. 2.In addition, there are some shortcomings in the experimental aspect of the paper. The visualization experimental results mentioned in the article on the Cifar10-DVS can be supplemented with appropriate experiments.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

4 excellent

Presentation

3 good

Contribution

4 excellent

Limitations

1.The parameter should be explained further. 2.More visualization results could be added.

Thanks for the response

I would like to thank for the response. My concerns have been addressed.