Spiking PointNet: Spiking Neural Networks for Point Clouds

Summary

This paper presents the Spiking PointNet, a novel solution for processing point cloud data. The authors propose a "train-less, learn-more" strategy, which leverages the Surrogate Gradient (SG) method to train the SNN over a single time step, while deploying multiple time steps during the inference phase to optimize performance. The authors provide both theoretical analysis and empirical evidence to demonstrate that their unique strategy surpasses the conventional method of training the SNN over multiple time steps. To further enhance the performance, they introduce a membrane potential perturbation method into their framework.

Strengths

1.The "train-less, learn-more" strategy is interesting. 2.The result is really good on various datasets, even outperforms its ANN counterpart in certain scenarios, an achievement rarely seen in SNNs. 3.The paper is characterized by its clear and concise writing style, complemented by easily comprehensible illustrations.

Weaknesses

1.Section 3.4 seems unclear. If the gradient distributions, as depicted in Figure 4, do not match up as expected, would the membrane potential perturbation method introduced in Section 3.5 still be effective? Please clarify this point. 2.Can this method be applied to point segment task? 3.In table 6, the 10^8 is misspelling. Please correct it.

Questions

Please see weaknesses.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

2 fair

Presentation

3 good

Contribution

3 good

Limitations

I find no potential negative societal impact.

Summary

To investigate whether SNN can be suitable for 3D data, this paper proposes a Spiking PointNet for point cloud classification tasks. The authors mainly focus on two key issues: (i) Training SNNs with large time steps; (ii) Reducing the extensive memory and computation cost for the proposed spiking point model. The results on ModelNet10 and ModelNet40 datasets show the effectiveness of the proposed Spiking PointNet.

Strengths

i) The topic of SNN for 3D point cloud classification is novel rather than 2D visual recognition. ii) The writing is straightforward, clear, and easy to understand.

Weaknesses

i) The two key issues (i.e., training large times steps, and reducing the extensive memory and computation cost) do not seem to be well solved. Table 1 and Table 2 demonstrate that the max time step is 4. Table 6 shows that the energy of Spiking PointNet is only 3.8 times less than of its ANN counterpart. In other words, the Spiking PointNet may bring a huge increase in inference speed than ANN-based architecture, but the reduction in power consumption is almost weak. ii) The simulation calculation of Spiking PointNet power consumption in Section 4.2 does not seem to be so convincing. The proposed Spiking PointNet are not really deployed on neuromorphic chips (e.g., Loihi2), thus the simple power consumption simulation does not make sense. As far as I know, most existing SNNs cannot be deployed to neuromorphic chips. Please explain whether the proposed Spiking PointNet can be friendly to deploying to neuromorphic chips with lower consumption.

Questions

i) Could you please explain why the proposed SNN method has better performance than the corresponding ANN? I am very curious about this conclusion. I would like more theoretical explanations, especially informational theory, rather than experimental explanations. ii) Could the proposed Spiking PointNet be deployed on a neuromorphic chip? Please explain from the perspective from the designed structure of Spiking PointNet. iii) Why did the authors add the MPP module to Spiking PointNet? Perhaps, the MPP is to improve the performance. However, this module is independent and generic for any SNN model. Thus, it is difficult to form a whole with the topic of the proposed Spiking PointNet.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

No. Some suggestions may be improved for Spiking PointNet: (i) Processing point clouds from an asynchronous way or event-driven SNN; (ii) Leveraging temporal cues from point cloud streams; (iii) Attempting to processing event-based data (i.e., 3D event cloud) from event cameras.

Summary

The authors replaced the ReLU activation function of the original PointNet with the LIF neurons and named it spiking PointNet. The work offers trained-less but learning-more tricks and corresponding analysis. The effectiveness of spiking PointNet is evaluated on the ModelNet10/40 datasets, specifically in point cloud classification.

Strengths

In general, the submission is well-written and easy to follow.

Weaknesses

-1. If I understand correctly, the authors only replaced ReLu with LIF. If so, whether the first or not does not make sense as we can make first spiking VGG-16, first spiking ResNet, etc. -2. It is unclear to me what's the output of a LIF neuron. Is it a potential value or spiking? I guess the authors used potential value. -3. How does the leaky part of LIF impact the spiking PointNet? The authors claimed inference multiple times offers better performance. I am wondering, if we input a point cloud multiple times with different time intervals, what are the performance differences? -4. What's the processing speed with different T? -5. The validations are only with ModelNet. I think validations with more datasets are necessary, such as ShapeNet dataset.

Questions

Please see weakness section

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

No limitation is discussed. If the work only replaces ReLU with LIF, I think the authors should show whether the replacement and training tricks would work with other models, such as VGG, and ResNet.

Summary

In this paper, the authors introduce the spiking neural network into 3D data processing, the point cloud data. In specific, the authors analyze the training difficulty when using SNNs for the point cloud recognition and find that number of timesteps should not be increased in the training phase. Their framework, the trained-less but learning more, obtains better performance than aligning the number of timesteps in training and inference.

Strengths

+ Well-studied SNNs on the point cloud dataset for the first time. This paper brings the advantage of energy efficiency of neuromorphic computing to 3D vision topic. + Interesting, counter-intuitive findings but a surprisingly well-performance method.

Weaknesses

- The authors claimed theoretical analysis in Section 3.4. However, I found it mostly empirical analysis. I wonder where is the theory? - What will happen if the timesteps are increased more than 4 during inference?

Questions

Is it possible to convert an ANN PointNet to SNN?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

See above for improvements

Thanks for the rebuttal

I would like to thank the authors for their reply. It is interesting to see the accuracy maintains a high value even after increasing the number of timesteps to 8. Based on this and the value of being the first to investigate the SNNs in the 3D vision area, I would recommend a publication at NeurIPS. Therefore, I increase my score to 6.

Thanks for the rebuttal

Thank you for the authors' response. I still have reservations regarding the deployment of the proposed method on neuromorphic chips. Additionally, I remain skeptical about the conclusion that the SNN method performs better than the corresponding ANN. If this conclusion is valid, it would be groundbreaking news in the field of artificial intelligence, particularly in neuromorphic computing. Taking all these factors into consideration, I will uphold my original score of Borderline Reject for this work.