Spiking Graph Neural Network on Riemannian Manifolds

Summary

The authors generalize spiking GNN to Riemannian manifold, and design a new architecture of parallel forwarding so as to boost model training. Then, the proposed MSG is evaluated with 12 baselines on real graphs.

Strengths

S1. A technical strong paper, and No error is detected. It presents a Riemannian optimization to boost the SNN training and connects the proposed model to Riemannian ODE. S2. The studied problem of Spiking GNN training is interesting, since it is time consuming to conduct BPTT training of SGNN especially when the spiking train is long. S3. The authors give a new architecture of parallel forwarding so that graph is modeled in Riemannian manifold, different from the previous SGNN in Euclidean space. S4. The experiment and visualization are convincing, and the results show that Riemannian modeling of SNN generally achieves superior performance.

Weaknesses

W1. The proposed model works with the backbone of GCN, and the performance of other backbones, GAT for instance, is not examined. W2. Some experimental details are not specified, e.g., the link prediction setting with SNN baselines, which is not frequently reported in graph SNN studies.

Questions

Q1. The authors add a parallel Riemannian forwarding to accelerate the SNN training, leveraging DvM. However, it needs to specify the additional parameter and will it damage the energy efficiency of SNN? Q2. In the ablation study of Section 6.2, the training time of DvM is much less than that of BPTT, and the question is that, is DvM’s backward time of independent from the steps of spikes? Q3. Please specify the manifold of Fig. 5 in the Experiment, i.e., the construction of the manifold or its equation. Q4. What is the different between this paper and the recent paper, Continuous spiking graph neural networks, regarding ODE?

Rating

8

Confidence

4

Soundness

3

Presentation

3

Contribution

4

Limitations

The authors specify the limitations and potential negative social impact in Sec. 8.

Summary

This work proposes a spiking graph neural network on Riemannian manifolds, named Manifold-valued Spiking GNN (MSG). This work also develops a new training algorithm of differentiation via manifold (DvM) that avoids the high training overhead of BPTT methods and proves that the MSG is a neural ODE solver. Experimental results show that MSG outperforms existing spiking GNNs with low energy consumption.

Strengths

1. It is a novel idea to explore spiking GNN on Riemannian manifolds. 2. This work is technical solid. It proves in detail the theory that the MSG is a neural ODE solver. 3. The proposed MSG outperforms existing spiking GNNs in both node classification and link prediction tasks with low energy consumption.

Weaknesses

1. Figure 4(a) is confusing. In line 356-357, the author state that backward time of DvM is significantly less than that of BPTT algorithm. But in figure 4(a), the bar of DvM is higher. Is this a mistake? If not, please explain. In addition, it would be helpful to list the complete training times. 2. Figure 2 shows that the manifold representations will feed forward with the spike trains. This is acceptable during training, but during inference, since $f(\cdot)$ is an exponential function, the computation of the manifold representations involves a lot of float-point operations, which is incompatible with the nature of spiking neural networks. Please describe in detail the computation of the manifold representations during inference, especially the float-point operations involved.

Questions

Please refer to the weaknesses

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors have stated the limitations and broader impact.

Summary

In order to improve energy efficiency and performance in graph learning, the research presents a unique Manifold-valued Spiking Graph Neural Network (MSG), which combines spiking neurons with Riemannian manifolds. Differentiation via Manifold (DvM), a unique training approach, is proposed by the authors to solve the high latency issue with spiking GNNs.

Strengths

Combining spiking neurons with Riemannian manifolds is a novel idea that addresses both energy efficiency and performance Numerous experiments show that MSG is more effective and energy-efficient than both traditional and spiking GNNs currently in use.

Weaknesses

It is not the first time the Riemann manifold has been introduced into spiking neural networks. The authors lack proper discussion of these works.  The energy consumption is unknown for the proposed method due to the complexity of integrating spiking neurons with Riemannian manifolds. While the theoretical contributions are significant, the paper lacks discussion on practical applications and real-world scenarios.

Questions

Can the authors provide more details on the potential neuromorphic implementation?  Are there any specific real-world applications where MSG could be particularly beneficial?

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

See weakness and questions

Summary

This paper first analyzes the limitations of spiking GNN, representation space and model training, and then present a Riemannian model called MSG, which is connected to manifold ODE. Finally, the authors conduct experiment to show the effectiveness and energy efficiency.

Strengths

On the significance, 1.This paper relates two previously disparate research areas, SNN and Riemannian geometry. 2.Energy efficiency is of importance to the wide use of Riemannian models, given that they are typically computationally expensive. On the originality, 1.Unlike existing Riemannian GNNs, the authors design a Riemannian GNN with the spiking neuron. 2.It is a novel idea to parallel Riemannian and spiking domains, providing the opportunity for a faster training algorithm, DvM. On the quality, this is a Solid paper with few technical, 1.The theoretical results (connection to ODE) are proved, and the closed form gradient is well elaborated. 2.The empirical results are adequate, and Codes are provided for reproducibility. On the clarity, this paper is well-organized and easy to follow in general.

Weaknesses

1.In the experiment, in Table 1, the authors do not provide some of link perdition results of SNNs. Thus, it needs to specify on those SNN baselines, e.g., is SpikeNet [33] able to conduct link prediction? Why not? 2.This paper requires systematic understanding of differential geometry, and thus is not friendly to the reader not familiar to this area.

Questions

See W1, and other questions are listed as follows: 1.I have double checked Proofs 4.1 and 5.1, and the proofs are correct. However, I would like to clarify: whether or not there is a typo in Line 275 of Thm 5.2, that z and y are swapped. 2.I notice that, the authors mention another optimization method, sub-gradient [15], also different from the BPTT scheme. Thus, specify the sub-gradient, and the different between sub-gradient and the proposed DvM. 3.Given that gradient exploding and vanishing are common problems in training SNN, will MSG suffer from these issues? Also, is there any treatment for the numerical stability of Riemannian operators, e.g, sinh and cosh in the Appendix?

Rating

8

Confidence

5

Soundness

4

Presentation

3

Contribution

4

Limitations

No limitation (negative societal impact) needs to be specifically addressed.

Summary

This paper focuses on the theoretical aspects of spiking neural networks, which are a variant of neural networks closer to the human brain, particularly incorporating a time component. One of the hopes for this type of networks is that they are much more energy efficient. In this paper, the authors extend the classical spiking graph neural networks setting, which only handle graphs in Euclidean space, to the setting of Riemannian manifolds. This is highly non-trivial, requiring a specific of each spiking neuron on manifolds. Intriguingly, they then relate this new model to manifold ODEs, and also show even theoretically a significant improvement of energy efficiency as opposed to classical GNNs.

Strengths

* SNNs are very timely due to the severe energy problems of classical NNs. * Goes significantly beyond the state of the art. * Very well-organized paper, intriguing to read. * From deep math to numerical experiments, this paper covers the full range * Comprehensive analysis of the newly introduced type of SNNs, and also showing their superiority

Weaknesses

* Argument, why Riemannian manifolds are required, could be strengthened, and also what about other types of manifolds?

Questions

see weaknesses

Rating

8

Confidence

4

Soundness

4

Presentation

4

Contribution

4

Limitations

---