Neuronal Competition Groups with Supervised STDP for Spike-Based Classification

Summary

This paper aims to enhance the classification capabilities of SNNs using STDP learning rule. The authors introduce the Neuronal Competition Group architecture to address the limitations of existing WTA mechanisms in supervised STDP classification. This architecture promotes balanced intra-class competition and improves class separation by employing a two-compartment threshold system. The paper demonstrates the effectiveness of NCGs in achieving accuracy improvements on some relatively large datasets like CIFAR-100.

Strengths

1. The focus on local learning and the time-to-first-spike coding is highly encouraged. It improves energy efficiency in both training and inference parts, which is crucial for neuromorphic hardware applications. 2. The extensive experiments show the effectiveness of the proposed framework. Compared with other local learning rules, the proposed one achieves better performance.

Weaknesses

Global learning methods, such as surrogate gradient and ANN-to-SNN conversions, achieve significantly better performance. Consequently, in terms of accuracy, the proposed method is far from attractive. Could the authors provide a quantitative comparison between global learning methods and the proposed method across different aspects (energy, hardware implementability, etc.) to highlight the advantages of the proposed approach?

Questions

What is the latency used in the paper?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

The scalability of the proposed method to larger and more complex datasets beyond CIFAR-10 and CIFAR-100 is not thoroughly explored. Future work could address how well the NCG architecture scales with increasing data complexity and size.

Summary

This paper introduces Neuronal Competition Groups (NCGs) with a two-compartment threshold mechanism to optimize Winner-Takes-All competition in spiking classification layers using supervised STDP. NCG integration with supervised STDP rules significantly boosts image recognition accuracy on CIFAR-10 and CIFAR-100 datasets, showcasing balanced competition and improved class separation.

Strengths

The authors proposed a supervised STDP method to achieve better results on this type of task. The authors proposed a method for NCG that attempts to solve the problems encountered in STDP training.

Weaknesses

The number of experiments was not sufficient; he compared only a few exist methods. stdp is an unsupervised algorithm, and it is hard to associate it with supervised methods. Although there are some supervised stdp methods before, it is hard to believe its rationality. The features of the authors' method are extracted by other methods and do not seem to have been designed or trained by themselves. That is to say, the author's work is only limited to the classification layer. The methods of stdp are not comprehensive, especially unsupervised, and should be investigated more widely.

Questions

Can the author's method be extended to the previous feature extraction layer. The authors' experiments seem to have only a few comparisons of methods. The experiments of the authors do not seem to have more analysis about the more detailed features of the method proposed by the authors. Experiments are now limited to the results of classification, and very little analysis of method properties. stdp itself is an unsupervised local algorithm, so what is the point of extending it to supervised algorithm? If it no longer meets the local property and unsupervised property, then why not use other supervised algorithm which is better.

Rating

5

Confidence

4

Soundness

2

Presentation

2

Contribution

2

Limitations

Yes

Summary

The authors proposed the Neuronal Competition Group (NCG) and a novel competition regulation mechanism based on two-compartment thresholds, to effectively implement intra-class WTA competition in a spiking classification layer employing first-spike coding and supervised STDP training, which improves classification capabilities by promoting the learning of various patterns per class. The results on several image recognition datasets such as CIFAR-10 and CIFAR-100, demonstrate the effectiveness of the proposed methods.

Strengths

1. The authors proposed the Neuronal Competition Group (NCG) for effective intra-class WTA competition in a spiking classification layer employing first-spike coding and supervised STDP training. 2. The authors proposed a novel competition regulation mechanism based on two-compartment thresholds to promote balanced competition and fair decision-making. 3. The proposed methods significantly increased the accuracy of SOTA supervised STDP rules on several common datasets.

Weaknesses

1. It lacks clear explanation why do we need to promote the learning of various patterns per class (intra-class competition), because the final aim is to separate samples from different classes. 2. The authors talk about the advantage of proposed methods for neuromorphic hardware, which needs more analyses or discussion. Besides, its unfavorable impacts should also be considered, such as using high-precision floating-point values to represent firing timestamps. 3. The classification accuracies lag behind BP-based models (e.g., see “Spike-driven Transformer” by Yao et al., NeurIPS 2024), which are usually trained on GPUs. And the trained model can be deployed on neuromorphic hardware for inference, which is also highly energy-efficient. 4. The proposed methods improve the performance with more cost, so it’s better to add analyses or discussion about this tradeoff.

Questions

1. It may benefit better understanding to include some important parts and brief pipeline of the training algorithm in the main text, such as the definition of the average firing time. 2. The definition and role of important hyperparameters should be described in the main text. 3. The right part of Figure1 is not clearly explained, especially the “Error” indication. Besides, inhibited spikes should not exist in the “y≠1” condition? 4. See the weaknesses listed above.

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

An additional limitation maybe the effectiveness of the proposed methods on large-scale datasets like ImageNet.

Summary

The paper proposes Neuronal Competition Group (NCG), an architecture that maps each class in the task to a group of neurons using intra-class Winner-Takes-All (WTA) and competition regulation. The authors aim to implement effective WTA competition mechanisms in spiking neural networks (SNNs) employing first-spike coding and Spike Timing-Dependent Plasticity (STDP) learning rules.

Strengths

The paper is well-written, and many preliminary concepts are discussed, making it easy for readers to follow the discussion. The mechanisms discussed, such as WTA, lateral inhibition, STDP, and modulated STDP, have a solid foundation in neuroscience. Moreover, the experiments have been tested on fairly complex tasks, such as CIFAR-100, demonstrating the proposed method's effectiveness. The ablation studies provide compelling support for the design choices made for the method.

Weaknesses

The main limitation of the proposed method and experiments is that essentially all the ideas used in the method have been previously explored, so the contributions might be limited to combining them and making them work together. Additionally, a major weakness is that the method only works for a model with one layer of spiking neurons, and this layer is the last layer in the model. It is not clear why using STDP would be a better alternative than simply using a delta rule, where weight updates are just the product of the classification error and the inputs. The proposed method also requires a significant increase in the number of parameters in the last layer (by a factor of 5 in the experiments). Furthermore, all the experiments focus exclusively on static image classification tasks where no temporal information is available, which are not the most suitable tasks for SNNs. It is well known that artificial neural networks (ANNs) perform significantly better in such tasks.

Questions

- Could the authors present a comparison of the method using STDP versus a simple delta rule over the group of neurons in the classification layer? - Could the authors present some results for sequential tasks where using SNNs could be more compelling than using other recurrent models? - Could the authors elaborate on why using STDP in the classification layer is important? Local learning methods are useful for learning in deep layers since using backpropagation on custom hardware is expensive, but the last layer does not suffer from this problem. - Could the authors discuss any computational benefits of using the proposed methods over others? - Could the authors discuss the relationship between accuracy performance and the number of neurons per NCG beyond the five units used in the experiments?

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

The main limitation of the work is that the proposed method targets only the last layer of a neural network, so it is unclear why it would be a preferable alternative to other simpler methods.