SpikePipe: Accelerated Training of Spiking Neural Networks via Inter-Layer Pipelining and Multiprocessor Scheduling

Spiking Neural Networks (SNNs) have gained popularity due to their high energy efficiency. Prior works have proposed various methods for training SNNs, including backpropagation-based methods. Training SNNs is computationally expensive compared to their conventional counterparts and would benefit from multiprocessor hardware acceleration. This is the first paper to propose inter-layer pipelining to accelerate training in SNNs using systolic array-based processors and multiprocessor scheduling. The impact of training using delayed gradients is observed using four networks training on different datasets, showing no degradation for small networks and <inline-formula><tex-math notation="LaTeX">$<10$</tex-math></inline-formula>% degradation for large networks. The mapping of various training tasks of the SNN onto systolic arrays is formulated, and the proposed scheduling method is evaluated on the four networks. The results are compared against standard pipelining algorithms. The results show that the proposed method achieves an average speedup of 1.7<inline-formula><tex-math notation="LaTeX">$\times$</tex-math></inline-formula> compared to standard pipelining algorithms, with an upwards of 2<inline-formula><tex-math notation="LaTeX">$\times$</tex-math></inline-formula> improvement in some cases. The incurred communication overhead due to the proposed method is less than <inline-formula><tex-math notation="LaTeX">$0.5$</tex-math></inline-formula>% of the total required communication of training in networks with convolutional layers.