Breaking the Bottleneck of BPTT: Joint Local Backpropagation for Deep SNNs
摘要
Spiking Neural Networks (SNNs) have benefited from advancements in time-based BackPropagation (BPTT) techniques, enabling the development of deep SNNs for large-scale tasks. However, this method faces challenges such as high memory usage and long training times due to its sequential, layer-by-layer approach. To address these issues, we propose Joint Local BackPropagation (J-LcBP), a method that enhances training stability by enabling joint learning across adjacent local layers, thereby improving the accuracy of local learning algorithms. We also introduce two strategies for integrating local outputs to further boost performance. Experimental results on the CIFAR-10 dataset show that J-LcBP improves test accuracy by 1%, 2%, and 4% for network depths of 10, 18, and 34, respectively, compared to traditional independent local learning. Our approach outperforms existing local training methods, reducing memory usage for storing intermediate states to 13.4% and potentially halving training time through parallel processing.