Spiking Neural Networks (SNNs) have benefited from advancements in time-based BackPropagation (BPTT) techniques, enabling the development of deep SNNs for large-scale tasks. However, this method faces challenges such as high memory usage and long training times due to its sequential, layer-by-layer approach. To address these issues, we propose Joint Local BackPropagation (J-LcBP), a method that enhances training stability by enabling joint learning across adjacent local layers, thereby improving the accuracy of local learning algorithms. We also introduce two strategies for integrating local outputs to further boost performance. Experimental results on the CIFAR-10 dataset show that J-LcBP improves test accuracy by 1%, 2%, and 4% for network depths of 10, 18, and 34, respectively, compared to traditional independent local learning. Our approach outperforms existing local training methods, reducing memory usage for storing intermediate states to 13.4% and potentially halving training time through parallel processing.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Breaking the Bottleneck of BPTT: Joint Local Backpropagation for Deep SNNs

  • Jing Wang,
  • Maozhen Han,
  • Lang Xue,
  • Ying Liu,
  • Hong Qu

摘要

Spiking Neural Networks (SNNs) have benefited from advancements in time-based BackPropagation (BPTT) techniques, enabling the development of deep SNNs for large-scale tasks. However, this method faces challenges such as high memory usage and long training times due to its sequential, layer-by-layer approach. To address these issues, we propose Joint Local BackPropagation (J-LcBP), a method that enhances training stability by enabling joint learning across adjacent local layers, thereby improving the accuracy of local learning algorithms. We also introduce two strategies for integrating local outputs to further boost performance. Experimental results on the CIFAR-10 dataset show that J-LcBP improves test accuracy by 1%, 2%, and 4% for network depths of 10, 18, and 34, respectively, compared to traditional independent local learning. Our approach outperforms existing local training methods, reducing memory usage for storing intermediate states to 13.4% and potentially halving training time through parallel processing.