Training-free multi-objective evolutionary search for efficient spiking neural networks
摘要
Spiking Neural Networks (SNNs) are gaining attention as energy-efficient and biologically inspired alternatives to Artificial Neural Networks (ANNs) for low-power and neuromorphic applications. However, current SNN models often rely on ANN architectures that may not fully exploit the unique properties of SNNs. While Neural Architecture Search (NAS) approaches have been successful in discovering suitable architectures for various applications, very few works have been presented on NAS for SNNs, particularly for identifying architectures that achieve high accuracy while improving the spike efficiency of SNNs. In this paper, we present a Multi-Objective Neural Architecture Search for Spike-Efficient SNNs (MONAS-ESNN) framework that uses training-free NAS to discover SNN architectures that optimize both accuracy and spike efficiency. The proposed MONAS-ESNN employs the Non-Dominated Sorting Genetic Algorithm II (NSGA-II) evolutionary algorithm to explore the architectural space and optimize both objective functions while leveraging the unique temporal dynamics of SNNs. We introduce a new Adjusted Sparsity-Aware Hamming Distance (ASAHD) metric that accurately represents spike activation diversity across different types of spiking neurons to improve the selection of high-potential architectures without training. Additionally, the proposed framework integrates specialized macro-architecture modifications to better capture temporal features in neuromorphic datasets. Experimental results on static datasets (CIFAR-10, CIFAR-100, Tiny-ImageNet-200, and SVHN), as well as neuromorphic datasets (CIFAR10-DVS and DVS128-Gesture), demonstrate that MONAS-ESNN performs better or comparable to existing SNN models in both accuracy and spike efficiency. These findings can advance automated SNN design for energy-efficient neural computing systems.