Scholarlink

<p>Spiking neural networks (SNNs) present a highly energy-efficient alternative to artificial neural networks (ANNs), due to their strong biological plausibility, complex spatial-temporal dynamics, and event-driven computation. While numerous researchers have made significant efforts to implement SNN-based models for resolving machine learning problems, their application in natural language processing (NLP) tasks remains relatively unexplored. The main reason is that the original SNNs suffer from low training efficiency due to the complex spatial-temporal dynamics as well, which constrains the capability of efficient language modeling using SNN-based systems. In this paper, we propose <span>SPASRNN</span>, an enhanced spiking recurrent neural network to address its application on NLP tasks, which is designed to improve the performance of SNNs with recurrent structures. Furthermore, we apply the sparse gradient descent algorithm to SPASRNN to speed up the training. By incorporating a complete layer of recurrent connections, <span>SPASRNN</span>can effectively capture the essential long-term dependencies that hold critical semantic information in texts, thus improving the performance of <span>SPASRNN</span>. We are the first to apply spiking recurrent networks with the sparse gradient descent algorithm to resolve NLP problems. We validate our approach on two NLP benchmark datasets (IMDB and AG NEWS). The experimental results indicate that our <span>SPASRNN</span>achieves competitive performance versus popular artificial neural network (ANN) models, positioning itself as a state-of-the-art solution among current SNN-based models. The codes are available at <a href="https://github.com/Nanhu-AI-Lab/spasrnn">https://github.com/Nanhu-AI-Lab/spasrnn</a></p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

SPASRNN: boosting spiking recurrent neural networks using sparse gradient decent for semantic representation learning

  • 期刊论文

  • 发表日期：

    2026年4月10日

查看全文
  • Wei Hua,
  • Yansong Chua,
  • Minghao Wang,
  • Qian Zhang,
  • Kun Wang,
  • Yangyang Shu

摘要

Spiking neural networks (SNNs) present a highly energy-efficient alternative to artificial neural networks (ANNs), due to their strong biological plausibility, complex spatial-temporal dynamics, and event-driven computation. While numerous researchers have made significant efforts to implement SNN-based models for resolving machine learning problems, their application in natural language processing (NLP) tasks remains relatively unexplored. The main reason is that the original SNNs suffer from low training efficiency due to the complex spatial-temporal dynamics as well, which constrains the capability of efficient language modeling using SNN-based systems. In this paper, we propose SPASRNN, an enhanced spiking recurrent neural network to address its application on NLP tasks, which is designed to improve the performance of SNNs with recurrent structures. Furthermore, we apply the sparse gradient descent algorithm to SPASRNN to speed up the training. By incorporating a complete layer of recurrent connections, SPASRNNcan effectively capture the essential long-term dependencies that hold critical semantic information in texts, thus improving the performance of SPASRNN. We are the first to apply spiking recurrent networks with the sparse gradient descent algorithm to resolve NLP problems. We validate our approach on two NLP benchmark datasets (IMDB and AG NEWS). The experimental results indicate that our SPASRNNachieves competitive performance versus popular artificial neural network (ANN) models, positioning itself as a state-of-the-art solution among current SNN-based models. The codes are available at https://github.com/Nanhu-AI-Lab/spasrnn