<p>Traditional ethnic sports constitute a vital component of intangible cultural heritage. Their digital preservation and automated analysis pose significant challenges, particularly in achieving high recognition accuracy under strict computational and energy constraints on edge devices. To address these challenges, we propose <i>SpikingMotionFormer</i>, a novel neuromorphic transformer architecture for efficient and accurate recognition of traditional ethnic sports movements. SpikingMotionFormer tightly integrates spiking neural networks (SNNs) with transformer-based attention mechanisms, and incorporates an adaptive spiking threshold strategy that dynamically adjusts neuronal firing behavior according to input statistics, improving both representational capacity and energy efficiency. To capture complementary motion cues, we further design a hierarchical attention-based multimodal fusion strategy that effectively combines inertial signals (accelerometers and gyroscopes) with visual information. To alleviate the reliance on large-scale labeled datasets, we introduce a self-supervised learning framework that explicitly exploits the intrinsic structure of motion data. The framework jointly leverages temporal contrastive learning to model short-term motion continuity and a cross-modal motion consistency objective to align representations across heterogeneous sensors, enabling the model to learn robust and discriminative motion representations from unlabeled data before supervised fine-tuning. Extensive experiments on benchmark datasets demonstrate the effectiveness of the proposed approach. SpikingMotionFormer achieves up to 99.85% accuracy while reducing inference latency and energy-related costs by up to 53% compared with conventional transformer-based models, highlighting its suitability for resource-constrained deployment scenarios.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

SpikingMotionFormer: a neuromorphic transformer architecture for low-power recognition of traditional ethnic sports movements

  • Binggang Zhang,
  • Jiacheng Wang,
  • Zhongliang He,
  • Yang Li

摘要

Traditional ethnic sports constitute a vital component of intangible cultural heritage. Their digital preservation and automated analysis pose significant challenges, particularly in achieving high recognition accuracy under strict computational and energy constraints on edge devices. To address these challenges, we propose SpikingMotionFormer, a novel neuromorphic transformer architecture for efficient and accurate recognition of traditional ethnic sports movements. SpikingMotionFormer tightly integrates spiking neural networks (SNNs) with transformer-based attention mechanisms, and incorporates an adaptive spiking threshold strategy that dynamically adjusts neuronal firing behavior according to input statistics, improving both representational capacity and energy efficiency. To capture complementary motion cues, we further design a hierarchical attention-based multimodal fusion strategy that effectively combines inertial signals (accelerometers and gyroscopes) with visual information. To alleviate the reliance on large-scale labeled datasets, we introduce a self-supervised learning framework that explicitly exploits the intrinsic structure of motion data. The framework jointly leverages temporal contrastive learning to model short-term motion continuity and a cross-modal motion consistency objective to align representations across heterogeneous sensors, enabling the model to learn robust and discriminative motion representations from unlabeled data before supervised fine-tuning. Extensive experiments on benchmark datasets demonstrate the effectiveness of the proposed approach. SpikingMotionFormer achieves up to 99.85% accuracy while reducing inference latency and energy-related costs by up to 53% compared with conventional transformer-based models, highlighting its suitability for resource-constrained deployment scenarios.