Scholarlink

<p>Human action recognition is pivotal in computer vision, with applications in human-computer interaction and intelligent surveillance. However, existing RGB-D action recognition methods often fail to fully capture spatio-temporal dependencies, either suffering from entangled spatio-temporal features or high computational costs in 3D feature extraction.To address this, we propose HML-STN (High-Middle-Low Spatio-Temporal Network), a lightweight RGB-D framework with an optimized 3D convolutional backbone. It adapts and extends SlowFast’s dual-path paradigm into a three-pathway architecture optimized for RGB-D multi-modal spatio-temporal learning, via three key designs: (1) Lightweight Spatio-Temporal Decomposition: 3D Shuffle and modified Inception blocks decouple spatio-temporal features via channel split-shuffle, resolving gradient conflicts; (2) Spatio-Temporal Global Attention: A Non-local-inspired module uses depth-wise/group convolution to model long-range dependencies efficiently; (3) Cross-Modal Channel Recalibration: Unlike standard gated/late fusion, CDF integrates an encoder-decoder structure and contrastive mutual information loss, dynamically fusing RGB and depth features with adaptive weights that account for inter-modal correlations. Validated on PKU-MMD, N-UCLA, and NvGesture datasets, HML-STN achieves state-of-the-art RGB+D performance: it reaches up to 97.7% (PKU-MMD, CV), 95.9% (PKU-MMD, CS), 99.6% (N-UCLA, CV), 98.2% (N-UCLA, CS) and 88.3% (NvGesture) accuracy, while maintaining lightweight efficiency for both single-modal and multi-modal scenarios.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

HML-STN: High-Middle-Low spatio-temporal network for RGB-D based human action recognition

  • 期刊论文

  • 发表日期：

    2026年3月10日

查看全文
  • Yumin Zhang,
  • Bo Zhao,
  • Yanyong Wang

摘要

Human action recognition is pivotal in computer vision, with applications in human-computer interaction and intelligent surveillance. However, existing RGB-D action recognition methods often fail to fully capture spatio-temporal dependencies, either suffering from entangled spatio-temporal features or high computational costs in 3D feature extraction.To address this, we propose HML-STN (High-Middle-Low Spatio-Temporal Network), a lightweight RGB-D framework with an optimized 3D convolutional backbone. It adapts and extends SlowFast’s dual-path paradigm into a three-pathway architecture optimized for RGB-D multi-modal spatio-temporal learning, via three key designs: (1) Lightweight Spatio-Temporal Decomposition: 3D Shuffle and modified Inception blocks decouple spatio-temporal features via channel split-shuffle, resolving gradient conflicts; (2) Spatio-Temporal Global Attention: A Non-local-inspired module uses depth-wise/group convolution to model long-range dependencies efficiently; (3) Cross-Modal Channel Recalibration: Unlike standard gated/late fusion, CDF integrates an encoder-decoder structure and contrastive mutual information loss, dynamically fusing RGB and depth features with adaptive weights that account for inter-modal correlations. Validated on PKU-MMD, N-UCLA, and NvGesture datasets, HML-STN achieves state-of-the-art RGB+D performance: it reaches up to 97.7% (PKU-MMD, CV), 95.9% (PKU-MMD, CS), 99.6% (N-UCLA, CV), 98.2% (N-UCLA, CS) and 88.3% (NvGesture) accuracy, while maintaining lightweight efficiency for both single-modal and multi-modal scenarios.