Scholarlink

<p>The problem of Human Action Recognition (HAR) continues to be difficult with the intricate temporal interactions, superfluous frames, and minor visual variations that usually define similar actions. A large number of current approaches are based on either transformer or multimodal architectures, which are computationally costly and cannot be used in real-time or resource constrained systems. To overcome these shortcomings, we proposed a novel and lightweight HAR model that integrates spatial, temporal and consistency modeling in an efficient architecture. In our model, we combine EfficientNet-B2, used in efficient spatial feature extraction, with Unitary Temporal Encoder (UTE), to train long-range temporal dependencies, and Adaptive Temporal Consistency Module (ATCM), to improve local temporal consistency. The proposed system is trained and tested on the datasets UCF101 and HMDB51 with a Top-1 accuracy of 97.10% and 87% under RGB input only which is competitive among RGB-based HAR methods while maintaining low computational cost. The 9.3 million parameter model with an inference rate of 9.5 ms per 16-frame video clip is highly accurate and efficient, thus suitable in inference in real-time and at edges. Ablation studies further demonstrate that the proposed components contribute consistent performance improvements across the evaluated benchmarks.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Efficient spatio-temporal modeling for human action recognition from RGB streams using unitary temporal encoding and adaptive consistency refinement

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年6月30日

查看全文
  • Abdul Majid,
  • Yulin Wang,
  • Aqsa,
  • Zhang Xing,
  • Anwarullah,
  • Melaku Berhe Belay

摘要

The problem of Human Action Recognition (HAR) continues to be difficult with the intricate temporal interactions, superfluous frames, and minor visual variations that usually define similar actions. A large number of current approaches are based on either transformer or multimodal architectures, which are computationally costly and cannot be used in real-time or resource constrained systems. To overcome these shortcomings, we proposed a novel and lightweight HAR model that integrates spatial, temporal and consistency modeling in an efficient architecture. In our model, we combine EfficientNet-B2, used in efficient spatial feature extraction, with Unitary Temporal Encoder (UTE), to train long-range temporal dependencies, and Adaptive Temporal Consistency Module (ATCM), to improve local temporal consistency. The proposed system is trained and tested on the datasets UCF101 and HMDB51 with a Top-1 accuracy of 97.10% and 87% under RGB input only which is competitive among RGB-based HAR methods while maintaining low computational cost. The 9.3 million parameter model with an inference rate of 9.5 ms per 16-frame video clip is highly accurate and efficient, thus suitable in inference in real-time and at edges. Ablation studies further demonstrate that the proposed components contribute consistent performance improvements across the evaluated benchmarks.