Scholarlink

<p>Effectively modeling discriminative spatio-temporal information is essential for segmenting activities in long action sequences. However, we observe that existing methods are limited in weak spatio-temporal modeling capability due to two forms of coupled modeling: (i) Cascaded interaction couples spatial and temporal modeling, which over-smooths motion modeling over the long sequence, and (ii) Joint-shared temporal modeling adopts shared weights to model each joint, ignoring the distinct motion patterns of different joints. In this paper, we present a <b>De</b>coupled <b>S</b>patio-<b>T</b>emporal Framework&#xa0;(<b>DeST</b>) to address the above issues. Firstly, we decouple the cascaded spatio-temporal interaction to avoid stacking multiple spatio-temporal blocks, while achieving sufficient spatio-temporal interaction. Specifically, DeST&#xa0;performs once unified spatial modeling and divides the spatial features into different groups of sub-features, which then adaptively interact with temporal features from different layers. Since the different sub-features contain distinct spatial semantics, the model could learn better interaction patterns at each layer. Meanwhile, inspired by the fact that different joints move at different speeds, we propose joint-decoupled temporal modeling, which employs independent trainable weights to capture distinctive temporal features of each joint. On four large-scale benchmarks of different scenes, DeST&#xa0;significantly outperforms current state-of-the-art methods with less computational complexity. Our code is available at: <a href="https://github.com/lyhisme/DeST">https://github.com/lyhisme/DeST</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

DeST: A Decoupled Spatio-Temporal Framework for Action Segmentation

  • 期刊论文

  • 发表日期：

    2026年4月7日

查看全文
  • Yunheng Li,
  • Zhong-Yu Li,
  • Shanghua Gao,
  • Qilong Wang,
  • Qibin Hou,
  • Ming-Ming Cheng

摘要

Effectively modeling discriminative spatio-temporal information is essential for segmenting activities in long action sequences. However, we observe that existing methods are limited in weak spatio-temporal modeling capability due to two forms of coupled modeling: (i) Cascaded interaction couples spatial and temporal modeling, which over-smooths motion modeling over the long sequence, and (ii) Joint-shared temporal modeling adopts shared weights to model each joint, ignoring the distinct motion patterns of different joints. In this paper, we present a Decoupled Spatio-Temporal Framework (DeST) to address the above issues. Firstly, we decouple the cascaded spatio-temporal interaction to avoid stacking multiple spatio-temporal blocks, while achieving sufficient spatio-temporal interaction. Specifically, DeST performs once unified spatial modeling and divides the spatial features into different groups of sub-features, which then adaptively interact with temporal features from different layers. Since the different sub-features contain distinct spatial semantics, the model could learn better interaction patterns at each layer. Meanwhile, inspired by the fact that different joints move at different speeds, we propose joint-decoupled temporal modeling, which employs independent trainable weights to capture distinctive temporal features of each joint. On four large-scale benchmarks of different scenes, DeST significantly outperforms current state-of-the-art methods with less computational complexity. Our code is available at: https://github.com/lyhisme/DeST.