<p>Text-to-Time 3D Human Motion Grounding (THMG) is an emerging and challenging task that aims to temporally localize semantically relevant motion segments from untrimmed 3D motion sequences using open-ended natural language queries. The key challenge lies in effectively injecting linguistic information for fine-grained temporal alignment. Current state-of-the-art (SOTA) methods employ dual-stream self-attention with cross-modal attention for fusion, achieving good alignment but suffering from quadratic complexity, leading to high memory usage and limited scalability on long sequences. To address this, we propose a task-specific Hybrid Mamba-Attention Encoder (MAE) that strategically applies Mamba exclusively to motion modeling while retaining lightweight unidirectional cross-attention for semantic fusion—an asymmetric design optimized for the unique characteristics of THMG where motion sequences are significantly longer than text queries. Each MA block uses Mamba for unimodal motion modeling, replacing dual-stream self-attention while preserving long-range dependency modeling at significantly reduced cost. For fusion, we adopt a single motion-to-text cross-attention direction, avoiding redundant textto-motion interaction and further improving efficiency. Extensive experiments show that our method achieves superior scalability: maintaining stable performance at T=2048 frames where baselines fail (OOM), with up to 87.3% FLOPs reduction, while achieving SOTA performance: IoU@0.5 = 35.85 on BABEL (+3.53) and 48.70 on HumanML3D (+1.98). Code will be released at https://anonymous.4open.science/r/Hybrid-MAE-8217/GitHub.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Hybrid Mamba-attention encoder for text-to-time 3D human motion grounding

  • Haoqiang Wang,
  • Lianyu Huang,
  • Duoli Xu,
  • Yuna Zhong,
  • Xingru Lu,
  • Biqun Xiang

摘要

Text-to-Time 3D Human Motion Grounding (THMG) is an emerging and challenging task that aims to temporally localize semantically relevant motion segments from untrimmed 3D motion sequences using open-ended natural language queries. The key challenge lies in effectively injecting linguistic information for fine-grained temporal alignment. Current state-of-the-art (SOTA) methods employ dual-stream self-attention with cross-modal attention for fusion, achieving good alignment but suffering from quadratic complexity, leading to high memory usage and limited scalability on long sequences. To address this, we propose a task-specific Hybrid Mamba-Attention Encoder (MAE) that strategically applies Mamba exclusively to motion modeling while retaining lightweight unidirectional cross-attention for semantic fusion—an asymmetric design optimized for the unique characteristics of THMG where motion sequences are significantly longer than text queries. Each MA block uses Mamba for unimodal motion modeling, replacing dual-stream self-attention while preserving long-range dependency modeling at significantly reduced cost. For fusion, we adopt a single motion-to-text cross-attention direction, avoiding redundant textto-motion interaction and further improving efficiency. Extensive experiments show that our method achieves superior scalability: maintaining stable performance at T=2048 frames where baselines fail (OOM), with up to 87.3% FLOPs reduction, while achieving SOTA performance: IoU@0.5 = 35.85 on BABEL (+3.53) and 48.70 on HumanML3D (+1.98). Code will be released at https://anonymous.4open.science/r/Hybrid-MAE-8217/GitHub.