Action recognition and segmentation are critical tasks for the applications requiring detailed analysis on human behavioral characteristics. However, current research primarily concentrates on temporal action segmentation assuming sequential occurrences of sub-actions. In practice, multiple actions temporarily overlapped or even co-occur in parallel. Inspired by image segmentation methods, we propose a joint-temporal action segmentation method that performs multi-action recognition at each human body joint. To conduct quantitative and qualitative evaluations, we construct a new skeleton-based multi-action dataset from the existing N-UCLA dataset (The code for our data generation is available at https://github.com/kiftiyani/NUCLAOverlap.git ). We propose learning objectives that incorporate the class distribution of each point to address the continuous label problem. Additionally, we argue that the inter-dependency between joints is crucial. We conduct multi-action segmentation experiments comparing well-known objectives such as CE, MAE, and MSE. Evaluation results demonstrate that our proposed approach achieves outstanding performance on five backbones.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Joint-Temporal Action Segmentation via Multi-action Recognition

  • Usfita Kiftiyani,
  • Seungkyu Lee

摘要

Action recognition and segmentation are critical tasks for the applications requiring detailed analysis on human behavioral characteristics. However, current research primarily concentrates on temporal action segmentation assuming sequential occurrences of sub-actions. In practice, multiple actions temporarily overlapped or even co-occur in parallel. Inspired by image segmentation methods, we propose a joint-temporal action segmentation method that performs multi-action recognition at each human body joint. To conduct quantitative and qualitative evaluations, we construct a new skeleton-based multi-action dataset from the existing N-UCLA dataset (The code for our data generation is available at https://github.com/kiftiyani/NUCLAOverlap.git ). We propose learning objectives that incorporate the class distribution of each point to address the continuous label problem. Additionally, we argue that the inter-dependency between joints is crucial. We conduct multi-action segmentation experiments comparing well-known objectives such as CE, MAE, and MSE. Evaluation results demonstrate that our proposed approach achieves outstanding performance on five backbones.