Improving skeleton action recognition with channel-temporal attention and multi-stream feature aggregation
摘要
In recent years, recognizing human actions based on skeleton data has become increasingly challenging due to its complexity. Specifically, accurately classifying action details is vital for improving recognition accuracy. Current skeleton-based action recognition methods often have difficulty capturing human movement details, mainly relying on increasingly complex neural networks to refine joint information. However, in action sequences, neighboring frames often include comparable poses, and relying on temporal convolutions to extract nearby features limits the transfer of important information during the process. Often, distinctive attributes may exist over extended period spans, and neglecting these features can limit the model’s expressive power. Therefore, it is essential to effectively capture these key features during computation to create stronger representations. Specifically, a new skeleton-based action recognition method is presented, enhancing accuracy by incorporating angle features, applying a multi-head attention mechanism, channel enhancement, and multi-scale feature aggregation. By incorporating angle representations, the model’s ability to detect subtle motion differences is improved, enabling it to better capture the geometric features in skeletal data. By utilizing the multi-head attention mechanism, the complex relationships between different joints are more precisely modeled, increasing the model’s focus on key moments and features within action sequences, thereby strengthening the expression of spatiotemporal features. Additionally, the model learns the internal correlations between different joints within each channel, dynamically capturing channel dependencies. In the channel enhancement phase, by strengthening the channel-related information and suppressing trivial correlations, the model’s ability to capture discriminative features is improved. The features from each channel are aggregated with relevant topological structures to generate the final results. Additionally, the integration of angle flow fusion techniques further enhances the overall representation capability, enabling the model to capture intricate action details and allowing it to flexibly and efficiently focus on the core aspects of actions while capturing both long-term dependencies and important short-term details. The proposed method was evaluated on three skeletal datasets, achieving an accuracy of 92.03% on the NTU-RGB+D 60 dataset, 89.60% on the NTU-RGB+D 120 dataset, and 95.32% on the N-UCLA dataset. The experimental results validate the effectiveness of the proposed method in skeletal action recognition.