Student action detection in class with individual temporal attention and co-occurrence awareness
摘要
Traditional spatio-temporal action detection methods typically rely on predefined candidate boxes or fixed spatial locations for feature extraction and action detection. This type of paradigm has limitations in fully modeling the continuity of human actions over time and the dynamic changes in contextual information. It also has obvious limitations in handling the co-occurrence structure of complex actions. In this paper, we propose a student action detection framework, ITCO, which is based on Individual Temporal Attention (ITA) and Co-Occurrence Awareness (COA). ITCO consists of two core components. First, ITA performs localized temporal aggregation on grouped spatio-temporal features without relying on explicit identity tracking, thereby enhancing the temporal coherence of individual feature representations. Second, COA leverages a co-occurrence matrix to provide the detector with rich contextual priors derived from real-world co-occurrence patterns, improving discrimination in complex scenarios involving multiple concurrent actions. By integrating these two components with the backbone network, we construct an effective multi-person action detector. Experimental results show that ITCO achieves an average accuracy of 85.78% on our self-built dataset, outperforming mainstream action detection methods including SlowFast, MViT, AIA, EVAD, and STMixer. Overall, ITCO provides a scalable and efficient solution for high-precision action detection in complex environments, serving as a valuable reference for applications such as smart classroom monitoring, human-computer interaction, and video action understanding.