MSTA-Net: Selecting High-Diagnostic-Value Segments in CLE Videos by Inferring Operator Cognitive Intent
摘要
The immense volume of video data generated by Confocal Laser Endoscopy (CLE) has led to a crisis of data review overload and diagnostic fatigue. To address this challenge, this study introduces a fundamental paradigm shift: moving the analytical focus from assessing the quality of static, context-deficient single-frame images to interpreting the operator's dynamic cognitive intent within the video stream. We propose MSTA-Net (Multimodal Spatio-Temporal Awareness Network), a deep learning framework designed to directly infer operator cognitive intent from video data. The core design philosophy of MSTA-Net is “divide and conquer” implemented through three parallel feature extraction branches that separately process microscopic texture, macroscopic dynamics, and temporal stability. These branches are inspired by clinical logic and are fused using a novel cross-attention mechanism to precisely capture the clinician's diagnostic intent. Comprehensive evaluations on a clinical dataset demonstrate that MSTA-Net achieves a macro-average F1-score of 84.3%, outperforming current state-of-the-art general-purpose video classification models by over 7.3% points while simultaneously realizing a more than nine-fold improvement in computational efficiency. These results validate the superiority of the MSTA-Net framework and the effectiveness of the proposed “operator-intent-centric” paradigm. The high-performance solution presented herein has the potential to significantly alter current CLE clinical workflows, enhancing both diagnostic efficiency and the accuracy of early cancer detection.