A Low-Latency Video Analytics Framework for Emotion Recognition and Stress Inference Using Region-Aware Swin–FANE and Temporal Modelling
摘要
Accurate interpretation of facial behaviour provides a strong foundation for non-invasive mental-state monitoring; however, most facial emotion recognition (FER) systems operate on static frames and do not capture the temporal evolution of emotions associated with stress. A spatial–temporal framework is proposed that integrates region-aware FER with temporal modelling for robust stress inference from facial video sequences. A Swin Transformer backbone, enhanced with Facial Attention Network Embedding (FANE), emphasises physiologically informative facial regions such as the eyes, eyebrows, and mouth, enabling fine-grained recognition of seven core emotions. The resulting frame-level embeddings are organised into temporal sequences and processed using a hybrid BiLSTM–ResNet module with residual refinement to model bidirectional emotional dynamics and generate a continuous stress-related signal. An interpretable stress index is derived from emotion probability trajectories using a deterministic formulation based on negative affect dominance and emotional volatility, allowing stress estimation in the absence of explicit annotations. The derived stress signal is intended as an interpretable behavioural proxy rather than a clinically validated stress measurement, reflecting the absence of explicit stress annotations in the dataset. Experimental evaluation on the FANE dataset demonstrates strong classification performance (90.7% accuracy) under a subject-independent protocol, along with robustness to variations in illumination, occlusion, pose, and motion blur. Temporal analysis further shows that the derived stress trajectories align with sustained negative affect and emotional variability. We demonstrate a low-latency video analytics framework that utilises region-aware transformer encoding to estimate stress trajectories with 90.7% accuracy, providing a scalable solution for affect-aware intelligent information systems.