Local and Global Features Interactive Fusion Network for Macro- and Micro-expression Spotting in Long Videos
摘要
Individual emotions are often manifested through facial expressions, where macro-expressions (MaEs) and micro-expressions (MEs) provide the corresponding visual cues for different emotion applications. The spotting of these intertwined expressions has attracted extraordinary interests, which is an indispensable procedure in expression applications. However, due to the presence of noise, irrelevant movements, and the confusion of MEs and MaEs, it is very challenging to learn discriminative intrinsic features by deep learning models. In this paper, we explore an efficient deep neural network to address the issue of MaEs and MEs spotting in long videos. Specifically, this study assigns optical flow features as the model input and proposes a deep model, named LGFINet, which concentrates on the fusion of local and global features to predict probability scores of frames during an expression interval. To further boost the learning capability of facial expression spotting, the LGFINet scheme integrates multi-head self-attention and multi-head cross-attention into the backbone of the spotting network. To validate the superiority of the LGFINet, spotting experiments are conducted on two public MEs datasets, CAS(ME)2 and SAMM-LV. The proposed spotting approach achieves the F1 scores of 0.3710 and 0.4129 on the SAMM-LV and CAS(ME)2 dataset respectively. Extensive experiments verify the robustness and superiority of the MEs spotting based on LGFINet to other models. The source code of the LGFINet is available on GitHub ( https://github.com/XionghuiYe/LGIF_Net ).