<p>Facial micro-expressions (MEs) are transient and spontaneous facial muscle motions which can reveal the genuine emotions that people strive to conceal. However, in facial expression recognition, the coupling of identity and emotion information can impact performance, especially in micro-expression recognition (MER). Moreover, due to the inherent properties of MEs, extracting robust and discriminative features to enhance recognition performance is a challenge. To address these issues, a novel Transformer-based architecture called the Dual Complementarity Transformer (DCT) is proposed. Specifically, to reduce the interference of identity information, a motion manipulator is designed to capture and amplify the emotional features. Then, the amplified emotional features are further enhanced through a complementary strategy which effectively captures global features by alternately computing spatial and channel self-attention. Subsequently, a convolutional representation enhancement module is designed, which aims to equip the DCT with the locality. Therefore, the proposed method achieves discriminative feature extraction by complementarily aggregating multiple types of features (spatial-channel, global–local), and significantly improves MER performance. Extensive experiments conducted under the Leave-One-Subject-Out protocol on widely adopted datasets, including SMIC, CASME II, SAMM, and MEGC 2019 composite dataset, demonstrate that the proposed DCT outperforms other related MER approaches.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Dual complementarity transformer for micro-expression recognition

  • Ying Zhou,
  • Lei Chen,
  • Tianhuan Huang,
  • Ju Liu,
  • Xianye Ben

摘要

Facial micro-expressions (MEs) are transient and spontaneous facial muscle motions which can reveal the genuine emotions that people strive to conceal. However, in facial expression recognition, the coupling of identity and emotion information can impact performance, especially in micro-expression recognition (MER). Moreover, due to the inherent properties of MEs, extracting robust and discriminative features to enhance recognition performance is a challenge. To address these issues, a novel Transformer-based architecture called the Dual Complementarity Transformer (DCT) is proposed. Specifically, to reduce the interference of identity information, a motion manipulator is designed to capture and amplify the emotional features. Then, the amplified emotional features are further enhanced through a complementary strategy which effectively captures global features by alternately computing spatial and channel self-attention. Subsequently, a convolutional representation enhancement module is designed, which aims to equip the DCT with the locality. Therefore, the proposed method achieves discriminative feature extraction by complementarily aggregating multiple types of features (spatial-channel, global–local), and significantly improves MER performance. Extensive experiments conducted under the Leave-One-Subject-Out protocol on widely adopted datasets, including SMIC, CASME II, SAMM, and MEGC 2019 composite dataset, demonstrate that the proposed DCT outperforms other related MER approaches.