Emotion perception through facial expressions is critical for human-computer interaction. However, occlusions, such as face masks, sunglasses, or scarves, greatly impair the effectiveness of classic facial expression identification systems. To overcome this challenge, the study proposes a two-way transfer learning system for occluded face emotion recognition that integrates MobileNet and DenseNet architectures using a soft decision fusion approach. The pre-trained embeddings of the masked facial image are extracted from MobileNetV2 and DenseNet-121 models, and learned separately using Logistic Regression and Linear Discriminant Analysis. The posterior class probabilities of the two models are fused by averaging to achieve the final classification. This strategy enhances model capacity for feature learning while being computationally inexpensive. The fusion process enables the model to make balanced decision-making by integrating the strengths of both architectures, increasing robustness against occlusion. We perform extensive analysis on the MCK+ masked face emotion recognition dataset featuring seven basic emotion classes and varied levels of occlusion. The proposed Mobile-Dense framework outperforms other fusion models in terms of accuracy, precision, recall, F1-Score, and Mathews Correlation Coefficient (95.92%, 0.96, 0.95, 0.95, and 0.9497, respectively, using the Logistic Regression classifier), and improves reliability for recognizing emotions under occlusion. This system thus provides a potential option for real-world emotion recognition applications that are resilient to facial occlusions.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Two-Way Transfer Learning Using Mobile-Dense Network for Occluded Face Emotion Recognition: A Soft Decision Fusion Approach

  • Ashi Agarwal,
  • Seba Susan

摘要

Emotion perception through facial expressions is critical for human-computer interaction. However, occlusions, such as face masks, sunglasses, or scarves, greatly impair the effectiveness of classic facial expression identification systems. To overcome this challenge, the study proposes a two-way transfer learning system for occluded face emotion recognition that integrates MobileNet and DenseNet architectures using a soft decision fusion approach. The pre-trained embeddings of the masked facial image are extracted from MobileNetV2 and DenseNet-121 models, and learned separately using Logistic Regression and Linear Discriminant Analysis. The posterior class probabilities of the two models are fused by averaging to achieve the final classification. This strategy enhances model capacity for feature learning while being computationally inexpensive. The fusion process enables the model to make balanced decision-making by integrating the strengths of both architectures, increasing robustness against occlusion. We perform extensive analysis on the MCK+ masked face emotion recognition dataset featuring seven basic emotion classes and varied levels of occlusion. The proposed Mobile-Dense framework outperforms other fusion models in terms of accuracy, precision, recall, F1-Score, and Mathews Correlation Coefficient (95.92%, 0.96, 0.95, 0.95, and 0.9497, respectively, using the Logistic Regression classifier), and improves reliability for recognizing emotions under occlusion. This system thus provides a potential option for real-world emotion recognition applications that are resilient to facial occlusions.