Enhancing Facial Emotion Recognition Accuracy Through Face Landmark Detection
摘要
Emotion recognition plays a crucial role in human-computer interaction, healthcare, and security applications. Traditional Facial Emotion Recognition (FER) models rely on convolutional neural networks (CNNs) to process entire facial images, often missing subtle facial cues that indicate specific emotions. This study aims to demonstrate how using facial landmarks can enhance emotion recognition accuracy, computational efficiency and can offer enhanced privacy since they process only specific facial points rather than entire images. For facial landmark detection we used MediaPipe- a powerful framework built by Google for building machine learning pipelines, we selectively focused on the most emotionally expressive facial regions-eyes, nose, mouth, and iris-rather than utilizing all detected landmarks, for a better performance. Those landmarks were processed through an MLP to detect the emotion classes. The performance was evaluated on three public datasets CK+, FER+, and AffectNet which are the most commonly used in this type of research. And Unlike some prior research that often focuses on a subset of emotions and eliminates similar ones like ‘fear’, ‘contempt’ and ‘neutral’, our model was tested on all emotion classes present in those datasets to ensure a comprehensive analysis. Comparative assessments were conducted against existing image-based FER models that have reported high accuracies on the same datasets. Our findings show that models using facial landmarks performed better than those processing full facial images, much faster, and achieved higher accuracy across all three datasets, and produced more accurate predictions. The Highest accuracy achieved by an image-based FER model On the FER+ dataset is 88.90%, our model achieved 91.91% accuracy, similarly on the AffectNet dataset highest accuracy achieved on eight emotion classes is 60.68%, our model achieved 91.40%, and on the CK+ highest accuracy achieved is 98.65%, our model achieved 99.92%. These results underscore the efficacy of facial landmark-based approaches in capturing essential emotional cues, offering a more efficient and accurate alternative to image-based models.