Facial expressions are some of the most important modes of universal non-verbal human communication. This is because our faces are maps of our emotions, a frown communicates sadness while a smile expresses joy, among so many others. By focusing on micro-expressions, this work aims at identifying emotions that individuals are trying to conceal. The brain’s Basal Ganglia and Amygdala are responsible for responding to stimuli and emotions by sending trigger signals to facial muscles like the mentalis and buccinator (craniofacial muscles) which in-turn pull on the skin and exert their effects. All this happens involuntarily and as much as humans try to conceal these emotions, voluntarily, they can be noticed by well-trained individuals and computer models. Developing a model that picks up these tiny concealable emotions and goes ahead to explain why it has produced a particular result will be important not only in law enforcement for deception detection and threat assessment but also in healthcare for therapy and in human-computer interaction for visual assistants and enhanced user experience. This work illuminates the development of a system that can be used to recognize and classify facial emotions using feature extraction approaches with graph convolutional network algorithms employing different architectures LeNet and ResNet in real-time with openCV libraries such as TensorFlow, Numpy, and Keras. In this work, TF-Explain and Grad-Cam are used to create informative visualizations of the model. By using graph convolutional networks with LeNet and ResNet architectures, high accuracy was attained in identifying micro-expressions from facial images and explainable predictions making the model trustworthy. The graph convolutional network produced different results for the LeNet and ResNet architectures with the LeNet performing much better. The LeNet model attained 0.9801, 0.9113, 0.8320, and 0.8611 for accuracy, F1 score, recall, and precision, respectively, while the ResNet model attained 0.6176, 0.5232, 0,6679, and 0.6210 for the respective metrics. Results for the other four models are shown under the model evaluation section.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Explainable Semantic Graph Convolutional Networks for Facial Micro-expression Detection

  • Lusiba Mark Collins,
  • Matsiko Ian Sezi,
  • Bizimaana Gad William,
  • Ahumuza Derrick,
  • Tibakanya Joseph,
  • Nakayiza Hellen,
  • Ggaliwango Marvin

摘要

Facial expressions are some of the most important modes of universal non-verbal human communication. This is because our faces are maps of our emotions, a frown communicates sadness while a smile expresses joy, among so many others. By focusing on micro-expressions, this work aims at identifying emotions that individuals are trying to conceal. The brain’s Basal Ganglia and Amygdala are responsible for responding to stimuli and emotions by sending trigger signals to facial muscles like the mentalis and buccinator (craniofacial muscles) which in-turn pull on the skin and exert their effects. All this happens involuntarily and as much as humans try to conceal these emotions, voluntarily, they can be noticed by well-trained individuals and computer models. Developing a model that picks up these tiny concealable emotions and goes ahead to explain why it has produced a particular result will be important not only in law enforcement for deception detection and threat assessment but also in healthcare for therapy and in human-computer interaction for visual assistants and enhanced user experience. This work illuminates the development of a system that can be used to recognize and classify facial emotions using feature extraction approaches with graph convolutional network algorithms employing different architectures LeNet and ResNet in real-time with openCV libraries such as TensorFlow, Numpy, and Keras. In this work, TF-Explain and Grad-Cam are used to create informative visualizations of the model. By using graph convolutional networks with LeNet and ResNet architectures, high accuracy was attained in identifying micro-expressions from facial images and explainable predictions making the model trustworthy. The graph convolutional network produced different results for the LeNet and ResNet architectures with the LeNet performing much better. The LeNet model attained 0.9801, 0.9113, 0.8320, and 0.8611 for accuracy, F1 score, recall, and precision, respectively, while the ResNet model attained 0.6176, 0.5232, 0,6679, and 0.6210 for the respective metrics. Results for the other four models are shown under the model evaluation section.