Explainable Semantic Graph Convolutional Networks for Facial Micro-expression Detection
摘要
Facial expressions are some of the most important modes of universal non-verbal human communication. This is because our faces are maps of our emotions, a frown communicates sadness while a smile expresses joy, among so many others. By focusing on micro-expressions, this work aims at identifying emotions that individuals are trying to conceal. The brain’s Basal Ganglia and Amygdala are responsible for responding to stimuli and emotions by sending trigger signals to facial muscles like the mentalis and buccinator (craniofacial muscles) which in-turn pull on the skin and exert their effects. All this happens involuntarily and as much as humans try to conceal these emotions, voluntarily, they can be noticed by well-trained individuals and computer models. Developing a model that picks up these tiny concealable emotions and goes ahead to explain why it has produced a particular result will be important not only in law enforcement for deception detection and threat assessment but also in healthcare for therapy and in human-computer interaction for visual assistants and enhanced user experience. This work illuminates the development of a system that can be used to recognize and classify facial emotions using feature extraction approaches with graph convolutional network algorithms employing different architectures LeNet and ResNet in real-time with openCV libraries such as TensorFlow, Numpy, and Keras. In this work, TF-Explain and Grad-Cam are used to create informative visualizations of the model. By using graph convolutional networks with LeNet and ResNet architectures, high accuracy was attained in identifying micro-expressions from facial images and explainable predictions making the model trustworthy. The graph convolutional network produced different results for the LeNet and ResNet architectures with the LeNet performing much better. The LeNet model attained 0.9801, 0.9113, 0.8320, and 0.8611 for accuracy, F1 score, recall, and precision, respectively, while the ResNet model attained 0.6176, 0.5232, 0,6679, and 0.6210 for the respective metrics. Results for the other four models are shown under the model evaluation section.