Pilots’ compound emotional states (such as anxiety-fear state) critically influence flight safety, yet remain understudied due to limitations of discrete emotion classification. This study proposes a multimodal emotion quantification framework integrating fuzzy logic and deep learning to address continuous mixed-state modeling. We optimized flight simulation experiments to collect synchronized facial expressions, ECG, and skin conductance data, establishing a 3D PAD (Pleasure-Arousal-Dominance) emotion space spanning five basic emotions. The novel ELG-Transformer dual-channel architecture was developed: 1) The ELG module employs multi-scale convolution and feature shrinkage attention to capture subtle spatiotemporal facial micro-expressions; 2) The Transformer module utilizes multi-head self-attention to model global dependencies in physiological signals. A fuzzy emotion analysis layer with Gaussian membership functions is introduced to map cross-modal features into continuous PAD space, enabling intensity quantification of mixed emotions. Experiments show that the model achieves MAE = 0.11 ± 0.03 in PAD dimension prediction, which is about 35.3% higher than the traditional model, and can effectively identify typical compound states such as anger-fear. The research results provide a quantitative emotional analysis tool for aviation human factors intelligent monitoring and flight safety early warning, which can support the dynamic decision optimization of flight safety early warning.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Research on Pilots’ Fuzzy Emotion Recognition Method Based on ELG-Transformer

  • Yan Zhao,
  • Haibo Wang,
  • Haiqing Si,
  • Zhongchang Cai,
  • Ting Pan

摘要

Pilots’ compound emotional states (such as anxiety-fear state) critically influence flight safety, yet remain understudied due to limitations of discrete emotion classification. This study proposes a multimodal emotion quantification framework integrating fuzzy logic and deep learning to address continuous mixed-state modeling. We optimized flight simulation experiments to collect synchronized facial expressions, ECG, and skin conductance data, establishing a 3D PAD (Pleasure-Arousal-Dominance) emotion space spanning five basic emotions. The novel ELG-Transformer dual-channel architecture was developed: 1) The ELG module employs multi-scale convolution and feature shrinkage attention to capture subtle spatiotemporal facial micro-expressions; 2) The Transformer module utilizes multi-head self-attention to model global dependencies in physiological signals. A fuzzy emotion analysis layer with Gaussian membership functions is introduced to map cross-modal features into continuous PAD space, enabling intensity quantification of mixed emotions. Experiments show that the model achieves MAE = 0.11 ± 0.03 in PAD dimension prediction, which is about 35.3% higher than the traditional model, and can effectively identify typical compound states such as anger-fear. The research results provide a quantitative emotional analysis tool for aviation human factors intelligent monitoring and flight safety early warning, which can support the dynamic decision optimization of flight safety early warning.