Scholarlink

<p>Emotion recognition is an important issue in healthcare, education, and human–computer interaction because correct recognition of different emotional conditions could dramatically improve the efficiency of applications. Nevertheless, the current methods have been limited by the lack of full utilization of modality-specific mechanisms, poor inter-modality cohesion, and lack of adaptability in different tasks. The traditional fusion models tend to be based on naive concatenation, that is, they do not reflect contextual dependencies between audio, video, and electroencephalography (EEG) signals, as a result of which they generally do not generalize optimally and are not especially efficient. Multimodal Meta-Learning-Augmented EmoTriSense System is proposed as a holistic system of multimodal emotion recognition. It takes audio, video and EEG modalities and processes them separately and then combines them using a Cross-Modality Relational Attention (CMRA) fusion process. Whereas direct concatenation does not train the relational mappings, CMRA can be trained to learn such mappings allowing audio cues to strengthen the visual signal and the EEG data to contextualize the auditory and visual signals. The modality pipelines include audio decoding done using transformers, optical flow stabilization using Retina Net face detection and a 3D Convolutional Neural Network (3D-CNN) with attention, as well as Multiscale Temporal Dynamic Convolution (MSTDC) with multiscale feature selection. Such sophisticated representations are further improved through meta-learning that includes an episodic memory module, which makes them adapt strongly to tasks. Experimental findings highlight the effectiveness of EmoTriSense, with 95.17, 96.76, 95.48, and 93.95 accuracy on the DEAP dataset (valence, arousal, dominance, liking), using only 66&#xa0;s of running time, 48.6 memory usage of the CPU, and 0.5&#xa0;M parameters. On the EAV dataset, the performance rises to 98.8, 99.2, 99.0 and 97.5, thus demonstrating the effectiveness of its modality-specific pipelines, CMRA fusion, and meta-learning platform of scalable multimodal emotion recognition.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A multimodal meta-learning-augmented EmoTriSense for emotion recognition

  • 期刊论文

  • 发表日期：

    2026年3月7日

查看全文
  • V. Sowmya,
  • A. Rajeswari

摘要

Emotion recognition is an important issue in healthcare, education, and human–computer interaction because correct recognition of different emotional conditions could dramatically improve the efficiency of applications. Nevertheless, the current methods have been limited by the lack of full utilization of modality-specific mechanisms, poor inter-modality cohesion, and lack of adaptability in different tasks. The traditional fusion models tend to be based on naive concatenation, that is, they do not reflect contextual dependencies between audio, video, and electroencephalography (EEG) signals, as a result of which they generally do not generalize optimally and are not especially efficient. Multimodal Meta-Learning-Augmented EmoTriSense System is proposed as a holistic system of multimodal emotion recognition. It takes audio, video and EEG modalities and processes them separately and then combines them using a Cross-Modality Relational Attention (CMRA) fusion process. Whereas direct concatenation does not train the relational mappings, CMRA can be trained to learn such mappings allowing audio cues to strengthen the visual signal and the EEG data to contextualize the auditory and visual signals. The modality pipelines include audio decoding done using transformers, optical flow stabilization using Retina Net face detection and a 3D Convolutional Neural Network (3D-CNN) with attention, as well as Multiscale Temporal Dynamic Convolution (MSTDC) with multiscale feature selection. Such sophisticated representations are further improved through meta-learning that includes an episodic memory module, which makes them adapt strongly to tasks. Experimental findings highlight the effectiveness of EmoTriSense, with 95.17, 96.76, 95.48, and 93.95 accuracy on the DEAP dataset (valence, arousal, dominance, liking), using only 66 s of running time, 48.6 memory usage of the CPU, and 0.5 M parameters. On the EAV dataset, the performance rises to 98.8, 99.2, 99.0 and 97.5, thus demonstrating the effectiveness of its modality-specific pipelines, CMRA fusion, and meta-learning platform of scalable multimodal emotion recognition.