Scholarlink

From past few years, dialogue system mimics human interactions, capable of engaging autonomously, and perceiving as well as expressing emotions had grown increasingly significant. Traditional approaches for multimodal dialogue system had faced several challenges which include integration complexity, context understanding, and modality specific errors. Therefore, this research proposes MTF for multimodal dialogue systems for healthcare applications. Initially, data is collected from DementiaBank Pitt and Alzheimer’s dementia recognition through spontaneous speech (ADReSS) training datasets. This data is pre-processed by using wavelet thresholding (WT) to eliminate noise, and similarly, the text data is pre-processed with the help of stop words, punctuations removal, and lemmatization to increase the quality of healthcare. After that, the features are extracted by using distilled version of bidirectional encoder representations from transformer (DistilBERT) and Mel-frequency cepstral coefficient (MFFC), and finally, proposed MTF-PLATO is employed for generating the dialogues in healthcare applications. The proposed MFT achieved better results in terms of accuracy of 99.23, 99.02% on DementiaBank Pitt, and ADReSS dataset when compared with existing recurrent neural network (RNN).

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Multimodal Dialogue Systems Multimodal Transformer Fusion for Using Audio, and Text Data

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • B. N. Aryalekshmi,
  • Sugandha Saxena,
  • U. Pavan Kumar,
  • G. Santhosh Kumar,
  • G. Hemanth Kumar

摘要

From past few years, dialogue system mimics human interactions, capable of engaging autonomously, and perceiving as well as expressing emotions had grown increasingly significant. Traditional approaches for multimodal dialogue system had faced several challenges which include integration complexity, context understanding, and modality specific errors. Therefore, this research proposes MTF for multimodal dialogue systems for healthcare applications. Initially, data is collected from DementiaBank Pitt and Alzheimer’s dementia recognition through spontaneous speech (ADReSS) training datasets. This data is pre-processed by using wavelet thresholding (WT) to eliminate noise, and similarly, the text data is pre-processed with the help of stop words, punctuations removal, and lemmatization to increase the quality of healthcare. After that, the features are extracted by using distilled version of bidirectional encoder representations from transformer (DistilBERT) and Mel-frequency cepstral coefficient (MFFC), and finally, proposed MTF-PLATO is employed for generating the dialogues in healthcare applications. The proposed MFT achieved better results in terms of accuracy of 99.23, 99.02% on DementiaBank Pitt, and ADReSS dataset when compared with existing recurrent neural network (RNN).