Scholarlink

<p>With high prevalence rates for mental health disorders and the emerging interest in digital behavioural data, computational approaches for depression detection have attracted increasing research attention in recent years. Works available in the literature primarily follow a single-modality approach, using either an audio, text, or visual signal, resulting in inadequate reflection of heterogeneous depressive features and the mitigation of high-dimensional behavioural signals. However, any such approaches typically struggle with the following issues, limiting their practical deployment: modality imbalance, robustness to heterogeneous data sources, and transparency. To overcome such limitations, we present a novel attention-guided multi-source multimodal depression detection framework, named MoodSenseAI, that constructs an ensemble deep learning model from multimodal data, including text, speech, and video. Our framework is based on a DeepMoodNet model with multiple modality-specific encoders (TextMoodEncoder, SpeechMoodEncoder, and FaceMoodEncoder) that learn rich semantic, acoustic, and behavioural representations from independent benchmark datasets for each modality. A modality-specific attention-based fusion module aggregates these embeddings at the representation level, thereby providing context-dependent importance weights that enable adaptive modelling of complementary depression-related patterns across heterogeneous datasets. The experimental results on standard benchmark datasets show that the proposed framework achieves significantly higher accuracy, precision, recall, and AUC-ROC than unimodal baselines and improper fusion strategies (macro accuracy = 94.3%). We also conduct ablation studies to confirm that each modality provides complementary information. MoodSenseAI has an architecture that is forever scalable and transferable, providing both model-level transparency via modality contribution analysis and transferable subject-level alignment for mental health monitoring, telehealth applications, and large-scale screening systems.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

MoodSenseAI an attention-guided multi-source multimodal deep learning system for interpretable depression detection from text speech and video

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月20日

查看全文
  • K. Varada Rajkumar,
  • Jayaprakash Sunkavalli,
  • Subramanyam M. Vadlamani,
  • Chandrasekhar Uddagiri,
  • P. Bharat Siva Varma,
  • Raju Anitha

摘要

With high prevalence rates for mental health disorders and the emerging interest in digital behavioural data, computational approaches for depression detection have attracted increasing research attention in recent years. Works available in the literature primarily follow a single-modality approach, using either an audio, text, or visual signal, resulting in inadequate reflection of heterogeneous depressive features and the mitigation of high-dimensional behavioural signals. However, any such approaches typically struggle with the following issues, limiting their practical deployment: modality imbalance, robustness to heterogeneous data sources, and transparency. To overcome such limitations, we present a novel attention-guided multi-source multimodal depression detection framework, named MoodSenseAI, that constructs an ensemble deep learning model from multimodal data, including text, speech, and video. Our framework is based on a DeepMoodNet model with multiple modality-specific encoders (TextMoodEncoder, SpeechMoodEncoder, and FaceMoodEncoder) that learn rich semantic, acoustic, and behavioural representations from independent benchmark datasets for each modality. A modality-specific attention-based fusion module aggregates these embeddings at the representation level, thereby providing context-dependent importance weights that enable adaptive modelling of complementary depression-related patterns across heterogeneous datasets. The experimental results on standard benchmark datasets show that the proposed framework achieves significantly higher accuracy, precision, recall, and AUC-ROC than unimodal baselines and improper fusion strategies (macro accuracy = 94.3%). We also conduct ablation studies to confirm that each modality provides complementary information. MoodSenseAI has an architecture that is forever scalable and transferable, providing both model-level transparency via modality contribution analysis and transferable subject-level alignment for mental health monitoring, telehealth applications, and large-scale screening systems.