Scholarlink

<p>Audio-visual deception detection is essential for understanding human intent by accurately identifying the truthfulness or deceptive behavior. However, current methods struggle with domain shift between datasets and cannot fully use deception-related information from multiple sources. Aiming at tackling domain shift and modality gap issues across source and target domains, we propose a multi-source multimodal progressive domain adaptation (MMPDA+) framework with gradient disentangling that transfers the audio-visual knowledge from multiple source deception domains to the target one. By gradually aligning the source and target domains at both feature and decision levels, our method effectively bridges domain shift across diverse multimodal deception datasets. Observing the gradient conflicts between unimodal and multimodal learning in both task prediction and domain adaptation, we incorporate a disentangled gradient learning strategy to further enhance the training stability and boost the adapting performance. Extensive experiments conducted on six audio-visual deception datasets demonstrate the effectiveness of the proposed approach, achieving 58.5% accuracy averaged on each target domain. Our code is available at <a href="https://github.com/RH-Lin/MMPDA">https://github.com/RH-Lin/MMPDA</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

From Uni- to Multi-modal: Progressive Multi-source Domain Adaptation with Gradient Disentangling for Audio-visual Deception Detection

  • 期刊论文

  • 发表日期：

    2026年4月8日

查看全文
  • Ronghao Lin,
  • Aolin Xiong,
  • Qiaolin He,
  • Sijie Mai,
  • Li Huang,
  • Haifeng Hu

摘要

Audio-visual deception detection is essential for understanding human intent by accurately identifying the truthfulness or deceptive behavior. However, current methods struggle with domain shift between datasets and cannot fully use deception-related information from multiple sources. Aiming at tackling domain shift and modality gap issues across source and target domains, we propose a multi-source multimodal progressive domain adaptation (MMPDA+) framework with gradient disentangling that transfers the audio-visual knowledge from multiple source deception domains to the target one. By gradually aligning the source and target domains at both feature and decision levels, our method effectively bridges domain shift across diverse multimodal deception datasets. Observing the gradient conflicts between unimodal and multimodal learning in both task prediction and domain adaptation, we incorporate a disentangled gradient learning strategy to further enhance the training stability and boost the adapting performance. Extensive experiments conducted on six audio-visual deception datasets demonstrate the effectiveness of the proposed approach, achieving 58.5% accuracy averaged on each target domain. Our code is available at https://github.com/RH-Lin/MMPDA.