Scholarlink

<p>We propose Dimitra++, a novel framework for audio-driven talking head generation, streamlined to learn lip motion, facial expression, as well as head pose motion. Specifically, we propose a conditional Motion Diffusion Transformer (cMDT) to model facial motion sequences, employing a 3D representation. The cMDT is conditioned on two inputs: a reference facial image, which determines appearance, as well as an audio sequence, which drives the motion. Quantitative and qualitative experiments, as well as a user study on two widely employed datasets, <i>i.e.,</i> VoxCeleb2 and CelebV-HQ, suggest that Dimitra++ is able to outperform existing approaches in generating realistic talking heads imparting lip motion, facial expression, and head pose. Code and qualitative results are provided on our project page: <a href="https://tashvikdhamija.github.io/dimitra/">https://tashvikdhamija.github.io/dimitra/</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

AI killed the Video Star. Audio-Driven Diffusion Model for Expressive Talking Head Generation

  • 期刊论文

  • 发表日期：

    2026年4月21日

查看全文
  • Baptiste Chopin,
  • Tashvik Dhamija,
  • Pranav Balaji,
  • Yaohui Wang,
  • Antitza Dantcheva

摘要

We propose Dimitra++, a novel framework for audio-driven talking head generation, streamlined to learn lip motion, facial expression, as well as head pose motion. Specifically, we propose a conditional Motion Diffusion Transformer (cMDT) to model facial motion sequences, employing a 3D representation. The cMDT is conditioned on two inputs: a reference facial image, which determines appearance, as well as an audio sequence, which drives the motion. Quantitative and qualitative experiments, as well as a user study on two widely employed datasets, i.e., VoxCeleb2 and CelebV-HQ, suggest that Dimitra++ is able to outperform existing approaches in generating realistic talking heads imparting lip motion, facial expression, and head pose. Code and qualitative results are provided on our project page: https://tashvikdhamija.github.io/dimitra/.