Scholarlink

<p>The naturalness and realism of the interaction between facial and torso movements is crucial in audio-driven head generation. To address this challenge, we propose DeformableTalker, a real-time pose-controllable talking head generation framework that combines a dynamic modality adjustment mechanism with a multi-head cross-attention structure. The core of DeformableTalker is the construction of canonical 3D Gaussian parameters for the head, with precise modeling of facial dynamics through the deformation offsets of Gaussian point attributes. In the feature processing stage, we introduce Edge-Aware Adaptive Interaction module, which integrates dynamic feature adjustment with cross-modal attention mechanisms. This approach adapts to enhance the response strength of key facial areas and improves the alignment of multimodal features. Experimental results show that DeformableTalker outperforms existing methods in terms of facial synchronization accuracy and rendering quality, fully validating its effectiveness and advantages in audio-driven facial animation modeling.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Deformabletalker: edge-aware adaptive interaction for audio-driven 3D face animation with 3D Gaussian splatting

  • 期刊论文

  • 发表日期：

    2026年2月6日

查看全文
  • Minghui Shao,
  • Haoran Lu,
  • Guodong Wang,
  • Junli Zhao

摘要

The naturalness and realism of the interaction between facial and torso movements is crucial in audio-driven head generation. To address this challenge, we propose DeformableTalker, a real-time pose-controllable talking head generation framework that combines a dynamic modality adjustment mechanism with a multi-head cross-attention structure. The core of DeformableTalker is the construction of canonical 3D Gaussian parameters for the head, with precise modeling of facial dynamics through the deformation offsets of Gaussian point attributes. In the feature processing stage, we introduce Edge-Aware Adaptive Interaction module, which integrates dynamic feature adjustment with cross-modal attention mechanisms. This approach adapts to enhance the response strength of key facial areas and improves the alignment of multimodal features. Experimental results show that DeformableTalker outperforms existing methods in terms of facial synchronization accuracy and rendering quality, fully validating its effectiveness and advantages in audio-driven facial animation modeling.