The proliferation of real-time applications for talking head synthesis, including video dubbing and virtual hosting, necessitates the use of computationally efficient, lightweight models. A significant drawback of these models is their tendency to produce facial artifacts, most notably in the challenging teeth region, which often appears blurry or anatomically incorrect. Conventional post-processing methods like super-resolution can mitigate blurriness but are ill-equipped to rectify geometric distortions and typically operate on a per-frame basis, leading to temporal incoherence. Furthermore, the domain of facial restoration has largely overlooked the specific problem of teeth synthesis, resulting in solutions with poor generalization. In this work, we present a solution that integrates temporal consistency layers into a pretrained diffusion model framework. Our approach overcomes the limitations of prior methods, generating teeth that are visually clear, structurally accurate, and stable across video frames.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Temporally Consistent Teeth Restoration for Talking Heads

  • Tianze Zhou,
  • Zhiyong Wu

摘要

The proliferation of real-time applications for talking head synthesis, including video dubbing and virtual hosting, necessitates the use of computationally efficient, lightweight models. A significant drawback of these models is their tendency to produce facial artifacts, most notably in the challenging teeth region, which often appears blurry or anatomically incorrect. Conventional post-processing methods like super-resolution can mitigate blurriness but are ill-equipped to rectify geometric distortions and typically operate on a per-frame basis, leading to temporal incoherence. Furthermore, the domain of facial restoration has largely overlooked the specific problem of teeth synthesis, resulting in solutions with poor generalization. In this work, we present a solution that integrates temporal consistency layers into a pretrained diffusion model framework. Our approach overcomes the limitations of prior methods, generating teeth that are visually clear, structurally accurate, and stable across video frames.