<p>Reconstructing animatable avatars from images and videos plays an important role in virtual domains and immersive telepresence. Existing methods that combine parametric face models with neural radiance fields ignore the influence of subtle expressions on facial geometry. They have difficulty reconstructing photo-realistic avatars with accurate geometries and expressions. To tackle this problem, we propose EE-Head, which introduces a parametric face model with high-accurate facial expressions to neural radiance fields, enhancing the rendering quality of animatable avatars. Specifically, we first propose an emotion estimation algorithm, which utilizes an emotion consistency loss to encourage emotion similarity between input images and parametric face models. This algorithm can estimate a parametric face model with more accurate facial expressions, particularly in the lip region. Then, we propose a joint training method that optimizes a neural radiance field by adjusting the weights of the image and parametric models across different frames. Our optimization can adaptively adjust the impact of different frames on the neural radiance field, improving rendering quality. As a result, EE-Head can reconstruct photo-realistic and animatable avatars with accurate expressions from multi-view images and monocular dynamic videos. Extensive experiments on various subjects demonstrate that EE-Head outperforms SOTA methods in quantitative and qualitative rendering quality.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

EE-Head: emotion estimation for precise facial expression in NeRF head avatars

  • Enxu Zhao,
  • Jianchi Sun,
  • Fei Luo,
  • Chunxia Xiao

摘要

Reconstructing animatable avatars from images and videos plays an important role in virtual domains and immersive telepresence. Existing methods that combine parametric face models with neural radiance fields ignore the influence of subtle expressions on facial geometry. They have difficulty reconstructing photo-realistic avatars with accurate geometries and expressions. To tackle this problem, we propose EE-Head, which introduces a parametric face model with high-accurate facial expressions to neural radiance fields, enhancing the rendering quality of animatable avatars. Specifically, we first propose an emotion estimation algorithm, which utilizes an emotion consistency loss to encourage emotion similarity between input images and parametric face models. This algorithm can estimate a parametric face model with more accurate facial expressions, particularly in the lip region. Then, we propose a joint training method that optimizes a neural radiance field by adjusting the weights of the image and parametric models across different frames. Our optimization can adaptively adjust the impact of different frames on the neural radiance field, improving rendering quality. As a result, EE-Head can reconstruct photo-realistic and animatable avatars with accurate expressions from multi-view images and monocular dynamic videos. Extensive experiments on various subjects demonstrate that EE-Head outperforms SOTA methods in quantitative and qualitative rendering quality.