Uncertainty-guided time–frequency feature enhancement for emotion-aware speech-driven 3D facial animation
摘要
Speech-driven 3D facial animation generation has significant application value in areas such as virtual live broadcasting and virtual idols. The core challenge lies in accurately learning the associative mapping between speech signals and 3D facial animations. Existing methods typically rely on single-perspective modeling, which limits the emotional diversity of facial expression representation. To address this issue, we propose a two-stage emotion-aware speech-driven 3D facial animation framework based on uncertainty-guided time-frequency feature enhancement. In the first stage, an uncertainty-guided dual-layer facial autoencoder is designed to fully extract emotional encoding information from both global and local features of 3D facial motion. The feature description operator based on an uncertainty quantification mechanism is introduced to model the probability distribution of facial expression variations across multiple categories, significantly improving the reliability and diversity of emotional representation. In the second stage, speech signals are encoded into multi-scale features, including content, emotion, and style representations. The time-frequency feature enhancement module is then applied to extract fine-grained emotional features from speech, enriching the subtlety of emotional expression. Finally, multi-level spatiotemporal feature coupling and a dynamic compensation mechanism are employed to align emotional and facial expression features, enabling the generation of emotionally diverse and natural 3D facial animations. Extensive experiments on the public 3DMEAD dataset demonstrate that our model achieves superior performance over state-of-the-art methods in both the accuracy and emotional richness.