A Two-Stage Multi-domain Collaborative Optimization Network for 3D Human Mesh Recovery
摘要
3D human mesh recovery aims to reconstruct the 3D human mesh by estimating the 3D human mesh parameters in the video. Recent approaches mainly focus on solving the problem of capturing the correlation between neighboring frames in the time domain to reconstruct an overall more accurate 3D human mesh sequence, however, these methods often ignore the local detail features as well as the smoothness of the video mesh sequence, resulting in the occurrence of severity jitter between different frames. To alleviate this problem, we propose a two-stage multi-domain collaborative optimization network for 3D human mesh recovery. Our Multi-Domain Collaborative Network (MDCN) enhances the accuracy of 3D human reconstruction by effectively integrating global and local information. Central to the MDCN is the Multi-Domain Adaptive Module (MDAM), which plays a pivotal role in capturing human motion characteristics. During the global feature stage, MDAM analyzes sequence cycle similarities across time and frequency domains, significantly improving the precision and fluidity of the reconstruction process. Specifically, MDAM identifies and leverages the periodicity inherent in human motion, thereby refining the reconstruction’s accuracy and smoothness. Furthermore, in the local feature stage, we design the Human Parameter Optimization Module (HPOM). This module fine-tunes local human parameters, allowing the reconstructed model to closely emulate the original input’s morphology. This local refinement not only improves the detailed expression of the model, but also enhances the overall visual effect. The collaborative operation of these two modules empowers MDCN to achieve a high-fidelity reconstruction of the human form while preserving the nuances of human motion. Extensive experimental results on multiple publicly available datasets show that our approach achieves competitive performance with the existing methods.