In computer vision, accurately estimating 3D human pose and shape is vital for various applications. While most existing methods rely on single-view RGB images and utilize deep learning techniques, they often struggle with incomplete and occluded human bodies, leading to estimation errors. Additionally, leveraging publicly available multi-view camera datasets can significantly enhance accuracy. To tackle these challenges, we introduce an innovative approach that iteratively adjusts a regressed 3D human model derived from a single image, utilizing optimized 2D pose estimates from multiple views. We obtain the parameters of the SMPL model by using a single image as input. We integrate 2D keypoints extracted from multiple views and employ a SMPLify-based approach to iteratively refine the fused keypoints within the single-view context. The optimized parameters subsequently inform the CNN training, creating a self-supervised multi-view framework. Our method exploits an iterative image loop process for regression across all views, effectively combining the strengths of CNN-based and optimization techniques. Experiments conducted on benchmark datasets indicate that our method outperforms current techniques in both qualitative and quantitative assessments.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multi-view Self-supervised 3D Human Pose and Shape Estimation on SMPL

  • Ruiyang Jing,
  • Yanni Zhang,
  • Jiaxuan Liu,
  • Jingyi Wu,
  • Wei Ni,
  • Liang Song

摘要

In computer vision, accurately estimating 3D human pose and shape is vital for various applications. While most existing methods rely on single-view RGB images and utilize deep learning techniques, they often struggle with incomplete and occluded human bodies, leading to estimation errors. Additionally, leveraging publicly available multi-view camera datasets can significantly enhance accuracy. To tackle these challenges, we introduce an innovative approach that iteratively adjusts a regressed 3D human model derived from a single image, utilizing optimized 2D pose estimates from multiple views. We obtain the parameters of the SMPL model by using a single image as input. We integrate 2D keypoints extracted from multiple views and employ a SMPLify-based approach to iteratively refine the fused keypoints within the single-view context. The optimized parameters subsequently inform the CNN training, creating a self-supervised multi-view framework. Our method exploits an iterative image loop process for regression across all views, effectively combining the strengths of CNN-based and optimization techniques. Experiments conducted on benchmark datasets indicate that our method outperforms current techniques in both qualitative and quantitative assessments.