Pose-Guided Prompting for Skeleton-to-Image Generation
摘要
The skeleton-to-image generation task is an essential branch of image-to-image translation, aiming to translate skeleton images of arbitrary poses into high-quality target images while preserving the original pose information. However, existing unpaired image translation methods often struggle to maintain the correct orientation and structural consistency of the target object due to the lack of explicit pose constraints. To this end, this paper proposes a skeleton-to-image generation method based on pose-guided prompting that uses pose information as a prompt to improve image translation performance. Specifically, we design a Contrastive Pose-Image Pre-training (CPIP) model, which establishes a cross-modal mapping between pose information and image representations through contrastive learning. Furthermore, we introduce a two-stage training strategy, leveraging the pre-trained CPIP model to construct a pose consistency loss that reinforces the alignment between the generated images and input skeletons. Finally, we construct three datasets and conduct extensive experiments to demonstrate the superiority of our model. Our method achieves a Fréchet Inception Distance (FID) of 11.798, a Structural Similarity Index (SSIM) of 0.8680, and a Peak Signal-to-Noise Ratio (PSNR) of 36.151 on the Excavator dataset. On the Dancer dataset, it attains an FID of 6.808, SSIM of 0.9764, and PSNR of 43.227. On the Anime dataset, it achieves an FID of 9.589, SSIM of 0.9518, and PSNR of 39.211.