Image animation aims to generate results that preserve the appearance of source image while transferring the motion from driving images. However, in unsupervised settings, significant differences between the source and driving objects often lead to incomplete motion extraction and suboptimal generation quality. To address these challenges, we propose a novel end-to-end unsupervised motion transfer framework. Specifically, we integrate semantic masks and gaussian-distributed keypoints from both source and driving images to enrich semantic and structural representations. We further introduce a cross-modal attention mechanism to effectively fuse global mask features with warped source features, and employ a multi-resolution generative network to better capture the motion structure of the driving image. Additionally, several auxiliary loss functions are designed to enhance training stability and generation quality. Extensive experiments on various benchmarks demonstrate that our approach achieves superior performance, especially in motion-related metrics, and generalizes well to diverse objects such as human faces, bodies, and animals.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Unsupervised Semantic Mask-Guided Image Animation with Fused Cross-Attention

  • Jixiang Zhu,
  • jianjun Li

摘要

Image animation aims to generate results that preserve the appearance of source image while transferring the motion from driving images. However, in unsupervised settings, significant differences between the source and driving objects often lead to incomplete motion extraction and suboptimal generation quality. To address these challenges, we propose a novel end-to-end unsupervised motion transfer framework. Specifically, we integrate semantic masks and gaussian-distributed keypoints from both source and driving images to enrich semantic and structural representations. We further introduce a cross-modal attention mechanism to effectively fuse global mask features with warped source features, and employ a multi-resolution generative network to better capture the motion structure of the driving image. Additionally, several auxiliary loss functions are designed to enhance training stability and generation quality. Extensive experiments on various benchmarks demonstrate that our approach achieves superior performance, especially in motion-related metrics, and generalizes well to diverse objects such as human faces, bodies, and animals.