Diffusion models have emerged as state-of-the-art generative frameworks, capable of producing high-quality realistic images. However, achieving precise and localized editing within specific regions of an image while preserving global consistency remains a significant challenge. In this paper, we propose a mask-based region-specific editing framework that leverages the latent space of diffusion models. Our approach enables fine-grained semantic modifications within a designated region of interest (ROI) while ensuring the unmasked regions remain unaffected. To achieve this, we compute the Jacobian matrix \({\mathbf{J}}_{\mathcal{M},{\varvec{t}}}\) restricted to the masked region and employ Singular Value Decomposition (SVD) to identify the most influential directions in the latent space. These directions are further refined via nullspace projection to eliminate undesired changes outside the ROI. By efficiently manipulating the latent space along the identified directions, our method achieves localized edits without requiring external labels, retraining, or additional supervision. Extensive experiments demonstrate that our framework produces visually consistent and semantically meaningful edits across diverse attributes, masks, and scale levels. The proposed method provides a mathematically principled, and highly controllable solution for localized image editing, advancing the capabilities of diffusion models in practical applications.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Mask-Guided Region-Specific Editing in Diffusion Models

  • Muhammad Shaheryar,
  • Jinhyeong Park,
  • Soon Ki Jung

摘要

Diffusion models have emerged as state-of-the-art generative frameworks, capable of producing high-quality realistic images. However, achieving precise and localized editing within specific regions of an image while preserving global consistency remains a significant challenge. In this paper, we propose a mask-based region-specific editing framework that leverages the latent space of diffusion models. Our approach enables fine-grained semantic modifications within a designated region of interest (ROI) while ensuring the unmasked regions remain unaffected. To achieve this, we compute the Jacobian matrix \({\mathbf{J}}_{\mathcal{M},{\varvec{t}}}\) restricted to the masked region and employ Singular Value Decomposition (SVD) to identify the most influential directions in the latent space. These directions are further refined via nullspace projection to eliminate undesired changes outside the ROI. By efficiently manipulating the latent space along the identified directions, our method achieves localized edits without requiring external labels, retraining, or additional supervision. Extensive experiments demonstrate that our framework produces visually consistent and semantically meaningful edits across diverse attributes, masks, and scale levels. The proposed method provides a mathematically principled, and highly controllable solution for localized image editing, advancing the capabilities of diffusion models in practical applications.