Scholarlink

<p>A scene consists of objects and relationships. Despite the success in large-scale text-to-image generation and text-guided image editing, most existing studies focus on single-object synthesis and editing, while controllable relationship or interaction synthesis and editing receive less attention. In this paper, we focus on a specific type of relationship, human–object interaction (HOI), and propose the HOIEdit method to achieve consistent HOI editing. Specifically, HOIEdit method learns the interaction token embeddings with less semantic ambiguity on several exemplar images, which would provide the information about the object locations and shapes. To edit the HOI and maintain the appearance of the human and object meanwhile, the object-wise mutual-attention control mechanism is proposed to query the correlated contents of the human and object from the source image. Furthermore, we employ a segmentation model to extract precise masks of the human and object from the first-round synthetic image, which are used to replace the coarse attention masks for a second-round refinement. Experiments show that HOIEdit achieves consistent interaction editing on both real images and synthetic images. Our implementation will be available at <a href="https://github.com/Kenneth-Wong/hoiedit">https://github.com/Kenneth-Wong/hoiedit</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

HOIEdit: Human–object interaction editing with text-to-image diffusion model

  • 期刊论文

  • 发表日期：

    2025年1月17日

查看全文
  • Tang Xu,
  • Wenbin Wang,
  • Alin Zhong

摘要

A scene consists of objects and relationships. Despite the success in large-scale text-to-image generation and text-guided image editing, most existing studies focus on single-object synthesis and editing, while controllable relationship or interaction synthesis and editing receive less attention. In this paper, we focus on a specific type of relationship, human–object interaction (HOI), and propose the HOIEdit method to achieve consistent HOI editing. Specifically, HOIEdit method learns the interaction token embeddings with less semantic ambiguity on several exemplar images, which would provide the information about the object locations and shapes. To edit the HOI and maintain the appearance of the human and object meanwhile, the object-wise mutual-attention control mechanism is proposed to query the correlated contents of the human and object from the source image. Furthermore, we employ a segmentation model to extract precise masks of the human and object from the first-round synthetic image, which are used to replace the coarse attention masks for a second-round refinement. Experiments show that HOIEdit achieves consistent interaction editing on both real images and synthetic images. Our implementation will be available at https://github.com/Kenneth-Wong/hoiedit.