Scholarlink

<p>Scene graph generation (SGG), which involves jointly detecting entities and inferring their relationships from images, plays a critical role in high-level visual scene understanding and reasoning tasks. Most existing SGG methods primarily focus on learning dependencies within individual triplets and follow a unidirectional reasoning paradigm, thereby overlooking the reverse constraints from predicates to entities. Moreover, they generally fail to capture inter-relationship dependencies, resulting in isolated predictions that ignore the global contextual information formed by shared entities or semantic associations. To address these limitations, this paper proposes I<sup>2</sup>D-SGG, a novel framework that jointly models both <b>I</b>ntra- and <b>I</b>nter-relationship Dependencies to improve the accuracy and efficacy of SGG. First, we introduce a triple-decoder architecture with dedicated modules for decoding subject, object, and predicate, connected through a prior-enhanced sparse relation matrix. Second, decoupled conditional queries comprising position queries and content queries are strengthened via cross-layer fusion and bidirectional attention, facilitating deeper geometric and semantic interaction within each triplet. Third, a global correlation graph-based reasoning module is employed to model inter-relationships across triplets. This module utilizes Graph Convolutional Networks (GCNs) to enable cross-triplet message passing and dynamic feature aggregation, thereby supporting global context-aware relational reasoning beyond isolated triplet. Experiments on the VG-150 dataset demonstrate that I<sup>2</sup>D-SGG achieves a mean Recall@100 (mR@100) of 35.41%, outperforming the state-of-the-art one-stage method by 1.57%. Qualitative analyses further confirm its superior capability in fine-grained scene understanding. Ablation studies validate the effectiveness and generalizability of our proposed dual dependency modeling mechanism. I<sup>2</sup>D-SGG enhances the model's capacity to comprehend both intra- and inter-relationship, overcoming limitations of unidirectional propagation, entangled query design, and isolated triplet reasoning in conventional approaches, thereby offering a new perspective for fine-grained relational modeling in complex visual scenes.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

I2D-SGG: scene graph generation via joint modeling of intra- and inter-relationship dependencies

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年12月28日

查看全文
  • Juan Lei,
  • Jiangpeng Tian,
  • Xiong You,
  • Zhiwei He

摘要

Scene graph generation (SGG), which involves jointly detecting entities and inferring their relationships from images, plays a critical role in high-level visual scene understanding and reasoning tasks. Most existing SGG methods primarily focus on learning dependencies within individual triplets and follow a unidirectional reasoning paradigm, thereby overlooking the reverse constraints from predicates to entities. Moreover, they generally fail to capture inter-relationship dependencies, resulting in isolated predictions that ignore the global contextual information formed by shared entities or semantic associations. To address these limitations, this paper proposes I2D-SGG, a novel framework that jointly models both Intra- and Inter-relationship Dependencies to improve the accuracy and efficacy of SGG. First, we introduce a triple-decoder architecture with dedicated modules for decoding subject, object, and predicate, connected through a prior-enhanced sparse relation matrix. Second, decoupled conditional queries comprising position queries and content queries are strengthened via cross-layer fusion and bidirectional attention, facilitating deeper geometric and semantic interaction within each triplet. Third, a global correlation graph-based reasoning module is employed to model inter-relationships across triplets. This module utilizes Graph Convolutional Networks (GCNs) to enable cross-triplet message passing and dynamic feature aggregation, thereby supporting global context-aware relational reasoning beyond isolated triplet. Experiments on the VG-150 dataset demonstrate that I2D-SGG achieves a mean Recall@100 (mR@100) of 35.41%, outperforming the state-of-the-art one-stage method by 1.57%. Qualitative analyses further confirm its superior capability in fine-grained scene understanding. Ablation studies validate the effectiveness and generalizability of our proposed dual dependency modeling mechanism. I2D-SGG enhances the model's capacity to comprehend both intra- and inter-relationship, overcoming limitations of unidirectional propagation, entangled query design, and isolated triplet reasoning in conventional approaches, thereby offering a new perspective for fine-grained relational modeling in complex visual scenes.