The scene graph generation task aims to parse individual entities and their intrinsic relationships from images, subsequently constructing structured graphical representations to accurately capture the semantic information embedded within visual scenes. Currently, most models exhibit weak perception capabilities for small-scale entities within scenes, adversely affecting their performance in scene graph generation. To address this issue, we introduce a dual-channel segmentation strategy and propose a locally salient feature-based scene graph generation model. Specifically, in addition to a global self-attention feature channel, a local feature enhancement channel utilizing small-window convolutions is incorporated to accentuate local features. Furthermore, an adaptive channel allocation strategy is employed, gradually reducing the proportion of features assigned to small-window convolutions, thus enabling the network to effectively perceive entities of varying scales at different depths. This approach balances local and global feature modeling, thereby enhancing the model’s ability to capture fine-grained local details. The effectiveness of the proposed model is verified on the public datasets Visual Genome and Open Images V6. On the VG dataset, compared with the mainstream model RelTR, the proposed method achieves improvements of 0.2% and 0.4% in mR@50 and mR@100, respectively. Similarly, on the Open Images V6 dataset, the model attains enhancements of 0.2% in both R@50 and \(\textrm{score}_{wtd}\) .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Local Feature Saliency Enhancement for Scene Graph Generation

  • Yuanlong Wang,
  • Zhiwei Wu,
  • Hu Zhang

摘要

The scene graph generation task aims to parse individual entities and their intrinsic relationships from images, subsequently constructing structured graphical representations to accurately capture the semantic information embedded within visual scenes. Currently, most models exhibit weak perception capabilities for small-scale entities within scenes, adversely affecting their performance in scene graph generation. To address this issue, we introduce a dual-channel segmentation strategy and propose a locally salient feature-based scene graph generation model. Specifically, in addition to a global self-attention feature channel, a local feature enhancement channel utilizing small-window convolutions is incorporated to accentuate local features. Furthermore, an adaptive channel allocation strategy is employed, gradually reducing the proportion of features assigned to small-window convolutions, thus enabling the network to effectively perceive entities of varying scales at different depths. This approach balances local and global feature modeling, thereby enhancing the model’s ability to capture fine-grained local details. The effectiveness of the proposed model is verified on the public datasets Visual Genome and Open Images V6. On the VG dataset, compared with the mainstream model RelTR, the proposed method achieves improvements of 0.2% and 0.4% in mR@50 and mR@100, respectively. Similarly, on the Open Images V6 dataset, the model attains enhancements of 0.2% in both R@50 and \(\textrm{score}_{wtd}\) .