Scholarlink

<p>Referring Image Segmentation (RIS) requires segmenting image regions based on natural language expressions, demanding fine-grained alignment between visual and textual modalities. However, existing approaches still suffer from multimodal misalignment and semantic degradation, particularly in complex scenes containing multiple visually similar objects, where uniquely described targets are prone to mislocalization or incomplete segmentation. Inspired by cognitive and memory processes in human visual–linguistic reasoning, we propose TFANet, a Three-stage Image–Text Feature Alignment Network that tackles these limitations in a principled and structured manner. Rather than performing cross-modal fusion in a single step, TFANet organizes interaction into a progressive three-stage alignment process, in which the Knowledge Plus Stage (KPS), Knowledge Fusion Stage (KFS), and Knowledge Intensification Stage (KIS) successively build, consolidate, and reinforce the aligned image–text representation. KPS employs a Multiscale Linear Cross-Attention Module (MLAM) that establishes the initial global alignment through efficient bidirectional multi-scale semantic exchange, mitigating early-stage misalignment. KFS introduces a Cross-modal Feature Scanning Module (CFSM) to capture long-range dependencies and enforce multimodal consistency, effectively reducing ambiguity in scenes with multiple similar objects. KIS incorporates a Word-level Feature-guided Semantic Deepening Module (WFDM), which repeatedly reinjects word-level linguistic cues throughout the multi-level mask aggregation process, counteracting semantic attenuation in deep layers and preserving fine-grained textual guidance for accurate mask prediction. Extensive experiments on RefCOCO, RefCOCO+, and G-Ref demonstrate that TFANet outperforms state-of-the-art methods, achieving mIoU improvements of 1.84%, 1.52%, and 2.29% on the respective validation subsets. More experiments results verify that the proposed three-stage hierarchical alignment framework effectively alleviates attention misallocation and semantic loss, enabling more precise segmentation in complex visual-linguistic scenarios.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

TFANet: three-stage image-text feature alignment network for robust referring image segmentation

  • 期刊论文

  • 发表日期：

    2026年1月16日

查看全文
  • Qianqi Lu,
  • Yuxiang Xie,
  • Jing Zhang,
  • Shiwei Zou,
  • Yan Chen,
  • Xidao Luan

摘要

Referring Image Segmentation (RIS) requires segmenting image regions based on natural language expressions, demanding fine-grained alignment between visual and textual modalities. However, existing approaches still suffer from multimodal misalignment and semantic degradation, particularly in complex scenes containing multiple visually similar objects, where uniquely described targets are prone to mislocalization or incomplete segmentation. Inspired by cognitive and memory processes in human visual–linguistic reasoning, we propose TFANet, a Three-stage Image–Text Feature Alignment Network that tackles these limitations in a principled and structured manner. Rather than performing cross-modal fusion in a single step, TFANet organizes interaction into a progressive three-stage alignment process, in which the Knowledge Plus Stage (KPS), Knowledge Fusion Stage (KFS), and Knowledge Intensification Stage (KIS) successively build, consolidate, and reinforce the aligned image–text representation. KPS employs a Multiscale Linear Cross-Attention Module (MLAM) that establishes the initial global alignment through efficient bidirectional multi-scale semantic exchange, mitigating early-stage misalignment. KFS introduces a Cross-modal Feature Scanning Module (CFSM) to capture long-range dependencies and enforce multimodal consistency, effectively reducing ambiguity in scenes with multiple similar objects. KIS incorporates a Word-level Feature-guided Semantic Deepening Module (WFDM), which repeatedly reinjects word-level linguistic cues throughout the multi-level mask aggregation process, counteracting semantic attenuation in deep layers and preserving fine-grained textual guidance for accurate mask prediction. Extensive experiments on RefCOCO, RefCOCO+, and G-Ref demonstrate that TFANet outperforms state-of-the-art methods, achieving mIoU improvements of 1.84%, 1.52%, and 2.29% on the respective validation subsets. More experiments results verify that the proposed three-stage hierarchical alignment framework effectively alleviates attention misallocation and semantic loss, enabling more precise segmentation in complex visual-linguistic scenarios.