Scholarlink

<p>Semantic Scene Completion (SSC) aims to recover complete 3D geometry and assign voxel-level semantic labels from RGB-D inputs. While SSC has been widely studied, especially in structured indoor environments, existing approaches often struggle to capture high-level semantic cues and to model spatial relationships among objects due to reliance on purely visual features. To address these challenges, we propose a cross-modal SSC framework tailored for indoor scene understanding. Transcending the inherent limitations of uni-modal visual perception, our framework establishes a deep cross-modal fusion paradigm that synergizes linguistic semantics with 3D geometry to resolve ambiguities. Our network integrates two key components: (1) a semantic fusion module with cross-modal textual knowledge distillation, which introduces global semantic priors from a pretrained vision-language model to enhance RGB-D feature representation; and (2) a spatially guided semantic prediction module that incorporates natural language descriptions of spatial layouts and uses axial attention to align 3D scene features with textual priors, improving spatial reasoning among nearby but semantically distinct objects. Furthermore, we introduce a Chamfer Distance–based geometric consistency loss, computed at both category and scene levels, to preserve fine-grained structural details and refine geometric predictions. We evaluate our method on two indoor benchmarks, NYUv2 and NYUCAD, where it consistently outperforms existing state-of-the-art methods on both geometry completion and semantic prediction tasks. Qualitative results show enhanced accuracy in cluttered or ambiguous regions, demonstrating the effectiveness of our cross-modal semantic reasoning framework. Code and pretrained models will be available at: <a href="https://github.com/Hibiki-Ula/TIASSC">https://github.com/Hibiki-Ula/TIASSC</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

TIA-SSC: Text-Image Cross-Modal Alignment for Semantic Scene Completion

  • 期刊论文

  • 发表日期：

    2026年6月1日

查看全文
  • Weichao Wu,
  • Yongyang Xu,
  • Zhong Xie

摘要

Semantic Scene Completion (SSC) aims to recover complete 3D geometry and assign voxel-level semantic labels from RGB-D inputs. While SSC has been widely studied, especially in structured indoor environments, existing approaches often struggle to capture high-level semantic cues and to model spatial relationships among objects due to reliance on purely visual features. To address these challenges, we propose a cross-modal SSC framework tailored for indoor scene understanding. Transcending the inherent limitations of uni-modal visual perception, our framework establishes a deep cross-modal fusion paradigm that synergizes linguistic semantics with 3D geometry to resolve ambiguities. Our network integrates two key components: (1) a semantic fusion module with cross-modal textual knowledge distillation, which introduces global semantic priors from a pretrained vision-language model to enhance RGB-D feature representation; and (2) a spatially guided semantic prediction module that incorporates natural language descriptions of spatial layouts and uses axial attention to align 3D scene features with textual priors, improving spatial reasoning among nearby but semantically distinct objects. Furthermore, we introduce a Chamfer Distance–based geometric consistency loss, computed at both category and scene levels, to preserve fine-grained structural details and refine geometric predictions. We evaluate our method on two indoor benchmarks, NYUv2 and NYUCAD, where it consistently outperforms existing state-of-the-art methods on both geometry completion and semantic prediction tasks. Qualitative results show enhanced accuracy in cluttered or ambiguous regions, demonstrating the effectiveness of our cross-modal semantic reasoning framework. Code and pretrained models will be available at: https://github.com/Hibiki-Ula/TIASSC.