Natural language is inherently holistic and free-form. When expressions involve multiple attributes, spatial relationships, or contextual references, ambiguity can easily arise. This challenge becomes even more pronounced in cluttered and structurally complex 3D scenes, where semantic ambiguity significantly increases the difficulty of accurate grounding. Existing monocular 3D visual grounding methods often struggle to precisely align linguistic intent with spatial referents under such conditions—particularly when dealing with free-form descriptions, intricate attribute combinations, and occlusions—which results in a substantial drop in grounding accuracy. To address these challenges, we propose a neural-symbolic reasoning framework that explicitly disambiguates holistic language for 3D visual grounding. Our approach comprises a Multi-Attribute Interaction (MAI) module, which aligns visual features with multiple linguistic attributes via cross-modal attention, and a Spatial Relation Reasoning (SRR) module, which infers spatial relationships among candidate objects. These two modules jointly resolve visual-linguistic ambiguity by first decomposing descriptions into attributes and relations, then integrating spatial cues across object candidates. By disentangling appearance and spatial cues from ambiguous natural language expressions, our framework improves both the accuracy and robustness of 3D visual grounding. Our method consistently outperforms state-of-the-art approaches on the standard 3D visual grounding benchmark MonoRefer, with additional validation on SA3DVG highlighting its robustness, especially in challenging cases where disambiguating natural language is essential for reliable grounding.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Disambiguating Holistic Language for 3D Visual Grounding via Neural-Symbolic Reasoning

  • Yunze Wu,
  • Yufan Zhu,
  • Jie Lin,
  • Fangfang Wu,
  • Mingtao Feng,
  • Weisheng Dong

摘要

Natural language is inherently holistic and free-form. When expressions involve multiple attributes, spatial relationships, or contextual references, ambiguity can easily arise. This challenge becomes even more pronounced in cluttered and structurally complex 3D scenes, where semantic ambiguity significantly increases the difficulty of accurate grounding. Existing monocular 3D visual grounding methods often struggle to precisely align linguistic intent with spatial referents under such conditions—particularly when dealing with free-form descriptions, intricate attribute combinations, and occlusions—which results in a substantial drop in grounding accuracy. To address these challenges, we propose a neural-symbolic reasoning framework that explicitly disambiguates holistic language for 3D visual grounding. Our approach comprises a Multi-Attribute Interaction (MAI) module, which aligns visual features with multiple linguistic attributes via cross-modal attention, and a Spatial Relation Reasoning (SRR) module, which infers spatial relationships among candidate objects. These two modules jointly resolve visual-linguistic ambiguity by first decomposing descriptions into attributes and relations, then integrating spatial cues across object candidates. By disentangling appearance and spatial cues from ambiguous natural language expressions, our framework improves both the accuracy and robustness of 3D visual grounding. Our method consistently outperforms state-of-the-art approaches on the standard 3D visual grounding benchmark MonoRefer, with additional validation on SA3DVG highlighting its robustness, especially in challenging cases where disambiguating natural language is essential for reliable grounding.