Scholarlink

<p>Visual navigation task seeks the optimal mapping from visual observation to the agent actions with given target object. How to improve perception ability and generalization ability in the unseen environment is a challenging problem. This paper presents an end-to-end navigation framework that leverages multimodal scene graphs (MSG) to enhance the visual representation and policy for robust visual navigation. MSG integrate scene graph information, previous navigation actions, training trajectories, and target objects, enabling agents to navigate efficiently towards target objects in unseen environments. Our approach distills valuable spatial and semantic clues from MSG, improving navigation performance by <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(10.1\%\)</EquationSource> </InlineEquation> in SPL and <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(25.4\%\)</EquationSource> </InlineEquation> in success rate compared to baselines. By utilizing graph attention networks, we demonstrate the effectiveness of MSG in generalizing across scenes, resulting in faster and more accurate target localization.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Leverage visual attention and multimodal scene graph for visual navigation

  • 期刊论文

  • 发表日期：

    2026年4月24日

查看全文
  • Zengkai Wang,
  • Cong Li,
  • Kang Zhou

摘要

Visual navigation task seeks the optimal mapping from visual observation to the agent actions with given target object. How to improve perception ability and generalization ability in the unseen environment is a challenging problem. This paper presents an end-to-end navigation framework that leverages multimodal scene graphs (MSG) to enhance the visual representation and policy for robust visual navigation. MSG integrate scene graph information, previous navigation actions, training trajectories, and target objects, enabling agents to navigate efficiently towards target objects in unseen environments. Our approach distills valuable spatial and semantic clues from MSG, improving navigation performance by \(10.1\%\) in SPL and \(25.4\%\) in success rate compared to baselines. By utilizing graph attention networks, we demonstrate the effectiveness of MSG in generalizing across scenes, resulting in faster and more accurate target localization.