Scholarlink

<p>The aim of vision-and-language navigation (VLN) is to develop agents that navigate mapless environments via linguistic and visual observations. Continuous VLN, which more accurately mirrors real-world conditions than its discrete counterpart does, faces unique challenges such as real-time execution, complex instruction understanding, and long sequence prediction. In this work, we introduce a multilevel instruction understanding mechanism and propose a multilevel attention network (MLANet) to address these challenges. Initially, we develop a nonlearning-based fast sub-instruction algorithm (FSA) to swiftly generate sub-instructions without the need for annotations, achieving a speed enhancement of 28 times over the previous methods. Subsequently, our multilevel attention (MLA) module dynamically integrates visual features with both high- and low-level linguistic semantics, forming multilevel global semantics to bolster the complex instruction understanding capabilities of the model. Finally, we introduce the peak attention loss (PAL), which enables the flexible and adaptive selection of the current sub-instruction, thereby improving accuracy and stability achieved for long trajectories by focusing on the relevant local semantics. Our experimental findings demonstrate that MLANet significantly outperforms the baselines and is applicable to real-world robots.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A multilevel attention network with sub-instructions for continuous vision-and-language navigation

  • 期刊论文

  • 发表日期：

    2025年4月15日

查看全文
  • Zongtao He,
  • Liuyi Wang,
  • Shu Li,
  • Qingqing Yan,
  • Chengju Liu,
  • Qijun Chen

摘要

The aim of vision-and-language navigation (VLN) is to develop agents that navigate mapless environments via linguistic and visual observations. Continuous VLN, which more accurately mirrors real-world conditions than its discrete counterpart does, faces unique challenges such as real-time execution, complex instruction understanding, and long sequence prediction. In this work, we introduce a multilevel instruction understanding mechanism and propose a multilevel attention network (MLANet) to address these challenges. Initially, we develop a nonlearning-based fast sub-instruction algorithm (FSA) to swiftly generate sub-instructions without the need for annotations, achieving a speed enhancement of 28 times over the previous methods. Subsequently, our multilevel attention (MLA) module dynamically integrates visual features with both high- and low-level linguistic semantics, forming multilevel global semantics to bolster the complex instruction understanding capabilities of the model. Finally, we introduce the peak attention loss (PAL), which enables the flexible and adaptive selection of the current sub-instruction, thereby improving accuracy and stability achieved for long trajectories by focusing on the relevant local semantics. Our experimental findings demonstrate that MLANet significantly outperforms the baselines and is applicable to real-world robots.