Embodied AI agents are trained to navigate complex environments using high-dimensional sensory inputs such as RGB images and depth maps. While pixel-based (RGB) navigation policies [1] are effective, they can struggle in ambiguous or visually challenging scenarios. In this work, we introduce an ensemble approach that combines a standard RGB-based policy with a dedicated depth-based policy, leveraging complementary cues from both modalities without explicit architectural fusion. We further extend our study to Zero-Shot Object Goal Navigation (ZSOGN) using BLIP [6], an open-source Vision-Language Model (VLM), demonstrating that it achieves results comparable to paid online API based solutions while remaining cost-effective. Experiments on standard embodied navigation benchmarks show that our ensemble policy achieves a \(\simeq \) 2% improvement over strong RGB-only baselines in Habitat-sim, highlighting the value of modality ensembles and open-source vision-language models for robust and generalizable embodied navigation. Codes are available at git repo https://github.com/lfovia/TEPEN.git.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

TEPEN: Towards an Ensemble Model for Pixel-Based Embodied Navigation

  • Athira Krishnan R,
  • Sumohana S. Channappayya

摘要

Embodied AI agents are trained to navigate complex environments using high-dimensional sensory inputs such as RGB images and depth maps. While pixel-based (RGB) navigation policies [1] are effective, they can struggle in ambiguous or visually challenging scenarios. In this work, we introduce an ensemble approach that combines a standard RGB-based policy with a dedicated depth-based policy, leveraging complementary cues from both modalities without explicit architectural fusion. We further extend our study to Zero-Shot Object Goal Navigation (ZSOGN) using BLIP [6], an open-source Vision-Language Model (VLM), demonstrating that it achieves results comparable to paid online API based solutions while remaining cost-effective. Experiments on standard embodied navigation benchmarks show that our ensemble policy achieves a \(\simeq \) 2% improvement over strong RGB-only baselines in Habitat-sim, highlighting the value of modality ensembles and open-source vision-language models for robust and generalizable embodied navigation. Codes are available at git repo https://github.com/lfovia/TEPEN.git.