Dual-Stream Gated Fusion for Audio-Visual Navigation
摘要
Audio-visual navigation (AVN) tasks require an embodied agent to locate sound sources in unfamiliar environments by integrating visual images and auditory cues. However, traditional multimodal fusion methods are limited in modeling deep collaborative relationships between vision and audio. To address this issue, this paper proposes a Dual-Stream Gated Fusion (DSGF) method. DSGF dynamically adjusts the fusion ratio between different modalities through a gating mechanism and introduces a residual structure to achieve multi-scale fusion and cross-modal feature enhancement. Experiments conducted on the Replica and Matterport3D datasets demonstrate that DSGF significantly outperforms existing methods in terms of navigation success rate and path efficiency, while also exhibiting greater robustness.