Disentangling Acoustic Ambiguities with Visual Cues for Audio-Visual Navigation
摘要
Audio-visual embodied navigation relies on sound and visual data from the environment to complete navigation tasks in a 3D environment. During its propagation, sound is influenced by various factors, such as the environment. However, it plays a crucial role in perceiving and recognizing objects. In this paper, we propose enhancing sound features with visual cues to disentangle acoustic ambiguities. By using audio with enhanced sound features for audio-visual navigation tasks, we can improve navigation efficiency. However, our experiments revealed that simply combining audio with extracted sound features and visual depth maps or RGB images does not significantly enhance navigation efficiency. Therefore, we propose using feature decoupling to process sound and vision separately. This approach improves the efficiency of audio-visual navigation.