This paper presents an innovative approach to improving situational awareness in autonomous vehicles by leveraging audio-visual fusion for enhanced object detection, particularly in urban environments prone to visual occlusions. Traditional autonomous vehicles situational awareness systems face limitations in complex, crowded settings where occlusions by other objects or environmental factors obstruct visual sensors. We propose a fusion of audio cues with visual data to overcome occlusions, utilizing a Labeled Random Finite Set model integrated with audio features such as engine sounds and horn signals. Our approach demonstrates that combining visual and auditory information through multi-object tracking enables autonomous vehicles to maintain accurate awareness of their surroundings, even during prolonged occlusions. Results from simulated urban traffic scenarios validate the effectiveness of this fusion method, showing a significant reduction in tracking errors and enhanced resilience against occlusions.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Audio-Visual Fusion for Intelligent Situational Awareness in Driverless Cars

  • Thushara R. Bandara,
  • Amirali K. Gostar,
  • Ehsan Asadi,
  • Reza Hoseinnezhad

摘要

This paper presents an innovative approach to improving situational awareness in autonomous vehicles by leveraging audio-visual fusion for enhanced object detection, particularly in urban environments prone to visual occlusions. Traditional autonomous vehicles situational awareness systems face limitations in complex, crowded settings where occlusions by other objects or environmental factors obstruct visual sensors. We propose a fusion of audio cues with visual data to overcome occlusions, utilizing a Labeled Random Finite Set model integrated with audio features such as engine sounds and horn signals. Our approach demonstrates that combining visual and auditory information through multi-object tracking enables autonomous vehicles to maintain accurate awareness of their surroundings, even during prolonged occlusions. Results from simulated urban traffic scenarios validate the effectiveness of this fusion method, showing a significant reduction in tracking errors and enhanced resilience against occlusions.