Multimodal Audio-Visual Approach to Vehicle Detection and Classification
摘要
Night time traffic monitoring is hindered by low light, motion blur, and occlusions, restricting the performance of vision-based approaches. This article introduces a multimodal CNN-RNN model that combines infrared video and stereo audio for accurate vehicle detection and classification under low-light environments. Leveraging complementary acoustic-visual cues, the approach enhances detection precision for cars, buses, trucks, and motorbikes, including off-screen and partially occluded instances. When tested on actual night-time data, the new approach makes considerable gains over baseline performance. Box-wise IoU is 0.415, a relative increase of 18.2%, and point-wise IoU by 26.5%. Peak AUC scores reached 98.67% (validation) and 96.86% (test), demonstrating high class-separability and generalization. These results show that multimodal learning is indeed powerful for accurate traffic monitoring in difficult urban settings.