Sound-Indicated Active Object Detection
摘要
With the development of sensing technologies, intelligent robots are usually equipped with both microphones and cameras. Therefore, it is possible for the robot to perceive and understand the physical world from both the visual and audio information. Especially in practical scenarios, it often happens that visual and audio information appear in the video streaming simultaneously and some objects are making sound, e.g., instrument playing, people talking, etc. In this chapter, we propose a novel task of sound-indicated active object detection, which leverages the audio information to actively detect the sounding object in the video, and a two-stream weakly supervised deep learning architecture is designed to solve this task accordingly. Case studies are conducted on both the dataset and realistic scenarios demonstrating that the proposed method is able to actively detect the sounding object effectively.