Adapting a Modern Siamese Framework for 360 \(^\circ \) Video Object Tracking
摘要
360 \(^\circ \) videos provide an immersive experience when viewed through 360 \(^\circ \) headsets. The integration of visual object tracking is essential for adding and synchronizing text descriptions of interesting objects in the recorded video, which is critical for various applications such as interactive education, tourism, virtual tours, and generating 2D footage of specific objects. Most current resources, including algorithms and datasets, focus on regular, limited-field-of-view videos. Therefore, we leverage existing resources and propose a methodology to adapt a modern visual tracker from 2D to 360 \(^\circ \) videos. Our proposed methodology focuses on processing a limited portion of the 360 \(^\circ \) frame, which is generated using our dynamic view generator (DVG). The DVG adjusts the projection parameters based on the tracked object’s position and size within the previous projection, thereby generating new footage of the target. To get the result on 360 \(^\circ \) videos, we simply convert each frame of the generated footage back onto the 360 \(^\circ \) frames. We report promising qualitative results on real 360 \(^\circ \) footage and conclude with insightful remarks and future work.