RT-VIS: Real-Time Video Instance Segmentation with Light-Weight Decoupled Framework
摘要
Video Instance Segmentation (VIS) is a pivotal technology for various applications including autonomous driving and video editing. While existing approaches primarily focus on enhancing the accuracy on datasets, they often neglect the real-time performance of segmentation results. Current online video instance segmentation models can sequentially output instance segmentation results for each frame and associate instances across adjacent frames; however, their inference speed limits practical applications. Additionally, these models require substantial memory during training and inference, which hinders deployment. To address these issues, we introduce the RT-VIS model, which adopts a decoupled strategy for VIS, allowing the model to be divided into segmenter and tracker components, each trained independently. To enhance inference speed, we developed a new light-weight instance segmentation model and employed a novel tracker to facilitate inter-frame instance association. Our model balances inference speed and accuracy, achieving 22.6 FPS and 42.2 AP on the YouTube-VIS2019 dataset, with memory requirements during training and inference approximately half of the previous methods. The code is available at https://github.com/STOVAGtz/RT-VIS .