Video shadow detection is a fundamental and challenging task. However, the research on video shadow detection has stalled for a decade in the era of deep learning due to a lack of high-quality datasets to drive it. To this end, we introduce a new dataset ViSha and extend it to ViSha+ specifically designed for video shadow detection, with 160 pixel-level high-quality annotated videos. We further develop a novel video shadow detection method \(\text {I}^{2}\text {VSD}\) with two main designs by fully exploring the intra-video and inter-video cooperation. The first design is a memory-driven intra-and inter-video contrastive learning (VCL) strategy. VCL facilitates learning more powerful feature representations to better harmonize global and local feature distributions, across different video scenarios and within given video frames. The second design is a dual-gated co-attention (DGC) module to mine the temporal complementary information. DGC calculates the pixel-level dense correlation between the current frame and reference frame and then uses it to generate the hybrid features which have a temporal enhancement in the current frame. In addition, we quantitatively and qualitatively evaluate our method on the proposed dataset ViSha/ViSha+ and a recent public dataset VISAD. Experimental results show that our method performs favorably against the state-of-the-art methods for both ViSha/ViSha+ and VISAD datasets.
