Scholarlink

<p>Video Salient Object Detection (VSOD) aims to identify the most visually conspicuous objects in videos and extract key information from complex visual scenes. Recent studies combine optical flow (OF) and depth for complementary feature extraction. However, suboptimal fusion strategies often treat these modalities merely as extensions of the RGB stream, failing to fully leverage their unique semantic contributions. To address this limitation, we propose a novel SAM-based Multimodal Energy Prompting Network (MEPNet), which utilizes implicit prompts derived from OF and depth within a pre-trained Segment Anything Model (SAM). This approach enhances VSOD by effectively integrating the complementary dynamic and structural information from these modalities. Particularly, we introduce a Spectrogram Energy Generator to extract Spectrogram Energy from OF and depth. These energy-driven prompts fine-tune SAM via the Modality Energy Adapter, effectively mitigating noise interference and improving segmentation accuracy. In addition, we propose a Circular High-frequency Filter to enhance RGB modality details using an adaptive circular mask. Extensive experiments on five VSOD benchmark datasets demonstrate that our MEPNet outperforms state-of-the-art approaches. Furthermore, our MEPNet generalizes effectively to the Video-Camouflaged Object Detection task and achieves competitive results. The module and predicted maps are publicly available at <a href="https://github.com/TOMMYWHY/MEPNet">https://github.com/TOMMYWHY/MEPNet</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Enhancing video salient object detection via SAM-based multimodal energy prompting

  • 期刊论文

  • 发表日期：

    2025年8月29日

查看全文
  • Tao Jiang,
  • Yi Wang,
  • Feng Hou,
  • Li-li Liu

摘要

Video Salient Object Detection (VSOD) aims to identify the most visually conspicuous objects in videos and extract key information from complex visual scenes. Recent studies combine optical flow (OF) and depth for complementary feature extraction. However, suboptimal fusion strategies often treat these modalities merely as extensions of the RGB stream, failing to fully leverage their unique semantic contributions. To address this limitation, we propose a novel SAM-based Multimodal Energy Prompting Network (MEPNet), which utilizes implicit prompts derived from OF and depth within a pre-trained Segment Anything Model (SAM). This approach enhances VSOD by effectively integrating the complementary dynamic and structural information from these modalities. Particularly, we introduce a Spectrogram Energy Generator to extract Spectrogram Energy from OF and depth. These energy-driven prompts fine-tune SAM via the Modality Energy Adapter, effectively mitigating noise interference and improving segmentation accuracy. In addition, we propose a Circular High-frequency Filter to enhance RGB modality details using an adaptive circular mask. Extensive experiments on five VSOD benchmark datasets demonstrate that our MEPNet outperforms state-of-the-art approaches. Furthermore, our MEPNet generalizes effectively to the Video-Camouflaged Object Detection task and achieves competitive results. The module and predicted maps are publicly available at https://github.com/TOMMYWHY/MEPNet.