Scholarlink

<p>The performance of a video summarization system relies on the effectiveness of its model in extracting the complex inter-frame dependencies within a video. Existing studies employed Recurrent Neural Network (RNN) variants like Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU) or Fully Convolutional Network (FCN), but their but their efficacy diminishes with long sequences. To mitigate the aforesaid limitation of existing models for the generation of video summaries in an unsupervised manner, the current study presents a novel Attention aware Dilated Recurrent SUMmarization network (ADR-SUM) trained via reinforcement learning. The unified framework comprises a dilated recurrent unit along with self-attention mechanism. The temporally dilated residual connections extract short-range as well as long-range temporal dependencies among long frame sequences effectively. The attention mechanism extracts contextual information among frames. The contextually rich temporal features are used to generate importance probabilities. The study also proposes a novel saliency reward to emphasize selection of interesting and meaningful frames under reinforcement learning environment. The extensive set of experiments indicate that the proposed model has outperforms most of the state-of-the-art existing approaches, both qualitatively and quantitatively, when evaluated on two benchmark datasets: SumMe and TvSum. Under the unsupervised learning environment, the obtained accuracies are 51.2% and 61.1%, while the supervised training of the proposed framework resulted in accuracies of 52.4% and 62.2% for the SumMe and TvSum datasets, respectively. This validates the potential of the proposed framework in the effective processing of long videos.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Reinforcement Learning Based Video Summarization Using Attention Aware Dilated RNN

  • 期刊论文

  • 发表日期：

    2025年10月12日

查看全文
  • Deeksha Gupta,
  • Manisha Kaushal,
  • Akashdeep Sharma

摘要

The performance of a video summarization system relies on the effectiveness of its model in extracting the complex inter-frame dependencies within a video. Existing studies employed Recurrent Neural Network (RNN) variants like Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU) or Fully Convolutional Network (FCN), but their but their efficacy diminishes with long sequences. To mitigate the aforesaid limitation of existing models for the generation of video summaries in an unsupervised manner, the current study presents a novel Attention aware Dilated Recurrent SUMmarization network (ADR-SUM) trained via reinforcement learning. The unified framework comprises a dilated recurrent unit along with self-attention mechanism. The temporally dilated residual connections extract short-range as well as long-range temporal dependencies among long frame sequences effectively. The attention mechanism extracts contextual information among frames. The contextually rich temporal features are used to generate importance probabilities. The study also proposes a novel saliency reward to emphasize selection of interesting and meaningful frames under reinforcement learning environment. The extensive set of experiments indicate that the proposed model has outperforms most of the state-of-the-art existing approaches, both qualitatively and quantitatively, when evaluated on two benchmark datasets: SumMe and TvSum. Under the unsupervised learning environment, the obtained accuracies are 51.2% and 61.1%, while the supervised training of the proposed framework resulted in accuracies of 52.4% and 62.2% for the SumMe and TvSum datasets, respectively. This validates the potential of the proposed framework in the effective processing of long videos.