<p>Video stabilization aims to mitigate or eliminate the shake presented within video frames. Existing online video stabilization technologies rely on information from future frames, which may introduce a lag during real-time video stabilization. To surmount this hurdle, an online video stabilization model called OVST is proposed, which leverages solely historical video frames and enhanced by an attention mechanism. To simplify the complexity of model training and enhance robustness, a two-stage training strategy is proposed to decouple the fitting of real poses and the stabilization of virtual poses, and a hybrid stabilization loss with interframe soft constraints is designed, which effectively regulates the changes in camera poses between adjacent frames through interframe displacement, angular distortion, and cropping rate, thereby suppressing the distortion effects caused by excessive pose smoothing while balancing stability and cropping rate. Experiments demonstrate the superiority of the proposed OVST method over existing state-of-the-art techniques, achieving a stability metric of 0.8878 and a distortion metric of 0.9870.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

OVST: online video stabilization with two-stage training transformer

  • Xing Wu,
  • Yimin Zhu,
  • Han Zhang,
  • Jun Song,
  • Junfeng Yao,
  • Dong Zhu,
  • Quan Qian,
  • Yike Guo

摘要

Video stabilization aims to mitigate or eliminate the shake presented within video frames. Existing online video stabilization technologies rely on information from future frames, which may introduce a lag during real-time video stabilization. To surmount this hurdle, an online video stabilization model called OVST is proposed, which leverages solely historical video frames and enhanced by an attention mechanism. To simplify the complexity of model training and enhance robustness, a two-stage training strategy is proposed to decouple the fitting of real poses and the stabilization of virtual poses, and a hybrid stabilization loss with interframe soft constraints is designed, which effectively regulates the changes in camera poses between adjacent frames through interframe displacement, angular distortion, and cropping rate, thereby suppressing the distortion effects caused by excessive pose smoothing while balancing stability and cropping rate. Experiments demonstrate the superiority of the proposed OVST method over existing state-of-the-art techniques, achieving a stability metric of 0.8878 and a distortion metric of 0.9870.