<p>360-degree video is becoming increasingly popular among users. To stream the 360-degree video efficiently to users, many works mainly focus on the prediction of users’ viewports and tile-based adaptive streaming algorithms. As for viewport prediction, existing methods struggle to achieve high accuracy to cover both static and moving salient objects. In response, we propose a Multi-Feature Driven Viewport Prediction model, which extracts potential features from multi-modal data, including saliency map, user head trajectory and object trajectory, to more accurately predict the user’s real viewport. Then, we design a Content-Aware Tile Priority Classification algorithm to compensate for possible viewport prediction errors. While for tile-based adaptive streaming, existing algorithms are mainly based on Deep Reinforcement Learning (DRL), which have a large decision space. In this paper, the proposed tile priority classification module optimizes the decision space of reinforcement learning agents to avoid unnecessary calculations and improve the efficiency. The experimental results on public datasets demonstrate the effectiveness of the proposed method, with the Quality of Experience (QoE) improved by 2.04−9.13% comparing to the state-of-the-art methods.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

VATP360: viewport adaptive 360-degree video streaming based on tile priority

  • Li Yu,
  • Zhiyu Pang,
  • Chao Yao,
  • Yao Zhao

摘要

360-degree video is becoming increasingly popular among users. To stream the 360-degree video efficiently to users, many works mainly focus on the prediction of users’ viewports and tile-based adaptive streaming algorithms. As for viewport prediction, existing methods struggle to achieve high accuracy to cover both static and moving salient objects. In response, we propose a Multi-Feature Driven Viewport Prediction model, which extracts potential features from multi-modal data, including saliency map, user head trajectory and object trajectory, to more accurately predict the user’s real viewport. Then, we design a Content-Aware Tile Priority Classification algorithm to compensate for possible viewport prediction errors. While for tile-based adaptive streaming, existing algorithms are mainly based on Deep Reinforcement Learning (DRL), which have a large decision space. In this paper, the proposed tile priority classification module optimizes the decision space of reinforcement learning agents to avoid unnecessary calculations and improve the efficiency. The experimental results on public datasets demonstrate the effectiveness of the proposed method, with the Quality of Experience (QoE) improved by 2.04−9.13% comparing to the state-of-the-art methods.