Virtual Reality (VR) has gained increasing attention in recent years due to its strong immersion and realistic simulation. VR devices often include a Head Mounted Display (HMD) and VR controllers, which enhance VR immersion and interaction. However, the combination of HMD and controllers can only track the position of the user’s head and hands. For the rest of the body, there is no widely used method, as full-body motion capture devices are cumbersome and expensive. In this paper, we propose combining Mediapipe with VR devices to enhance VR immersion. We use VR devices to adjust the avatar scale and Mediapipe’s output. Considering that 2D camera full-body motion capture results are jittery, we apply filtering to reduce the jitter. Also, Mediapipe’s outputs are pixel positions, so it is not suitable for controlling the avatar directly. To solve this problem, we calculate the direction vectors of the body parts, which are calculated from Mediapipe’s output, to control the avatar. We evaluated our method through both visual and quantitative comparisons with other methods. It shows that our method performs better than the pure Mediapipe method and the Kinect method. As a result, we believe that our method can be an effective solution for full-body motion capture for general users and enhances VR immersion.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing VR Immersion Through Avatar Scaling and Sensor Fusion with Mediapipe

  • Shidao Zhao,
  • Tomochika Ozaki

摘要

Virtual Reality (VR) has gained increasing attention in recent years due to its strong immersion and realistic simulation. VR devices often include a Head Mounted Display (HMD) and VR controllers, which enhance VR immersion and interaction. However, the combination of HMD and controllers can only track the position of the user’s head and hands. For the rest of the body, there is no widely used method, as full-body motion capture devices are cumbersome and expensive. In this paper, we propose combining Mediapipe with VR devices to enhance VR immersion. We use VR devices to adjust the avatar scale and Mediapipe’s output. Considering that 2D camera full-body motion capture results are jittery, we apply filtering to reduce the jitter. Also, Mediapipe’s outputs are pixel positions, so it is not suitable for controlling the avatar directly. To solve this problem, we calculate the direction vectors of the body parts, which are calculated from Mediapipe’s output, to control the avatar. We evaluated our method through both visual and quantitative comparisons with other methods. It shows that our method performs better than the pure Mediapipe method and the Kinect method. As a result, we believe that our method can be an effective solution for full-body motion capture for general users and enhances VR immersion.