Scholarlink

<p>To solve the problem of “human-machine gap” caused by insufficient understanding of intentions in traditional Virtual Reality (VR) interaction, this study proposes a multimodal perception fusion interaction optimization algorithm called Sparse Attention Transformer Proximal Policy Optimization (SAT-PPO). This method constructs a “perception-decision” closed-loop framework: The perception layer adopts a lightweight Transformer model with sparse attention mechanism, and uses large-scale Ego4D datasets for pre-training to deeply integrate multi-modal data streams such as user’s sight, posture and voice. The decision-making layer trains an agent that can dynamically adjust the interaction strategy through the Proximal Policy Optimization (PPO) Deep Reinforcement Learning (DRL) algorithm. Experimental results in complex virtual assembly tasks show that, compared with many baseline models, SAT-PPO algorithm optimizes the Task Completion Time (TCT) to 110.2&#xa0;s, reduces the Error Rate (ER) to 4.4%, and significantly reduces the user’s NASA Task Load Index (NASA-TLX) index to 28.1. The study proves that the algorithm successfully realizes efficient, smooth and intuitive adaptive VR interaction through accurate user intention prediction, which provides an effective technical solution for building the next generation immersive experience.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Optimization algorithm for immersive VR spatial interaction based on multimodal perceptual fusion

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月3日

查看全文
  • Hua He

摘要

To solve the problem of “human-machine gap” caused by insufficient understanding of intentions in traditional Virtual Reality (VR) interaction, this study proposes a multimodal perception fusion interaction optimization algorithm called Sparse Attention Transformer Proximal Policy Optimization (SAT-PPO). This method constructs a “perception-decision” closed-loop framework: The perception layer adopts a lightweight Transformer model with sparse attention mechanism, and uses large-scale Ego4D datasets for pre-training to deeply integrate multi-modal data streams such as user’s sight, posture and voice. The decision-making layer trains an agent that can dynamically adjust the interaction strategy through the Proximal Policy Optimization (PPO) Deep Reinforcement Learning (DRL) algorithm. Experimental results in complex virtual assembly tasks show that, compared with many baseline models, SAT-PPO algorithm optimizes the Task Completion Time (TCT) to 110.2 s, reduces the Error Rate (ER) to 4.4%, and significantly reduces the user’s NASA Task Load Index (NASA-TLX) index to 28.1. The study proves that the algorithm successfully realizes efficient, smooth and intuitive adaptive VR interaction through accurate user intention prediction, which provides an effective technical solution for building the next generation immersive experience.