Recent advances in Extended Reality (XR) technologies, such as increased computational power and sensor integration, have broadened XR applications in education, healthcare, and collaboration. In parallel, machine learning (ML) has become a powerful tool for modeling complex human behaviors, including attention, intention, and social interaction. Despite this, integration between ML models and XR systems remains limited. This review analyzes 18 selected studies from 2018–2025 and focuses on two key directions: (1) multi-scale modeling of human and human interaction and (2) abstraction from movement to intention to goal. These works apply models such as Convolutional Neural Networks (CNNs), Recurrent Neural Network (RNNs), Graph Convolutional Networks (GCNs), and Transformers to tasks like attention and motion prediction, group behavior analysis, and short-term goal inference. However, most systems remain single-scale and lack pipelines that connect behavioral models across scales. The challenges of integrations include sensor misalignment, latency, and the absence of standardized multimodal data sets.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Integrating Machine Learning into Extended Reality: A Critical Review of Computational Approaches to Human Behavior Modeling

  • Houze Yang,
  • Caroline Cao,
  • Alexandra Hosny,
  • Jeeheon Ryu,
  • Inki Kim

摘要

Recent advances in Extended Reality (XR) technologies, such as increased computational power and sensor integration, have broadened XR applications in education, healthcare, and collaboration. In parallel, machine learning (ML) has become a powerful tool for modeling complex human behaviors, including attention, intention, and social interaction. Despite this, integration between ML models and XR systems remains limited. This review analyzes 18 selected studies from 2018–2025 and focuses on two key directions: (1) multi-scale modeling of human and human interaction and (2) abstraction from movement to intention to goal. These works apply models such as Convolutional Neural Networks (CNNs), Recurrent Neural Network (RNNs), Graph Convolutional Networks (GCNs), and Transformers to tasks like attention and motion prediction, group behavior analysis, and short-term goal inference. However, most systems remain single-scale and lack pipelines that connect behavioral models across scales. The challenges of integrations include sensor misalignment, latency, and the absence of standardized multimodal data sets.