Integrating Machine Learning into Extended Reality: A Critical Review of Computational Approaches to Human Behavior Modeling
摘要
Recent advances in Extended Reality (XR) technologies, such as increased computational power and sensor integration, have broadened XR applications in education, healthcare, and collaboration. In parallel, machine learning (ML) has become a powerful tool for modeling complex human behaviors, including attention, intention, and social interaction. Despite this, integration between ML models and XR systems remains limited. This review analyzes 18 selected studies from 2018–2025 and focuses on two key directions: (1) multi-scale modeling of human and human interaction and (2) abstraction from movement to intention to goal. These works apply models such as Convolutional Neural Networks (CNNs), Recurrent Neural Network (RNNs), Graph Convolutional Networks (GCNs), and Transformers to tasks like attention and motion prediction, group behavior analysis, and short-term goal inference. However, most systems remain single-scale and lack pipelines that connect behavioral models across scales. The challenges of integrations include sensor misalignment, latency, and the absence of standardized multimodal data sets.