Scholarlink

We present an XR-based gesture interface for intuitive human-robot collaboration with an Autonomous Mobile Robot (AMR), targeting tasks that require spatial awareness and real-time control. The system integrates a Varjo XR-3 headset with Ultraleap hand tracking and a Unity-based Digital Twin, synchronized with the physical robot via ROS. Operators interact naturally through hand gestures, supported by low-latency communication and real-time feedback. To address challenges in localization accuracy and co-carrying coordination, we introduce a synthetic data generation pipeline within the virtual environment. This enables the simulation of diverse scenarios for training models that improve AMR localization and human-robot coordination. While the current implementation focuses on a single AMR, the architecture is designed for scalability, with future extensions exploring distributed teleoperation using edge computing. The proposed framework contributes to the development of scalable, data-driven, and immersive XR interfaces for advanced human-robot collaboration in Industry 5.0 environments.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Toward Scalable XR Interfaces for Human-Robot Collaboration with AMRs: Localization and Co-carrying via Synthetic Data

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Eva Almansa,
  • Xiao Lin,
  • Iñigo Mendizabal-Arrieta,
  • Sara García

摘要

We present an XR-based gesture interface for intuitive human-robot collaboration with an Autonomous Mobile Robot (AMR), targeting tasks that require spatial awareness and real-time control. The system integrates a Varjo XR-3 headset with Ultraleap hand tracking and a Unity-based Digital Twin, synchronized with the physical robot via ROS. Operators interact naturally through hand gestures, supported by low-latency communication and real-time feedback. To address challenges in localization accuracy and co-carrying coordination, we introduce a synthetic data generation pipeline within the virtual environment. This enables the simulation of diverse scenarios for training models that improve AMR localization and human-robot coordination. While the current implementation focuses on a single AMR, the architecture is designed for scalability, with future extensions exploring distributed teleoperation using edge computing. The proposed framework contributes to the development of scalable, data-driven, and immersive XR interfaces for advanced human-robot collaboration in Industry 5.0 environments.