Robotic-assisted surgery (RAS) has improved the precision and consistency of minimally invasive procedures, yet most systems remain teleoperated and lack adaptability. We present a clinically grounded framework that integrates digital twins, virtual reality (VR), and embodied AI to enable autonomous trocar insertion in Mid-Urethral Sling (MUS) surgery—a representative task involving blind anatomical navigation. Our approach combines reinforcement learning (RL), behavioral cloning (BC), and generative adversarial imitation learning (GAIL) to train robotic agents from expert demonstrations and synthetic interactions. A high-fidelity 3D digital twin environment supports safe policy training and sim-to-real transfer to a physical robot (UFactory Lite6). Evaluation across simulation and deployment shows a 23% improvement in procedural fidelity, 0.78 mm average deviation, and superior performance compared to manual and gesture-based control. Three operational prototypes, VR-controlled, hand-tracked, and fully autonomous, demonstrate the system’s flexibility and clinical relevance. This work underscores the potential of embodied AI and digital twins to enhance surgical autonomy, reduce variability, and scale safe, adaptive training systems for intelligent operating rooms.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Embodied Surgical Intelligence via Digital Twins: Autonomous Trocar Insertion

  • Duy Ho,
  • Ahmed Alanazi,
  • Saeed Alqarni,
  • Chi Lee,
  • Gary Sutkin,
  • Yugyung Lee

摘要

Robotic-assisted surgery (RAS) has improved the precision and consistency of minimally invasive procedures, yet most systems remain teleoperated and lack adaptability. We present a clinically grounded framework that integrates digital twins, virtual reality (VR), and embodied AI to enable autonomous trocar insertion in Mid-Urethral Sling (MUS) surgery—a representative task involving blind anatomical navigation. Our approach combines reinforcement learning (RL), behavioral cloning (BC), and generative adversarial imitation learning (GAIL) to train robotic agents from expert demonstrations and synthetic interactions. A high-fidelity 3D digital twin environment supports safe policy training and sim-to-real transfer to a physical robot (UFactory Lite6). Evaluation across simulation and deployment shows a 23% improvement in procedural fidelity, 0.78 mm average deviation, and superior performance compared to manual and gesture-based control. Three operational prototypes, VR-controlled, hand-tracked, and fully autonomous, demonstrate the system’s flexibility and clinical relevance. This work underscores the potential of embodied AI and digital twins to enhance surgical autonomy, reduce variability, and scale safe, adaptive training systems for intelligent operating rooms.