A vision-guided approach to 3D spatial safety monitoring in human–robot systems
摘要
Effective human–robot collaboration (HRC) requires real-time understanding of human pose, shape, and location within shared workspaces. This informs robot control strategies, enabling safe and efficient trajectory planning for collision prediction and avoidance. Existing methods for human motion tracking in collaborative environments often lack accurate 3D localization, human surface mesh reconstruction, or the practicality needed for real-world deployment. This paper introduces a method to digitally reconstruct an HRC work cell in 3D, enabling real-time monitoring of human motion, robot movement, and key spatial metrics, such as critical distances. Our proposed approach, TRIPL3, is a fully neural network-based approach that operates on input from a single RGB-D camera and achieves real-time deployment due to its low computational requirements. Extensive experiments and benchmarking validate our method’s capability for real-time 3D safety monitoring of human–robot interaction (HRI). Additionally, our approach outperforms state-of-the-art (SoTA) methods in critical distance monitoring.