Robotic unscrewing is a critical operation in the disassembly of waste of electric and electronic equipment (WEEE), attracting considerable research interest due to its complexity and high precision demands. To simplify the screw detection challenge, most existing methodologies have limited their scope to a constrained degree of freedom (DoF), typically focusing on 2-DoF for disassembling products with flat geometries. However, this approach confronts limitations in addressing the complexity of domestic appliances, where screws may be positioned in varied locations with different orientations, leading to potential failures in accurate screw detection and pose estimation. To address these challenges, this study introduces a novel multimodal fusion approach designed to enhance screw detection and accurately estimate the corresponding depth and normal orientations. Both RGB and depth information are leveraged through a Faster RCNN-based architecture to preserve detailed texture and spatial information. Different from prior conventional methods that often rely on segmenting objects as a preliminary step, our system is capable of directly identifying multiple objects and determining their 6D pose without prior segmentation. To overcome the limitation caused by insufficient data and to effectively speed up the training process, a synthetic dataset comprising various workpieces is created by a physically-based simulator. Extensive experiments conducted on both synthetic and real-world datasets demonstrate the superiority of our proposed method in detecting and localizing multiple screws with high precision, verifying the effectiveness of our proposed approach, and highlighting its potential to significantly improve the efficiency and accuracy of robotic unscrewing tasks.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Synthetic Data-Driven Multimodal Fusion-Based Screw Pose Estimation in Robotic Disassembly

  • Chuangchuang Zhou,
  • Hui Zhang,
  • Yifan Wu,
  • Jef R. Peeters

摘要

Robotic unscrewing is a critical operation in the disassembly of waste of electric and electronic equipment (WEEE), attracting considerable research interest due to its complexity and high precision demands. To simplify the screw detection challenge, most existing methodologies have limited their scope to a constrained degree of freedom (DoF), typically focusing on 2-DoF for disassembling products with flat geometries. However, this approach confronts limitations in addressing the complexity of domestic appliances, where screws may be positioned in varied locations with different orientations, leading to potential failures in accurate screw detection and pose estimation. To address these challenges, this study introduces a novel multimodal fusion approach designed to enhance screw detection and accurately estimate the corresponding depth and normal orientations. Both RGB and depth information are leveraged through a Faster RCNN-based architecture to preserve detailed texture and spatial information. Different from prior conventional methods that often rely on segmenting objects as a preliminary step, our system is capable of directly identifying multiple objects and determining their 6D pose without prior segmentation. To overcome the limitation caused by insufficient data and to effectively speed up the training process, a synthetic dataset comprising various workpieces is created by a physically-based simulator. Extensive experiments conducted on both synthetic and real-world datasets demonstrate the superiority of our proposed method in detecting and localizing multiple screws with high precision, verifying the effectiveness of our proposed approach, and highlighting its potential to significantly improve the efficiency and accuracy of robotic unscrewing tasks.