A Robust Framework for 3D Object Reconstruction via Instance Segmentation and Neural Registration
摘要
In robotic tasks such as medical surgery and agricultural harvesting, accurate 3D perception of target objects remains challenging due to unstructured environmental occlusions, lighting variations. Pre-reconstructed 3D model of the target can enhance the reliability and accuracy of robotic systems. Existing object-level 3D reconstruction methods often separate scene reconstruction from object detection, relying on manual processing to refine 3D models. We introduce a novel framework for robust 3D object reconstruction from RGB-D sequences. Specifically, we propose a spatiotemporally consistent object segmentation mechanism that integrates a 2D segmentation model with 3D geometric clustering. The PREDATOR model is then leveraged to perform chunk-to-model registration, with a voting-based strategy effectively identifying outlier registration results. Experiments on the TUM dataset demonstrate the framework’s adaptability to challenging scenarios. 3D reconstruction achieves a Chamfer Distance of 0.017 m, and the proposed outlier registration filtering algorithm achieves a precision of 0.96.