Accurate registration between intraoperative 2D images and preoperative 3D anatomical structures is a prerequisite for image-guided minimally invasive surgery. Existing approaches for 2D/3D rigid registration, particularly those for X-ray to CT image registration, primarily rely on grayscale-based image similarity metrics. However, such metrics often fail to capture the optimal projection transformation due to their limited contextual information. To address this issue, we propose a novel and intuitive correspondence representation: the overlap of multiple corresponding Regions of Interest (ROIs). By introducing the differentiable Dice coefficient computed on the projection image, we establish a direct link between segmentation and registration within our weakly supervised 2D/3D registration framework. This framework comprises two stages–a learning-based preoperative stage and an optimization-based intraoperative stage–both of which leverage the ROI-based Dice score as a differentiable supervision signal. Additionally, we integrate automatic segmentation methods (e.g., UNet) and prompt-based methods (e.g., MedSAM) into the framework to investigate the impact of different segmentation approaches on registration performances. Furthermore, we validate the generalization ability of the proposed framework by integrating the ROI-based similarity with various similarity measures. Extensive experiments conducted on the DeepFluoro dataset yielded an mTRE of \(0.67\pm 1.34\)  mm, with rotational and translational error values being \(0.2\pm 0.5^{\circ }\) and \(1.6\pm 2.9\)  mm respectively, outperforming existing state-of-the-art methods. The codes are available at https://github.com/CYXYZ/WSReg .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Weakly-Supervised 2D/3D Image Registration via Differentiable X-ray Rendering and ROI Segmentation

  • Yuxin Cui,
  • Max Q.-H. Meng,
  • Zhe Min

摘要

Accurate registration between intraoperative 2D images and preoperative 3D anatomical structures is a prerequisite for image-guided minimally invasive surgery. Existing approaches for 2D/3D rigid registration, particularly those for X-ray to CT image registration, primarily rely on grayscale-based image similarity metrics. However, such metrics often fail to capture the optimal projection transformation due to their limited contextual information. To address this issue, we propose a novel and intuitive correspondence representation: the overlap of multiple corresponding Regions of Interest (ROIs). By introducing the differentiable Dice coefficient computed on the projection image, we establish a direct link between segmentation and registration within our weakly supervised 2D/3D registration framework. This framework comprises two stages–a learning-based preoperative stage and an optimization-based intraoperative stage–both of which leverage the ROI-based Dice score as a differentiable supervision signal. Additionally, we integrate automatic segmentation methods (e.g., UNet) and prompt-based methods (e.g., MedSAM) into the framework to investigate the impact of different segmentation approaches on registration performances. Furthermore, we validate the generalization ability of the proposed framework by integrating the ROI-based similarity with various similarity measures. Extensive experiments conducted on the DeepFluoro dataset yielded an mTRE of \(0.67\pm 1.34\)  mm, with rotational and translational error values being \(0.2\pm 0.5^{\circ }\) and \(1.6\pm 2.9\)  mm respectively, outperforming existing state-of-the-art methods. The codes are available at https://github.com/CYXYZ/WSReg .