Purpose Augmented Reality in Minimally Invasive Surgery has made tremendous progress in organs including the liver and the uterus. The core problem of Augmented Reality is registration, where a preoperative patient’s geometric digital twin must be aligned with the image of the surgical camera. The case of the kidney is yet unresolved, owing to the absence of anatomical landmarks visible in both the patient’s digital twin and the surgical images. Methods We propose a landmark-free approach to registration, which is particularly well-adapted to the kidney. The approach involves a generic kidney model and an end-to-end neural network, which we train with a proposed dataset to regress the registration directly from a surgical RGB image. Results Experimental evaluation across four clinical cases demonstrates strong concordance with expert-labelled registration, despite anatomical and motion variability. The proposed method achieved an average tumour contour alignment error of <InlineEquation ID="IEq1"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="11548_2025_3473_Article_IEq1.gif" Format="GIF" Height="14" Rendition="HTML" Resolution="72" Type="Linedraw" Width="64" /> </InlineMediaObject> <EquationSource Format="TEX">$7.3 \pm 4.1$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>7.3</mn> <mo>±</mo> <mn>4.1</mn> </mrow> </math></EquationSource> </InlineEquation> mm in <InlineEquation ID="IEq2"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="11548_2025_3473_Article_IEq2.gif" Format="GIF" Height="14" Rendition="HTML" Resolution="72" Type="Linedraw" Width="64" /> </InlineMediaObject> <EquationSource Format="TEX">$9.4 \pm 0.2$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>9.4</mn> <mo>±</mo> <mn>0.2</mn> </mrow> </math></EquationSource> </InlineEquation> ms. Conclusion This landmark-free registration approach meets the accuracy, speed and resource constraints required in clinical practice, making it a promising tool for Augmented Reality-Assisted Partial Nephrectomy.

Landmark-free automatic digital twin registration in robot-assisted partial nephrectomy using a generic end-to-end model

期刊论文
发表日期：

2025年7月17日

查看全文

Kilian Chandelon,
Alice Pitout,
Mathieu Souchaud,
Julie Desternes,
Gaëlle Margue,
Julien Peyras,
Nicolas Bourdel,
Jean-Christophe Bernhard,
Adrien Bartoli

摘要

Purpose

Augmented Reality in Minimally Invasive Surgery has made tremendous progress in organs including the liver and the uterus. The core problem of Augmented Reality is registration, where a preoperative patient’s geometric digital twin must be aligned with the image of the surgical camera. The case of the kidney is yet unresolved, owing to the absence of anatomical landmarks visible in both the patient’s digital twin and the surgical images.

Methods

We propose a landmark-free approach to registration, which is particularly well-adapted to the kidney. The approach involves a generic kidney model and an end-to-end neural network, which we train with a proposed dataset to regress the registration directly from a surgical RGB image.

Results

Experimental evaluation across four clinical cases demonstrates strong concordance with expert-labelled registration, despite anatomical and motion variability. The proposed method achieved an average tumour contour alignment error of $7.3 \pm 4.1$ $7.3 \pm 4.1$ mm in $9.4 \pm 0.2$ $9.4 \pm 0.2$ ms.

Conclusion

This landmark-free registration approach meets the accuracy, speed and resource constraints required in clinical practice, making it a promising tool for Augmented Reality-Assisted Partial Nephrectomy.