Purpose: Realistic online 3D reconstruction from endoscopic video is essential for intraoperative inspection and navigation. However, existing approaches often neglect realistic light modeling, rely on offline optimization, or depend on fragile photometric tracking, thereby limiting physically plausible rendering and stable tracking. This study addresses these limitations through an online framework enabling stable tracking and photorealistic endoscopic rendering. Methods: We propose LumenGSLAM, an online RGB-D Gaussian Splatting framework for highly texturized dense reconstruction of endoscopic scenes. The method leverages dense depth input to enable stable geometry estimation and photorealistic appearance modeling through per-Gaussian physically based rendering (PBR). Surface-aligned Gaussian initialization and per-parameter gradient scaling are introduced to enhance anatomical fidelity and geometric consistency. Robust camera pose estimation is achieved via a Gaussian-coupled feature-based tracking module using SuperPoint/LightGlue and Perspective-n-Point (PnP), ensuring reliable localization under rapid motion and challenging illumination conditions. Results: Evaluated on C3VD and SCARED datasets, LumenGSLAM achieves superior online reconstruction and tracking performance. It attains PSNR = 30.6, SSIM = 0.89, and LPIPS = 0.23 on C3VD, outperforming all online baselines and approaching state-of-the-art offline PR-ENDO quality. In tracking, it delivers the lowest Absolute Trajectory Error (ATE = 0.93 mm) and Rotational Error (ARE = 0.98<InlineEquation ID="IEq1"> <EquationSource Format="TEX">$^{\circ }$</EquationSource> <EquationSource Format="MATHML"><math> <mmultiscripts> <mrow /> <mrow /> <mo>∘</mo> </mmultiscripts> </math></EquationSource> </InlineEquation>), demonstrating robustness even under large inter-frame motions. Conclusion: LumenGSLAM establishes a new benchmark for online RGB-D endoscopic reconstruction, achieving photometrically consistent and anatomically accurate mapping through explicit light modeling and geometry-aware Gaussian optimization. Its robustness makes it a promising candidate for intraoperative navigation and future extensions toward dynamic tissue modeling. Project page: <a href="https://github.com/FrancescoLeni/LumenGSLAM">https://github.com/FrancescoLeni/LumenGSLAM</a>.

LumenGSLAM: online physically based rendering with Gaussian Splatting for robust endoscopic reconstruction and tracking

期刊论文
出版模式：

开放获取
发表日期：

2026年5月13日

查看全文

Francesco Leni,
Chiara Lena,
Zhehua Mao,
Sierra Bonilla,
Luca Carlini,
Danail Stoyanov,
Elena De Momi,
Sophia Bano

摘要

Purpose:

Realistic online 3D reconstruction from endoscopic video is essential for intraoperative inspection and navigation. However, existing approaches often neglect realistic light modeling, rely on offline optimization, or depend on fragile photometric tracking, thereby limiting physically plausible rendering and stable tracking. This study addresses these limitations through an online framework enabling stable tracking and photorealistic endoscopic rendering.

Methods:

We propose LumenGSLAM, an online RGB-D Gaussian Splatting framework for highly texturized dense reconstruction of endoscopic scenes. The method leverages dense depth input to enable stable geometry estimation and photorealistic appearance modeling through per-Gaussian physically based rendering (PBR). Surface-aligned Gaussian initialization and per-parameter gradient scaling are introduced to enhance anatomical fidelity and geometric consistency. Robust camera pose estimation is achieved via a Gaussian-coupled feature-based tracking module using SuperPoint/LightGlue and Perspective-n-Point (PnP), ensuring reliable localization under rapid motion and challenging illumination conditions.

Results:

Evaluated on C3VD and SCARED datasets, LumenGSLAM achieves superior online reconstruction and tracking performance. It attains PSNR = 30.6, SSIM = 0.89, and LPIPS = 0.23 on C3VD, outperforming all online baselines and approaching state-of-the-art offline PR-ENDO quality. In tracking, it delivers the lowest Absolute Trajectory Error (ATE = 0.93 mm) and Rotational Error (ARE = 0.98 $^{\circ }$ $^{\circ}$ ), demonstrating robustness even under large inter-frame motions.

Conclusion:

LumenGSLAM establishes a new benchmark for online RGB-D endoscopic reconstruction, achieving photometrically consistent and anatomically accurate mapping through explicit light modeling and geometry-aware Gaussian optimization. Its robustness makes it a promising candidate for intraoperative navigation and future extensions toward dynamic tissue modeling. Project page: https://github.com/FrancescoLeni/LumenGSLAM.