Reconstructing 3D scenes from blurred images is a challenging task, especially under fast motions. Event cameras complement standard RGB frames with microsecond-level temporal resolution and asynchronous outputs. To harness these advantages, we introduce SK2-eNeRF, short for SSM-KAN-enhanced event based NeRF, an event-driven reconstruction framework that significantly enhances Neural Radiance Fields (NeRF) for motion-blur environments. Our framework begins with a multi-layer perceptron (MLP) encoder that maps camera poses to extract raw volumetric density and color features. These representations are then further interpreted via two specialized branches: (1) a Kolmogorov–Arnold Network (KAN) to model nonlinear density distributions, and (2) a State Space Model (SSM) to enhance color estimation. Moreover, a hybrid supervision scheme is introduced, incorporating both blur loss and event loss to ensure accurate modeling of the blur generation process. Extensive experiments on both synthetic and real-world datasets demonstrate that SK2-eNeRF outperforms existing conventional RGB-based and event-based NeRF methods, particularly in fast-motion scenarios. These results underscore the potential of event-guided neural reconstruction as a robust and effective solution for dynamic scene understanding in real-world conditions.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

SK2-eNeRF: Event-Driven Neural Radiance Fields for 3D Reconstruction in Dynamic and Blurry Scenes

  • Guozhen Liu,
  • Jiaming Yu,
  • Panlong Tan,
  • Xiaoyu Zhang,
  • Qinglin Sun,
  • Hao Sun

摘要

Reconstructing 3D scenes from blurred images is a challenging task, especially under fast motions. Event cameras complement standard RGB frames with microsecond-level temporal resolution and asynchronous outputs. To harness these advantages, we introduce SK2-eNeRF, short for SSM-KAN-enhanced event based NeRF, an event-driven reconstruction framework that significantly enhances Neural Radiance Fields (NeRF) for motion-blur environments. Our framework begins with a multi-layer perceptron (MLP) encoder that maps camera poses to extract raw volumetric density and color features. These representations are then further interpreted via two specialized branches: (1) a Kolmogorov–Arnold Network (KAN) to model nonlinear density distributions, and (2) a State Space Model (SSM) to enhance color estimation. Moreover, a hybrid supervision scheme is introduced, incorporating both blur loss and event loss to ensure accurate modeling of the blur generation process. Extensive experiments on both synthetic and real-world datasets demonstrate that SK2-eNeRF outperforms existing conventional RGB-based and event-based NeRF methods, particularly in fast-motion scenarios. These results underscore the potential of event-guided neural reconstruction as a robust and effective solution for dynamic scene understanding in real-world conditions.