In virtual environments, where dynamic and interactive 3D content is essential, realistic lighting plays a crucial role in enhancing immersion. However, traditional relighting methods often rely on multiview captures or computationally expensive neural rendering techniques, making them impractical for real-time applications. In this work, we propose SV-GaSRelight, a novel single-view 3D scene relighting approach specifically designed for human relighting. Our main contribution is to exploit human bodies in the scene as 3D probes that can help to decompose lighting conditions from surface albedo and orientations. This is possible thanks to very powerful models that are able to reconstruct 3D human bodies from a single view. Our approach reconstructs a 3D human model from a single image and generates synthetic multi-view data with corresponding camera poses. We then simulate realistic lighting effects by incorporating physically inspired reflectance modeling and light transport into the 3D representation. To introduce lighting information into this representation, we leverage a neural-based light field mechanism. We assess the effectiveness of our approach by comparing it to an existing method that requires multi-view inputs for relighting. Additionally, we conduct a user study to evaluate the perceived realism of the rendered scenes. Experimental results demonstrate that our single-view-based method achieves comparable visual quality to multi-view relighting techniques. Our project is available on https://sj-programming.github.io/SV-GaSRelight/ .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

SV-GaSRelight: Single-View Gaussian Splatting for 3D Human Relighting

  • Sonain Jamil,
  • Damien Muselet,
  • Alain Trémeau,
  • Philippe Colantoni

摘要

In virtual environments, where dynamic and interactive 3D content is essential, realistic lighting plays a crucial role in enhancing immersion. However, traditional relighting methods often rely on multiview captures or computationally expensive neural rendering techniques, making them impractical for real-time applications. In this work, we propose SV-GaSRelight, a novel single-view 3D scene relighting approach specifically designed for human relighting. Our main contribution is to exploit human bodies in the scene as 3D probes that can help to decompose lighting conditions from surface albedo and orientations. This is possible thanks to very powerful models that are able to reconstruct 3D human bodies from a single view. Our approach reconstructs a 3D human model from a single image and generates synthetic multi-view data with corresponding camera poses. We then simulate realistic lighting effects by incorporating physically inspired reflectance modeling and light transport into the 3D representation. To introduce lighting information into this representation, we leverage a neural-based light field mechanism. We assess the effectiveness of our approach by comparing it to an existing method that requires multi-view inputs for relighting. Additionally, we conduct a user study to evaluate the perceived realism of the rendered scenes. Experimental results demonstrate that our single-view-based method achieves comparable visual quality to multi-view relighting techniques. Our project is available on https://sj-programming.github.io/SV-GaSRelight/ .