Neural Radiance Fields (NeRF) excels in generating realistic novel views of 3D scenes. However, generating these views based on few input views is challenging because of the insufficient data to recover the radiance field of the entire scene. To address this, we present a method for reconstructing NeRF from just three closely spaced input views, leveraging both geometric and semantic consistencies. Geometric consistency is ensured using a cost volume and variance evaluation across voxels, effectively reconstructing visible areas. For unseen areas, semantic consistency aligns semantic vectors between rendered and input images using pre-trained feature extractors. Combining these consistencies allows for precise reconstruction of both seen and unseen areas. Additionally, we enhance NeRF learning through entropy minimization for volume density regularization, black blending to eliminate floating artifacts, and relative learning-rate decay to facilitate learning of volume density. This multifaceted approach outperforms existing methods that rely on single consistency types, showing superior quantitative and qualitative results.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Few-Shot View Synthesis Based on Geometric and Semantic Consistency

  • Mizuki Kojima,
  • Rei Kawakami,
  • Masatoshi Okutomi

摘要

Neural Radiance Fields (NeRF) excels in generating realistic novel views of 3D scenes. However, generating these views based on few input views is challenging because of the insufficient data to recover the radiance field of the entire scene. To address this, we present a method for reconstructing NeRF from just three closely spaced input views, leveraging both geometric and semantic consistencies. Geometric consistency is ensured using a cost volume and variance evaluation across voxels, effectively reconstructing visible areas. For unseen areas, semantic consistency aligns semantic vectors between rendered and input images using pre-trained feature extractors. Combining these consistencies allows for precise reconstruction of both seen and unseen areas. Additionally, we enhance NeRF learning through entropy minimization for volume density regularization, black blending to eliminate floating artifacts, and relative learning-rate decay to facilitate learning of volume density. This multifaceted approach outperforms existing methods that rely on single consistency types, showing superior quantitative and qualitative results.