<p>Radiance field methods, while achieving impressive quality in novel view synthesis, require abundant input views. In the few-shot setting, both Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) are highly susceptible to overfitting to the training images, resulting in significant performance degradation at novel viewpoints. Furthermore, the majority of existing sparse-view NeRF-based methods suffer from high computational inefficiency during optimization. To address these limitations, we propose <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(\text {F}^2\text {Plenoxels}\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <msup> <mtext>F</mtext> <mn>2</mn> </msup> <mtext>Plenoxels</mtext> </mrow> </math></EquationSource> </InlineEquation> (<b>F</b>ast <b>F</b>ew-shot <b>Plenoxels</b>), a novel and efficient framework for few-shot view synthesis built upon an explicit voxel radiance field representation. <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(\text {F}^2\text {Plenoxels}\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <msup> <mtext>F</mtext> <mn>2</mn> </msup> <mtext>Plenoxels</mtext> </mrow> </math></EquationSource> </InlineEquation> leverages the discrete nature of voxel grids to integrate a dense depth prior, obtained from a powerful 3D vision foundation model with multi-view consistency, to pre-prune voxels in free space. This key step implicitly regularizes scene geometry and eliminates the computational overhead associated with explicit depth supervision during training. Moreover, we introduce a curriculum training strategy that regularizes the degree of Spherical Harmonics (SH) at the early stage of training to mitigate high-frequency overfitting, thereby further improving geometry accuracy with negligible cost. Crucially, we provide an alternative explanation for near-camera overfitting beyond sampling imbalance and propose transmittance-based gradient scaling to effectively reduce floaters close to cameras. Extensive experiments conducted on synthetic and real-world scenes validate that <InlineEquation ID="IEq3"> <EquationSource Format="TEX">\(\text {F}^2\text {Plenoxels}\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <msup> <mtext>F</mtext> <mn>2</mn> </msup> <mtext>Plenoxels</mtext> </mrow> </math></EquationSource> </InlineEquation> significantly outperforms its baseline. Comparative studies against previous few-shot methods demonstrate the superiority of our framework in terms of both synthesized image quality and training efficiency. Code, data, and demos are available at <a href="https://github.com/PJunGH/F2Plenoxels">https://github.com/PJunGH/F2Plenoxels</a>.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

F2Plenoxels: fast voxel radiance fields without neural networks for few-shot view synthesis

  • Jun Peng,
  • Chunyi Chen,
  • Yunbiao Liu

摘要

Radiance field methods, while achieving impressive quality in novel view synthesis, require abundant input views. In the few-shot setting, both Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) are highly susceptible to overfitting to the training images, resulting in significant performance degradation at novel viewpoints. Furthermore, the majority of existing sparse-view NeRF-based methods suffer from high computational inefficiency during optimization. To address these limitations, we propose \(\text {F}^2\text {Plenoxels}\) F 2 Plenoxels (Fast Few-shot Plenoxels), a novel and efficient framework for few-shot view synthesis built upon an explicit voxel radiance field representation. \(\text {F}^2\text {Plenoxels}\) F 2 Plenoxels leverages the discrete nature of voxel grids to integrate a dense depth prior, obtained from a powerful 3D vision foundation model with multi-view consistency, to pre-prune voxels in free space. This key step implicitly regularizes scene geometry and eliminates the computational overhead associated with explicit depth supervision during training. Moreover, we introduce a curriculum training strategy that regularizes the degree of Spherical Harmonics (SH) at the early stage of training to mitigate high-frequency overfitting, thereby further improving geometry accuracy with negligible cost. Crucially, we provide an alternative explanation for near-camera overfitting beyond sampling imbalance and propose transmittance-based gradient scaling to effectively reduce floaters close to cameras. Extensive experiments conducted on synthetic and real-world scenes validate that \(\text {F}^2\text {Plenoxels}\) F 2 Plenoxels significantly outperforms its baseline. Comparative studies against previous few-shot methods demonstrate the superiority of our framework in terms of both synthesized image quality and training efficiency. Code, data, and demos are available at https://github.com/PJunGH/F2Plenoxels.