OPONeRF: One-Point-One NeRF for Robust Few-shot Rendering
摘要
In this paper, we propose a One-Point-One NeRF (OPONeRF) framework which tackles a critical yet underexplored challenge in few-shot rendering: maintaining robust scene reconstruction and rendering quality under unexpected scene variations. To overcome this challenging issue, existing novel view synthesis approaches either require massive cross-scene training data or struggle to maintain consistent quality under test-scene variations after scene-specific pre-training. To address this, we propose a divide-and-conquer framework in OPONeRF that adaptively responds to local scene variations via personalizing appropriate point-wise parameters, instead of fitting a single set of NeRF parameters that are inactive to test-scene unseen changes. Moreover, to explicitly capture the local uncertainty, we decompose the point representation into deterministic mapping and probabilistic inference. In this way, OPONeRF learns the sharable invariance and unsupervisedly models the unexpected scene variations between the training and testing scenes. To validate the effectiveness of the proposed method, we construct benchmarks from both realistic and synthetic data with diverse test-scene variations including foreground motions, illumination variations and multi-modality noises, which are more challenging than conventional generalization and temporal reconstruction benchmarks. Experimental results show that our OPONeRF outperforms state-of-the-art few-shot neural rendering approaches on various evaluation metrics through benchmark experiments and cross-scene evaluations. We further show the efficacy of the proposed method via experimenting on other existing generalization-based benchmarks and incorporating the idea of One-Point-One NeRF into other advanced baseline methods. Project Page: https://yzheng97.github.io/OPONeRF/.