Neural Radiance Fields (NeRF) are at the forefront of view synthesis technology, renowned for their versatility and ease of implementation across various applications. However, their integration into multimedia environments faces challenges: objects occlude background information during motion, which usually compromises the quality of reconstructions. In this paper, we present a novel framework to exclude dynamic interference from NeRF scenes, enhancing the practicability in multimedia applications. Our method leverages perceptual optimization, informed by image quality assessment (IQA), and employs text-guided 2D image inpainting to address view synthesis inaccuracies. Furthermore, we propose a new and challenging dataset of real-world scenes to address the lack of evaluation ground truth for dynamic object inpainting in scenes. Experimental results show that our method significantly outperforms existing methods in terms of appearance metrics for the task of removing dynamic objects from scenes.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing Multimedia Applications by Removing Dynamic Objects in Neural Radiance Fields

  • XianBen Yang,
  • Tao Wang,
  • He Liu,
  • Yi Jin,
  • Congyan Lang,
  • Yidong Li

摘要

Neural Radiance Fields (NeRF) are at the forefront of view synthesis technology, renowned for their versatility and ease of implementation across various applications. However, their integration into multimedia environments faces challenges: objects occlude background information during motion, which usually compromises the quality of reconstructions. In this paper, we present a novel framework to exclude dynamic interference from NeRF scenes, enhancing the practicability in multimedia applications. Our method leverages perceptual optimization, informed by image quality assessment (IQA), and employs text-guided 2D image inpainting to address view synthesis inaccuracies. Furthermore, we propose a new and challenging dataset of real-world scenes to address the lack of evaluation ground truth for dynamic object inpainting in scenes. Experimental results show that our method significantly outperforms existing methods in terms of appearance metrics for the task of removing dynamic objects from scenes.