Mixed-Reference Quality Assessment for Novel View Synthesis Scenes
摘要
Novel View Synthesis (NVS) aims to synthesize new viewpoints of unknown camera poses through the features of a source image sequence of an existing object or scene. In terms of the quality assessment of the generated scene, the Full-reference (FR) method often ignores the scene quality outside the reference viewpoint, and the No-reference (NR) method lacks the quality reference of ground truth. Considering the perceptual characteristics of the human visual system (HVS), it is worth studying to obtain the scene quality score by simulating the human visual perception process. Specifically, we use Transformer as the backbone network and design a Mixed Reference (MR) quality assessment method, which captures the quality characteristics of the FR field with obtaining the feature differences of the scene, provides the quality reference to the NR field to extract distortion features, and integrates the quality characteristics from the FR and NR field. Finally, the probability distribution of the viewpoint is used to obtain a complete scene score. In order to make the algorithm have sufficient generalization ability, we extend six complex community scene datasets to deal with typical distortion types such as dynamic artifacts and light reflection in the rendering processes. Extensive experimental results validate that the proposed MR method outperforms existing methods in scene assessment and is more robust in multiple cross-dataset scenes.