ACDiff: Angle Craft Diffusion Model for Novel View Synthesis
摘要
Single-view novel view synthesis, the process of generating images from new perspectives using only a single reference image, is a vital yet formidable task. Traditional methods often employ conventional computer graphics techniques or deep learning approaches, but these frequently grapple with issues such as limited adaptability to intricate scenes, inability to maintain fine-grained details, and challenges in handling occlusions and texture inconsistencies. To tackle these challenges, we present the Angle Craft Diffusion Model (ACDiff), a groundbreaking network framework specifically tailored to merge CLIP’s multimodal understanding with the unique demands of our task. This innovative approach effectively bridges the gap between angle-specific textual descriptions and visual data in novel view synthesis. Moreover, within the ACDiff framework, we integrate a refining module that uses the image generated in the previous phase as a conditioning factor. This module is instrumental in restoring texture and enhancing the consistency of fine details in the synthesized images. Our method’s superior fidelity and realism are evidenced through both qualitative and quantitative results, underscoring its ability to produce visually compelling images that closely resemble the ground truth across various evaluation metrics.