Advances in Image Inpainting: A Comparative Study of CNN, GAN, Transformer, and Diffusion-Based Models
摘要
Image Inpainting refers to the process in computer vision task in which missing parts in a region is completed. Early methods were based on patch propagation in which the surrounding pixels were used to complete the image with missing regions. Such techniques were limited in handling complex information. Advancements were done in the deep neural networks through Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs) which significantly improved the inpainting process. But these methods suffered from instability. The most recent advancements are the transformer architecture and diffusion models. These have established very high benchmark values. These models are able to capture long range dependencies. As a result, these can produce very high-quality images. This paper presents a comparative study of these methods. It tracks the evolution of methods from classical algorithm to generative AI (Artificial Intelligence). The study focuses on some popular datasets – Places2, CelebA-HQ and also ParisStreetView. Using these datasets, we evaluate the performance of six models. The evaluation metrics consists of pixel and structure level metrics such as PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index Measure). There are other high level metrics such as LPIPS (Learned Perceptual Image Patch Similarity) and FID (Frechet Inception Distance) values. Using the evaluation metric, the quantitative performance of the models is analyzed. Further, the qualitative performance of the models is evaluated. This is done by conducting a user study. The qualitative and quantitative evaluation allows us to perform a comprehensive comparative analysis. The results show that diffusion-based models achieve the highest image quality, while the transformer architecture achieves highest consistency. The study further identifies the challenges and future direction in image inpainting.