In recent years, the wide application of deep learning has made rapid progress in image inpainting technology. However, the current image inpainting algorithms still have some shortcomings in the ability to grasp the overall structure and understand the detailed information, which will lead to texture blur and structural distortion in the generated content. For the purpose of tackling the problems mentioned above, we propose a two-stage image inpainting algorithm based on structural information guidance. In the first stage, we enhance the model’s capacity to capture the global structure through the guidance of gradient maps and wireframes. In the next stage, we propose the Content Filling Network based on GAN. Specifically, we propose the Prior Addition Module to avoid the information loss during the process of convolution and normalization, and we propose the Texture Generation Module to assist the model in generating more detailed texture information. In addition, we improve the discriminator to make the model generate more realistic and vivid images. Extensive experiments are carried out to demonstrate that our approach performs noticeably better than current methods in both quantitative and qualitative evaluation.