A Gated Convolution and Self-Attention-Based Pyramid Image Inpainting Network

Abstract

Aiming at the problems of imperfect inpainting edges, mismatching inpainting content and slow training caused by large network model parameters and high requirements for image inpainting edge consistency and semantic integrity, this paper designs a gated convolution and self-attention-based pyramid network (GAP-Net), the network is based on U-Net, and it integrates the gated convolution method and the pyramid loss and changes the feature extraction strategy. In addition, we design a self-attention mechanism module and an attention transfer module for the network, designing and adding content loss and perceptual loss for the network, generating a new data distribution between generated and real images. The comparative analysis of experiment with the PEN-Net method and the Gated method is conducted in the same experimental environment. The experimental results show that our method can increase the extraction of useful information from damaged image areas by gated convolution and pyramid loss. Self-attention mechanism module and the attention transfer module can guide the conversion process of high-level semantic features to image information more accurately, and the content and perceptual loss can accelerate and improve the learning ability of the network, this method improves the repair effect and accelerates the network learning speed.