Research On High Quality Image Inversion And Editing

Image generation is an essential issue in computer vision.Deep learning-based image generation aims at training a generative model which fits the data distribution.GAN(Generative Adversarial Network)is one of the mainstream generative models.This paper focuses on the image generation based on the condition of images,including unpaired image-to-image translation,face attribute editing based on the pre-trained Style GAN,and image inversion,which is the preceding task of face attribute editing.This paper enhances the quality of editing images in the manner of improving the selection of anchors,modeling global relations and promoting disentanglement among attributes.Main research contents of this paper are as follows:(1)For the unpaired image-to-image translation,the self-attention mechanism is brought in the autoencoder.We compute the attention matrix of source features and select the important ones as anchors in contrastive loss according to the entropy.At the same time,the attention matrix is employed to route features in both domains,enhancing the relations between them,thus improving the quality of translated images.The proposed method is validated in three datasets,showing that the source contents maintain in the translated images without adding learnable parameters.(2)For the face attribute editing based on the pretrained Style GAN,we propose to employ cascading transformer blocks in the multi-scale image encoder,where the latent code mines the image features in a coarse-to-fine manner,modeling global relations thus achieving fast and accurate image inversion.Based on the latent code,we investigate the label-and reference-based editing in the latent space,improving the disentanglement among attributes.Extensive experiments are carried out in two datasets,showing high-quality inversion and editing results with a lightweight and efficient model.(3)For high-fidelity face attribute editing,we propose a fine-grained image inversion method based on the refinement of dynamic convolution kernels.The low-rank residuals is presented,which is constructed of low-rank residual codes,refining the dynamic convolution kernels in the pretrained Style GAN.Intensive experiments on face dataset show that the proposed method remains the detailed content in images while achieving the global and local attribute editing flexibly.Moreover,the low-rank residuals can be applied in both the text-driven and the reference-driven domain adaptation tasks.