Research On Performance Optimization And Lightweight Methods Of Image-to-image Translation Model Based On NICE-GAN

Image-to-image translation is a task similar to text translation,which aims to convert images from one style to another.In real life,image-to-image translation technique has many application scenarios,such as image stylization,medical image segmentation,data augmentation,etc.Although many works have been carried out in this field,the poor quality of image generation and the difficulty of implementing the huge image-to-image translation model have always been urgent problems to be solved.Based on the representative model——NICE-GAN in the field of image-to-image translation,this thesis proposes some methods to improve results from the perspectives of network structure and convolution kernel,and uses the knowledge distillation to lighten the model.The main research contents are as follows:(1)Aiming at the problem of poor image-to-image translation results caused by insufficient feature extraction of NICE-GAN in some datasets,this thesis proposes an image-to-image translation model,called AFF-UNIT(Adaptive Feature Fusion for Unsupervised Image-to-Image Translation),which uses an adaptive feature fusion method.Firstly,an adaptive fusion method is introduced in the feature extraction stage,combining the feature attributes of each dataset to achieve corresponding feature attention.Then,a similarity loss is designed to constrain the features extracted by the network before and after translation,and a multi-scale discriminator is used to evaluate the quality of images generated from different levels.Finally,the superiority of the AFF-UNIT is verified through comparative experiments with other image-to-image translation models on public datasets.(2)In view of the problem that there are a large number of convolution operations in NICE-GAN,the conventional convolution’s representation capability is limited.This thesis introduces a convolution method called Cond Conv for parameterized conditional calculation.Firstly,Cond Conv designs multiple convolution kernels for each convolution operation.Next,it uses the routing function to calculate the weight occupied by each kernel and performs a linear combination of the above convolution kernels to improve the representation ability of the convolution.Then,the conventional convolution in NICE-GAN’s residual network is replaced by Condconv.Finally,the results of the model before and after the improvement are compared,and the effectiveness of the Cond Conv in improving the image-to-image translation model is verified.At the same time,considering that both the adaptive feature fusion and Cond Conv can improve the result of NICE-GAN,a natural idea is to combine them to obtain AFF-UNIT*.The experiments show that the AFF-UNIT* has better results.(3)To solve the problem that image-to-image translation models,which occupy a lot of storage space and computing resources,can not be effectively applied on mobile devices,this thesis designs a knowledge distillation algorithm for AFF-UNIT* for lightweight.Firstly,the channels in the residual network of AFF-UNIT* and the redundant part of convolution kernel in Cond Conv are trimmed.Then,the global kernel alignment method is used to transfer knowledge between teacher model and student model,and AFF-UNIT*-light is obtained combined with other loss functions for training.Finally,the complexity and performance of the lightweight model are evaluated.It is shown that the lightweight method based on knowledge distillation designed in this thesis can compress AFF-UNIT* more than ten times without greatly reducing the quality of image-to-image translation.