Research On GAN Hierarchical Semantic Decomposition Model For Street View Image Generation And Editing Technology

In recent years,Generative Adversarial Networks(GANs)have been widely used to process various machine learning and artificial intelligence tasks.However,it is often difficult to edit the output of the model due to its over-parameterized nature.The research shows that different layers of GAN generator can capture semantic information with different levels of abstraction,which can be separated by an explicit method to control the semantic properties of the output.Many related studies have been proposed in recent years.However,the supervision method is limited by the supervision conditions and can find less semantic direction.The unsupervised method needs to overcome the problem of attribute entanglement.In addition,existing researches mainly focus on the semantic properties of faces,animals and cartoon images,and there is a lack of relevant researches on Street View images.In recent years,artificial intelligence has been deeply developed and applied in the field of autonomous driving.The research on Street View image generation and editing tasks can expand the sample training set required for autonomous driving training and promote the development of related fields.To solve the above problems,this thesis proposes a Street view image generation and attribute editing model based on GAN hierarchical semantic decomposition.In the generator,a potential mapping network is used to map the input noise z to an intermediate potential space W with rich disentanglement semantics.Then,orthogonal linear subspaces are introduced to separate explainable dimensions in different layers of the generator.The orthogonality of these dimensions is guaranteed by combining Hessian penalty with Jacobi regularization.These dimensions can be trained to control the attributes of the output image individually or through a linear combination.By traversing the corresponding coefficients within a certain range,the generated street view image can be continuously changed in specific semantic attributes.In order to apply the hierarchical semantic decomposition model to the editing task of real images,a GAN inversion algorithm is also designed.An autoencoder network structure is formed by using the encoder that needs to be optimized and the generators of the hierarchical semantic decomposition model that have been pre-trained.By minimizing the difference between the real image and the reconstructed output,An encoder which can map the real image to the implicit encoding is obtained.Based on the research results of hierarchical decomposition model and GAN inversion algorithm,a Street View image generation and editing system is designed to facilitate interactive street view image generation and editing.The semantic decomposition model was tested on three data sets,WPI Traffic Light,Stanford Cars and WPI Traffic Light,and various semantic concepts such as scene brightness,target position and image tone were accurately decomposed.The implicit code generated by the inversion model in this paper can faithfully reconstruct the input real image,and can be combined with the semantic decomposition model to edit multiple attributes of the real image.The experimental results of this paper have achieved good performance in both qualitative and quantitative analysis,which shows the effectiveness of the proposed method.