The Research On Generative Model Based On Zero-shot Learning

The purpose of Zero-Shot Learning(ZSL)is to train the model only by using the visual features and semantic representation of visible classes,and then use the semantic representation of unseen classes as a bridge,so that the model has the ability of recognizing unseen classes.The existing ZSL method generally uses the semantic representation of the unseen class to generate the corresponding visual samples,so as to solve the problem of missing visual data in the unseen class.However,the ZSL generative model above still has the following problems: first,the number of images of each category in the training datasets of ZSL is small,which limits the generalization ability of the generative model.Secondly,most of the methods ignore the semantic information of the unseen categories.Finally,the existing methods are based on the characteristics of the convolutional neural network(CNN),which contains less information than the RGB images.In response to the above problems,this article proposes the following solutions:(1)Data augmentation strategies such as horizontal flipping,slight rotation and slight scaling are usually used to improve the generalization ability of the model.However,the input of ZSL generative model is the CNN feature of category,so the data augmentation strategies above are difficult to implement.The Mixup mechanism constructs a virtual training example,which makes up for the difficulty of traditional data augmentation strategy in CNN features and is easy to implement.In this paper,we use the Mixup mechanism in the pre-trained classifier and the Generative Adversarial Nets to verify its feasibility for ZSL.In this paper,the mechanism is used in pre-trained classifiers and generative adversarial networks to perform ablation experiments on CUB,AWA2,APY,and SUN datasets.Compared with existing zero-sample learning methods,the classification accuracy of the model is improved.(2)To solve the problem of how to use the semantic representation of unseen classes,this paper proposes a sparse coding method to construct the relationship between seen classes and unseen classes from the semantic representation,and to restrict the generation process of visual features,so as to enhance the similarity between visual features of similar classes and reduce the similarity between visual features of unsimilar classes.This article uses the category relationship constraint mechanism in the generator to conduct ablation experiments,and visualize the generated unseen class samples through the TSNE method.The experimental results show that the category relationship constraint mechanism improves the recognition accuracy of the zero-sample learning model by improving the generation quality of unseen class visual samples.(3)In this paper,experiments on the ZSL method based on RGB image generation are conducted on the CUB and AWA2 datasets.The experimental results show that due to the poor image quality of the datasets and the simple structure of the network model,the quality of the unseen RGB images generated is poor,resulting in the classification accuracy based on the RGB image generation model compared to the CNN feature generation model is poor.