A Sketch-based Image Retrieval Via Zero-Shot Learning For Painting Online Education And Its Application Research

With the popularization of intelligent devices,the convenience of social platforms and the rapid development of Internet of Things technology,the accumulated data shows explosive growth at this stage.How to use big data and artificial intelligence related technologies to quickly retrieve from diversified data has drawn more and more intensive attention from researchers.Since the 1970s,image retrieval has become a hot research topic in the field of computer vision.It mainly includes two directions:Text-based Image Retrieval(TBIR)and Content-based Image Retrieval(CBIR).The research of CBIR is mainly divided into Photo-based Image Retrieval(PBIR)and Sketch-based Image Retrieval(SBIR).At present,SBIR technology is more and more popular among researchers since the drawing of these abstract and simple sketches is relatively easy for users.However,there are some challenges in the research of SBIR.Firstly,the sketches and natural images belong to different data fields,so their data distributions in the high-dimensional space varies,i.e.,the issue of inconsistent distributions of data fields.Secondly,the inherent pivot points in high vitellian space will also seriously affect the performance of the model and the accuracy of experiments.Thirdly,the judgment bases of image similarity for human and computer are different,so there is inconsistency between the visual information obtained by computer and the semantic information understood by human,i.e.,the inherent semantic gap in the field of computer vision.Fourthly,in the era of big data,researchers need to label the new emerging data,and some extra work is also needed to reconstruct and retrain the existing models.Therefore,the expensive manual annotation and time-consuming model retraining seriously hinder the wide application of SBIR technology in various intelligent products.Based on the above-mentioned issues,this paper proposes a Joint Embedding Semantic Feature for Multi-Branch Framework(JESF-MBF model),which belongs to the research field of zero-sample image retrieval based on sketch.JESF-MBF model can successfully construct the mapping relationship between sketch and image in visual feature space and semantic embedding space,as well as the internal semantic correlation between known tags and unknown tags in tag set,so in the test phase given a sketched new data with unknown tags,the JESF-MBF model can still retrieve real images of the same tags based on JESF-MBF model(Note:the known tags refer that there are sketch-image labels in the training stage and the unknown tags refer otherwise).JESF-MBF model uses domain discrimination loss,triplet loss and semantic loss to optimize the model,thus effectively alleviating the above problems.In the experiments,we use sketchy dataset to train and test the proposed model.The experimental results show that JESF-MBF model is superior to other algorithms in terms of relevant experimental indicators,and show its excellent ability in image retrieval.In order to actively respond to the national intelligent education policy,we also design an online painting education platform based on the proposed JESF-MBF model.Using this platform,users can draw simple object outlines to retrieve natural images with the same labels.This process can not only enhance users'cognitive level and perception of the natural world,but also improve their painting skills.