Research On Key Technologies Of Semantic Retrieval And Organization For Thangka Image

Tibetan culture represented by Thangka images is an important part of China’s minority culture.Its rich national characteristics,unique local style and diverse cultural forms constantly enrich and develop the connotation of Chinese culture.However,there are some problems in the development of Thangka image cultural resources,such as low acceptation by audience,scattered resources,lack of systematic knowledge and material data base,etc.,which hinder the spread and development of Tibetan culture.In order to solve the problems encountered in the protection,inheritance and development of Thangka,this paper proposes a "digital solution of Thangka image",which is supported by automatic annotation technology of Thangka image,retrieval technology of Thangka image based on relevance feedback,extraction technology of Buddhist cultural elements and some other application technologies.The main work of this paper is to achieve the "Thangka image digitization solution" as the goal,combined with the characteristics of Thangka image,in-depth study of various key technologies,to solve the specific scientific problems faced in the realization of the key technologies.The main contents and innovations of this paper are as follows:1.A multi-feature fusion strategy for Thangka image is proposedIn view of the lack of Thangka image data,a Tangka data set is provided,and the production process,annotation strategy and characteristics of the Tangka data set are described in detail,which provides data support for the realization of the key technologies.Because Thangka image is rich in color,texture and feature points,in order to make full use of the advantages of various single image features,a Thangka multi-feature fusion method based on deep learning is proposed,it provides robust image feature representation for subsequent Thangka image automatic annotation technology and relevance feedback retrieval technology.The experimental results show that the fusion of Thangka image features has good generalization performance,and it performs better than the single image features.2.A multi-label Thangka image annotation model based on latent semantic space learning is proposedAiming at solving the problem of small scale of Thangka image dataset,by comparing existing annotation models and referring to the nearest neighbor model,this paper proposed a label extended annotation model based on Bayesian and Thangka semantic nearest neighbor set.Firstly,the model obtains the features of the training set image,and then uses the self-defined Thangka semantic nearest neighbor set and the Thangka semantic label probability defined by Bayesian to obtain the probability of the image to be labeled on each label,and then labels the image.Furthermore,an improved multi-label Thangka annotation model based on latent semantic space learning is proposed to solve the problems of high dimension of image features,unbalanced contribution of feature components and high-level semantic relevance utilization.Using latent semantic space learning technology,Thangka image features are reduced from high-dimensional space to low dimensional space,and the learned weights are assigned to the image features;At the same time,the correlation information between the high-level semantic labels of thangka is embedded into the multi-label classification step to make full use of the high-level semantic information.Experiments show that the proposed automatic annotation model can effectively annotate Thangka images semantically.3.A framework of relevance feedback model based on support vector machine is proposed.Aiming at the problem of high dimension and weight distribution of Thangka image features,an improved method based on latent semantic space is proposed to divide image features into subspaces;In order to solve the problem of insufficient training samples of support vector machine(SVM)caused by the small number of Thangka data sets and the subjectivity of users,a sample expansion method is proposed,and the training samples are expanded by combining with the pseudo label strategy;Aiming at the problem of sample class balance,a method based on clustering is proposed to make the training sample class balance.Finally,experiments show that with the increase of user feedback rounds,the retrieval accuracy of Thangka image is improved continuously,which can be used as a powerful supplement for semantic annotation-based retrieval.4.A multi-scale and two-stage model for extracting Buddhist statues elements is proposed.In the process of extracting Buddhist cultural elements,aiming at the problem of insufficiency of manual annotation boxes in Thangka image data set,this paper proposes a training samples generation algorithm for the classification network of Buddhist cultural elements,which is used to generate a more robust Buddhist classification training set;Aiming at the multi-scale problem of Buddha statues in Thangka,this paper proposes a multi-scale image segmentation method based on the drawing rules of Buddha statues,which enables the Buddha classification network to detect Buddha statues of different size in Thangka;In order to solve the negative coordinate problem of regression network in the detection step,a method of sample generation with negative coordinate is proposed.Finally,the experimental results show that the proposed model can extract the target effectively when the Thangka contains a large number of Buddhist elements with different scales.