Research On Short Text Representation Model For Images And Texts

Compared with the text content,the image content can present the information at a glance,so the trend of combining the two is displayed in reading.Taking functional neuroimaging literature as an example,the graphic information in the literature often appears in front of the reader in a complementary manner.However,the current mainstream text representation model can not make full use of the graphic information in the literature.There are a series of problems such as insufficient organization of graphic corpus information,insufficient semantic representation of short text,and lack of comprehensive utilization of graphic information..In response to these problems,this paper takes functional neuroimaging literature as an entry point and conducts research on short text representation models for graphics and text.The main work includes:(1)In view of the current lack of multimodal corpus information in the field of functional neuroimaging,this paper constructs a functional neuroimaging training set and a text training set.First,according to the different perspectives of the pictures in the functional neuroimaging literature,a training dataset of functional neuroimaging pictures is constructed;Secondly,for the texts in the functional neuroimaging literature,through the calculation of the semantic similarity of short texts,the corresponding sets of related semantic pairs and non-related sets of semantic pairs are obtained,and a short text training data set is constructed.Through the construction of graphic data sets,it provides sufficient data support for the training of text representation model and graphic fusion representation model.(2)Aiming at the problem of lack of semantic representation ability of the text model due to the sparse short text features and low topic attention,this paper proposes the LDA-based short text representation model “Weighted-LDA-TVM”.The latent Dirichlet allocation method is used to capture the latent topics of the short text,and the particle swarm optimization algorithm is used to learn the corresponding weight values of the topic words.The weighted calculation of the text topic words realizes the semantic focus of the short text and a short text representation model based on weighted subject word vectors is obtained.In this paper,the model is evaluated by using the similarity measurement experiment between short texts.The experimental results show that the model has a good ability to distinguish and semantically express the similarity of short texts.(3)In view of the lack of comprehensive utilization of graphic information in functional neuroimaging literatures,a graphic fusion representation model based on attention mechanism is proposed.The traditional theme learning method can only express the text,but cannot effectively use the related images.This article innovatively applies the image description technology based on the attention mechanism to the field of neuroimaging document mining,and merges with the traditional text-based LDA topic model to form a new Attend-NIC-LDA topic model.The model has focused on the recognition of the subject keywords in the literature,combined with the neuroimaging image information in the literature,and has focused on the mining of brain cognitive function keywords.The experimental results based on the PLo S One journal data set show that the Attend-NIC-LDA topic model proposed in this paper is not only superior to the current mainstream topic model in semantic clustering,but also makes up for the systematicity of the brain regions and corresponding cognitive functions under the current rapid growth of open storage literature resources and the continuous acceleration of brain cognitive function updates.