Research On Multi-Modal Knowledge Graph Completion Based On Contrastive Learning

Knowledge graphs,as a knowledge base for information storage and organization,store a large number of fact triples that exist objectively in the real world.Currently,these large-scale knowledge graphs have become an important part of many artificial intelligence tasks,such as intelligent recommendation,question answering,etc.However,existing knowledge graphs usually suffer from data incompleteness while scaling up,i.e.,there is a huge amount of missing entity or relation information,which seriously affects their application performance in various intelligent scenarios.Therefore,research on knowledge graph completion is significant.Knowledge graph completion aims to predict missing entities or relations from the known knowledge in the knowledge base,thereby making the knowledge graph more complete.However,most existing knowledge graph completion methods only consider text data,usually ignoring the multi-modal information of the knowledge graph,so it cannot provide rich semantics for knowledge;and most methods tend to deal with triples independently,ignoring the multi-relational heterogeneity structure of entity neighborhoods,so it is unable to capture the complex information hidden in entity neighborhoods;In addition,many methods only rely on triple information and graph structure information,the entity features obtained by learning is poor in semantics,resulting in the large semantic distance and low similarity between related entities in semantic space.To solve the above problems,this thesis proposes two methods for knowledge graph completion:(1)Considering the existing methods could ignore the multi-modal information of knowledge graph,this thesis proposes a multi-modal knowledge graph completion based on graph convolutional networks(MGCN).MGCN firstly learns multi-modal feature representations of entities from text data and image data.Secondly,MGCN utilizes the graph convolutional network to capture the topological features of graphs.Finally,it uses the decoder to perform link prediction tasks for knowledge graph completion.The experimental results on two knowledge graph datasets show that the MGCN model can make full use of the multi-modal information and topology information,improving the effectiveness of the knowledge graph completion.(2)Considering the multi-relational heterogeneous structure of entity neighborhoods and the semantic similarity between entities in knowledge graphs,this thesis proposes a multi-modal knowledge graph completion based on contrastive learning(MKCL),which can encapsulate comprehensive features from multi-modal content and multi-relational structures.Specifically,MKCL first learns entity embeddings from multi-modal content,and then utilizes an attention mechanism to model multi-relational heterogeneity structure to obtain latent representations of entities and relations simultaneously.Particularly,to increase the semantic similarity between related entities,a subgraph contrast constraint is introduced to enhance the quality of entity embeddings.Experimental results on two public datasets show that MKCL outperforms the state-of-the-art methods in various metrics.