Research On Knowledge Graph Completion Method Based On Text And Image

Most existing knowledge representation learning models often use the structured information of triples in knowledge graph,and ignore the text descriptions and images of entities beyond the knowledge graph.Besides,embedding the knowledge graph with tree-like hierarchical structure in Euclidean space would cause a distortion in embeddings.Currently,for knowledge graph completion,the multi-modal information of entities contains a lot of background knowledge,which can enrich the semantic representations of entities.Compared with Euclidean space,hyperbolic space has great advantages in the low dimension of embeddings and in preserving tree-like hierarchical structure of data.Therefore,in order to solve the above problem,this paper proposes a knowledge graph completion method based on the Poincaré ball model of hyperbolic space,which combines structured information of knowledge graph with entity descriptions and entity images.The following research has been done in the paper:1)A knowledge graph completion method based on hyperbolic space entity descriptions is designed.For the descriptions of entities,the sentence-level embedding model is adopted,which better preserves the semantic information between entity contexts.By taking the exponential function defined in hyperbolic space as a bridge,the text representation of entities in European space is mapped into hyperbolic space,and cross fused with the structured representation of the knowledge graph in hyperbolic space.Then,the unbalanced energy functions are adjusted by designing the balance factors,the knowledge representations of triples with the corresponding semantic information are obtained,and the result of knowledge graph completion is improved.2)A knowledge graph completion method based on hyperbolic space entity images is designed.Similarly,the images of entities are also mapped from Euclidean space to hyperbolic space through exponential function,and fused with the structured representation of knowledge graph.Due to the complex relational patterns(1-N,N-1,N-N)and hierarchical structure in knowledge graph,in order to preserve both characteristics of knowledge graph,the complex relationships in knowledge graph are modeled by using the operations of Reflections and Rotations defined in hyperbolic geometry,and the images of entities are fused to enhance the knowledge representation ability of triples,thus improving the experimental results.3)A knowledge graph completion method based on hyperbolic space multi-modal information of entities is designed.The sentence-level embedding model is used to process the descriptions of entities,and the VGGNet model is used to process the images of entities.For the way of entity multi-modal information fusion,two methods of vector concatenation and vector addition are provided,and the influence of entity descriptions and entity images on the experimental results are explored.On the basis of considering the complex relation of knowledge graph,by improving the method of triple distance calculation based on the hyperbolic space,effective fusion of multi-modal information in hyperbolic space is achieved,and filling in missing links in knowledge graph.