Research Of Complex Reasoning And Multimodal Representation Learning Based On Knowledge Graph Embedding

Combining human knowledge with artificial intelligence is one of the directions that the academic community has been exploring,and it is also a necessary means to enhance artificial intelligence algorithms to solve complex problems.Knowledge graphs can refine human knowledge,and then enable subsequent algorithm applications through embedding technology,which has become a hot research field in recent years.At this stage,the complex problems in the knowledge graph scene mainly include increasingly complex query problems and more and more extensive data modes.How to deal with complex query problems and multi-modal scenarios based on the research of traditional knowledge graph embedding has become the focus of the industry and academia.Current research on knowledge graph embedding focuses on how to better model knowledge triplets,which only have simple reasoning capabilities in a single modality,and are difficult to adapt to downstream problems such as complex reasoning in knowledge graphs and multimodal knowledge graph representation.This topic focuses on complex reasoning problems and multimodal representation problems based on knowledge graph embedding.The main contents are as follows:This thesis improves the existing knowledge graph complex reasoning algorithm based on knowledge graph embedding,and proposes the SignalE algorithm.Aiming at the problem that the traditional knowledge graph embedding form cannot handle logical non-operations well,SignalE proposes a new embedding form-discrete signal embedding to deal with complete first-order logic query problems.In the same dimension,discrete signal embedding has a time-domain representation and a frequencydomain representation that can be transformed into each other.Based on the frequency-domain representation,the relational projection operation is simplified from a neural network to a translation operation,and logical negation models the logical-semantic inversion by flipping the imaginary part of the frequency-domain representation.Experimental results on multiple benchmark datasets show that the SignalE algorithm proposed in this topic not only surpasses many baseline models in terms of predictive performance,but also reduces the scale of model parameters and greatly improves the training speed.This thesis improves the existing multimodal knowledge graph representation method,proposes the MPKGAC algorithm,and applies it to the multimodal knowledge graph completion task.Existing multimodal knowledge graph completion methods focus on using the features of text or visual modalities to enhance structured knowledge representation,but they are not sufficient for the interaction and utilization of multimodal information,and ignore knowledge modalities,text Modal and Visual Modal Independence.This topic proposes a multi-modal knowledge graph representation method based on the three-stream mechanism,independently modeling knowledge,text,and image three modalities,and controls multi-modal knowledge through a directional prefix attention mechanism and a token-level similarity matrix between modalities.The interactive process of modal information.A multimodal synthesis decoder is proposed to fully utilize the complementary information of modalities to gain prediction performance.Compared with the current state-of-the-art various baseline models,MPKGAC achieves the best prediction performance in all indicators.In addition,MPKGAC has a certain practical application ability in the application of multi-modal product attribute value completion in actual e-commerce scenarios.This thesis carried out engineering practice for knowledge map technology to empower higher education.For the stage of higher education,this topic builds a subject knowledge map ontology around electronic information disciplines,and develops a knowledge map intelligent search and visualization website.Through map visualization,map roaming,knowledge positioning,knowledge association and other functions,users can more intuitively and quickly Browse the subject knowledge system more accurately and locate the target knowledge points more accurately,and understand the knowledge points from various modes such as pictures,formulas,explanatory videos,and encyclopedias.In addition,the abovementioned complex reasoning algorithm has been concretely practiced on the subject knowledge graph platform,and the supporting platform has the ability to retrieve complex problems.The whole system provides a variety of knowledge visualization functions for student users,enabling students in the higher education stage to actively grasp the subject knowledge system.