Research And Implementation Of Knowledge Graph Reasoning Method Based On Reinforcement Learning

The knowledge graph represents the massive amount of semantic information in the Internet in a way that can be understood by computers,structures the world knowledge,and stores it in the database in the form of triples,which helps us better organize and manage data.However,due to the limitations of the knowledge graph construction algorithm and the updateability of human knowledge,it is incomplete,so the knowledge reasoning task is proposed to solve the completion task of the knowledge graph.This thesis mainly studies the knowledge reasoning problem in the knowledge graph,which is used for the completion of the knowledge graph.This thesis analyzes the current research status at home and abroad,and proposes a knowledge reasoning method based on reinforcement learning.The process of knowledge reasoning is modeled as a Markov decision problem,and the strategy agent and environment in the problem are represented respectively.Also reward mechanisms are modeled.The main work accomplished in this thesis is as follows:Firstly,this thesis proposes a policy agent for hierarchical action selection.During the interaction between the knowledge graph environment and the agent,the reinforcement learning agent usually faces hundreds of feasible actions,that is,the output layer of the policy network often has a large number of actions.dimension,facing the problem of dimensional disaster.Therefore,this thesis divides the selection process of reasoning path into three steps,which are the selection of relation clusters,the selection of relations and the selection of entities,which greatly reduces the action space and alleviates the dimensional disaster.Secondly,this thesis proposes a convolutional neural network based environment encoder.In this thesis,the starting entity,query relationship,current entity,path information,etc.in the process of agent walking in the knowledge graph environment are represented by vectors,and in order to enhance the interaction between different parts of the agent,it can better describe the current location of the agent.environment,using the environment feature extractor based on convolutional neural network,and using mixed vector representation for the knowledge map environment to better guide the agent’s decision-making.Thirdly,design a reward mechanism that integrates path information.The currently adopted non-01 agent-guided model only pays attention to whether the destination is reached,and the path quality that triggers the agent’s reasoning.The quality of the inference path considers the inference process of the body in the intelligent model of the advanced reward mechanism.Finally,this thesis constructs a knowledge reasoning path visualization system by taking advantage of the path interpretability of the knowledge reasoning model based on reinforcement learning.The system includes three parts:dataset management module,model visualization training module and inference path visualization.