Research On Knowledge Graph Link Prediction Algorithms Towards Few-shot Data

Link prediction is a crucial task in constructing large-scale knowledge graphs,abbr.,KG,which aims to predict new facts using existing entity and relations in the KG.With the rapid development of AI-related technologies and computer hardware,link prediction techniques based on deep learning have been widely studied in recent years.Under the assumption that each relation has sufficient associated facts,the deep learning based models have achieved impressive performance by leveraging the topological structure of KG.However,due to the diversity of knowledge sources,real-world KGs generally contain many relations with only a few associated facts,known as few-shot relations.Current link prediction models tend to encounter over-fitting problem when handling these relations,leading to severe performance decrement and limiting the application of these models to real-world scenarios.Therefore,research on link prediction tasks for few-shot relations in KG,which involves exploring effective utilization of known data to achieve high-quality link prediction for few-shot relations and completing the knowledge graph completion task,is a valuable and practically significant research topic.In recent years,with the growth of few-shot leaning,many researchers have focused on few-shot link prediction for knowledge graphs,or few-shot link prediction problem.Even so,current researches still face three vital challenges on 1)how to mine entity semantic associations across different facts,2)how to tackle the influence of sparsity and diversity in large-scale knowledge graphs when encoding entities,and 3)how to handle the temporal properties on fewshot temporal link prediction problem.To answer these key challenges,this paper conducts research on few-shot link prediction and proposes dividing the few-shot link prediction process into three core stages:"entity representation learning enhancement","relation representation learning," and"query semantic scoring".This paper has made the following three research contributions:First,a method for entity sequence matching based on entity interaction information.Existing few-shot link prediction studies typically use shallow mathematical methods such as concatenating entity vectors to represent triplets,ignoring the structural information within the triplets.Meanwhile,in the "relation representation learning" stage,only the interactions between the support sets(the references)is considered,neglecting the influence of entity-level interaction between facts,such as co-occurrence correlation between entities.This leads to a decrease in prediction accuracy in restricted scenarios such as 1-shot.To address these two challenges,this paper proposes a few-shot link prediction method called TransAM that considers both intra-and inter-triplet entity interaction information.The TransAM converts few-shot link prediction into an entity sequence prediction problem and designs a Transformer-based matching network with an innovative "local-global" attention mechanism for both"relation representation learning" and "query semantic scoring" stages.The local attention mechanism learns the structural information within the triplets by introducing entity role encoding based on rotation operations and attention mask matrices.Meanwhile,the global attention proposes a separate triplet position encoding mechanism to distinguish entities from different triplets when learning the inter-fact entity interaction information.Experiments on two public few-shot link prediction datasets(NELL-One and Wiki-One)validates the effectiveness of the proposed model design.Second,an enhanced entity representation method for solitary entities based on task semantic pattern graphs.Enriching the semantic representation of entities using the topological structure of knowledge graphs in the "entity representation learning enhancement" stage is critical for existing research to solve fewshot link prediction problems.However,in the real world,large-scale knowledge graphs such as Wikipedia are sparse and diverse,meaning that many entities have very few semantic neighbors connected to them in the knowledge graph,and these neighbors have significantly different associations with different few-shot relationships.This paper call these entities as "solitary entities".Existing research over-relies on the semantic neighbors of entities in the knowledge graph to represent entities in the "entity representation learning enhancement" stage,which limits the robustness of solitary entity representation and restricts the model’s performance on large-scale knowledge graphs.To alleviate the negative impact of sparsity and diversity,this paper extends the idea of entity interaction information and introduces four abstract semantic relations independent of specific knowledge graphs to describe the correlation between entities in the few-shot link prediction task and construct task semantic pattern graphs that add dynamic neighbor nodes from the few-shot task to solitary entities.On this basis,this paper proposes the use of a multi-relation graph neural networks with highway connection mechanisms to perform representation learning on the message propagation mechanism in the task semantic pattern graph,and enriches the semantic representation of entities using features from dynamic neighbor nodes.Furthermore,to address the diversity of knowledge graphs,a dynamic gating mechanism based on signals from query is proposed,which gives the model the ability to dynamically integrate semantic information from both task semantic pattern graphs and knowledge graph topological structures based on different queries.Experiments show that the proposed method achieves better performance results than existing models on sparse knowledge graphs Wiki-One,and the validity and effectiveness of the four abstract semantic relations are further verified through ablation experiments.Third,a few-shot temporal link prediction method based on entity temporal neighbors and relationship evolution patterns.Temporal knowledge graphs use timestamps to record the valid time of each fact,giving knowledge graphs the ability to represent dynamically changing facts and solve problem of contradictory facts caused by missing time information in traditional knowledge graphs.Currently,there are still a large gap in research on few-shot temporal knowledge graph link prediction tasks,mainly in two core issues.First,how to model the semantic information provided by temporal neighbors in the "entity representation learning enhancement" stage.Second,how to mine the implicit evolution pattern information among known instances of few-shot relations in the "relation representation learning" stage under multi-instance scenarios,which has not been effectively addressed and discussed in existing studies.To address the first issue,this paper devises a novel twin graph attention network to measure the contribution of high-frequency and ever-changing facts to entity representation separately and adopts a triplets’ head-tail entities interaction based gate mechanism to dynamically integrate semantic information from high-frequency and ever-changing neighbor sets.To address the second issue,this paper first designs a time difference-sensitive entity pair representation method to learn representations of temporal facts,and then proposes an instance-level evolution pattern learning network based on gated recurrent unit(GRU)networks for known instances of few-shot relations to mine evolution patterns among instances.Finally,a semantic information evaluation network is used to complete the link prediction.Experiments show that the proposed model achieves the best results in two real-world few-shot temporal knowledge graph datasets,ICEWSfew-intp and GDELT-few-intp,validating the effectiveness of this method.