Research And Application Of Open Domain Oriented Knowledge Graph Representation Learning

Knowledge Representation Learning(KRL)aims to project entities and relations in Knowledge Graphs(KGs)to densely distributed embedding space and learns the representation of entities and relations in low-dimension.KRL improves the computation efficiency and mitigates the data-sparse issue to some extent.These features enable KRL to be applied in real-time computing and inferring scenarios.KRL has become one of the most concerning fields in Artificial Intelligence and exerts more and more important roles in many downstream applications such as Semantic Search,Recommender System,and Knowledge-based Question-answering System.Recently,many KRL models have been proposed to improve the performance of downstream applications,but the challenges still remain in existing works: 1)existing works hardly handle the complex relation issue effectively; 2)most of the existing works can not model the zero-shot entities,which leads to lots of zero-shot entities failing to be linked to the existing knowledge base; 3)when facing the multi-source information fusion problem,most existing works only exploit the structured information and neglect a large amount of description information of entities.To address these issues,the thesis studies the complex relation modeling and open-world knowledge representation learning and proposes two models,including:1.April E: Attention with pseudo residual connection embedding.This proposed model aims to model complex relational patterns such as symmetric and anti-symmetric relations.There are two main components in April E: triple-level self-attention mechanism and pseudo residual connection.Firstly,it combines the head entity,relation entity,and tail entity in a triple as a sequence and applies the triple-level self-attention to capture the intrinsic dependency within triples.Then,in order to learn deep semantic features while preserving useful shallow semantic features,the proposed pseudo residual connection connects the product of triple-level self-attention and the pseudo original representation to form the final representation of entities and relations.To overcome the limitation of symmetric and anti-symmetric problems of existing works,two schemas were proposed in April E to handle the symmetric and anti-symmetric patterns respectively through different combinations of representations of different parts of entities and relations.In a word,April E can learn rich semantic information within a triple and can improve the performance of modeling the symmetric and anti-symmetric relations.The experimental results on the widely-used benchmark datasets outperform most baseline models,which illuminates that April E not only can process symmetric and anti-symmetric relations effectively but also can handle 1-to-N,N-to-1,N-to-N complex relations.2.ZeroE: Zero-shot entity-based open-world embedding.This model is proposed to handle the zero-shot entity representation problem.Zero E consists of a structured knowledge embedding layer,entity description encoding layer,and distribution alignment between entity embedding and entity description representation layer and adopts a three-stage optimization strategy.Firstly,in the structured knowledge embedding layer,it employs the widely-used backbone models to learn the embedding of entities and relations.Then,in the entity description encoding layer,it applies BERT to encode the entity description and fine-tune the representation.Next,in the distribution alignment layer,the KullbackLeibler Divergence is exploited to align the entity description representation to the entity embedding.To learn the representation of zero-shot entities effectively,a three-stage optimization strategy is adopted.In the first stage,it applies entity-based structured learning to learn the embedding of entities and relations.In the second stage,Zero E combines the description-based and entity-based structured learning to jointly learn the representation of entities and relations and applies the Kullback-Leibler Divergence to align the description-based representation to the entity-based representation.In the third stage,to further improve the description-based learning,the model continues to be trained only using the description-based learning.In the inferring phase,for the zero-shot entities,their representations are derived from the description-based encoder.This design enables Zero E to solve the zero-shot entity learning problem.In addition,Zero E can make full use of the description information to enhance the structured representation of entities and relations in KG.The experimental results show that Zero E consistently achieved better performance than most baseline models in open-world datasets.This evidence illuminates that Zero E is competent in modeling zero-shot entities and alleviates the link prediction problem of zero-shot entities and shows the application values in the knowledge graph completion task.