Research On Technologies Of Document-Level Relation Extraction

Natural language processing is widely used in various knowledge-based scenarios,such as question-answering systems,machine translation,and knowledge graphs.As a prerequisite and foundation for knowledge generation in natural language processing,document-level relation extraction has attracted increasing attention from scholars.Currently,experts and scholars at home and abroad have to some extent improved the performance of document-level relation extraction by using graph structures or pre-training models.However,factors such as the fuzzy semantics and complex contexts of documents themselves,as well as the reduced dependency between entities caused by the distribution of entities among different sentences,seriously affect the performance of document-level relation extraction.Therefore,this dissertation focuses on the two key issues of local semantic dependencies of entities and global contextual dependencies of entities in document-level relation extraction.It conducts in-depth research from four aspects:entity semantic representation,interaction between different entity pairs,utilization of document context,and long-distance dependencies between entity pairs.The main studies and contributions of this dissertation are:1.To address the problem of inaccurate characterization of entity semantic representation due to missing entity information in document-level relation extraction,a model that semantic-guided attention and adaptive gated is proposed.The model uses attention mechanism and multi-head attention mechanism to obtain sentence and document semantic representations through semantic guidance,and selectively outputs document semantic representations through the adaptive gating mechanism,forming the final representation of entity semantics.The model incorporates different levels of sentence and document semantics,effectively solving the problem of missing entity semantic representation information.Experimental results show significant improvements in different performance metrics on three public datasets(DocRED,CDR,and GDA).Compared with the strong baseline MRN model,the F1 value for intra-sentence relation extraction is improved by 2.50%,the F1 value for inter-sentence relation extraction is improved by 0.98%,and the F1 value for document-level relation extraction is improved by 1.03% on DocRED dev set.On CDR dev set,when the number of entity mentions exceeds four,the F1 value is 2.40% higher than that of the GLRE model.2.To address the problem of inaccurate characterization of the mutual relation between entity pairs due to the lack of information interaction between different entity pairs in document-level relation extraction,a model that dual attention fusion is proposed.The model uses co-attention mechanism and multi-head axial attention mechanism to obtain global and local features between entity pairs,and by integrating these two types of features,it forms a representation that can characterize the mutual relation between target entity pairs.The model integrates the global and local features of entity pairs into the representation of entity pair relation,effectively improving the relation representation of target entity pairs through interaction between entity pairs.Experimental results show significant improvements in different performance metrics on three public datasets.Compared with the typical ATLOP model,the F1 value for document-level relation extraction is improved by 1.80% on GDA dev set.100 epoches experiments were conducted on DocRED dev set,and the F1 value of dynamic asymmetric loss applied to the model was 1.3%-4.6% higher than that of binary cross entropy loss.3.To address the problem of inaccurate characterization of contextual information for target entities due to missing different types of contextual information in document-level relation extraction,a model that multi-perspective context aggregation is proposed.The model incorporates different levels of contextual information,such as adjacent entity node information,entity similarity information,document topic information,and distance information,into the entity representation to obtain a target entity representation that includes contextual information.By aggregating contextual information at different levels,the model enriches the contextual information of the target entity representation.Experimental results show strong competitiveness in different performance metrics on three public datasets.Compared with the classical SSAN model,the F1 value for document-level relation extraction is improved by 1.12% on CDR dev set.On DocRED dev set,when the input document length is between [401,500],the F1 value for document-level relation extraction is 0.70% higher than that of the ATLOP model.4.To address the problem of missing multi-granularity reasoning information due to long-distance dependencies between entity pairs in document-level relation extraction,a model that collaborative local-global reasoning network is proposed.The model incorporates local and global information by constructing mention-level graph and concept-level graph,and uses an independent graph approach to assist in creating the final entity-level graph.By combining local-global information with a mixed reasoning mechanism,the model achieves collaborative entity-related inference information.Through different granularity graph models,the model effectively integrates local and global information into the model reasoning.Experimental results show significant improvements in different performance metrics on three public datasets.Compared with the best-performing HAIN model on DocRED dev set,the Ign F1 value for document-level relation extraction is improved by 1.76%.In the logical reasoning type of the DocRED dev set,the F1 value for document-level relation extraction is 2.51% higher than that of the DRN model.