Research On Relation Extraction Based On Semantic Weight And Attention Network

As a sub-task of information extraction,relation extraction can discriminate the relationship between entity pairs in the given text,and plays a very important role in the field of natural language processing.It is time-consuming and laborious to rely solely on the manual establishment of relation extraction corpus,and with the help of distant supervision we can easily build large corpus,but the inevitable mislabeling errors bring serious data noise.In order to solve this problem,many methods have emerged,the most popular method is the attention network model with multi-instance learning.However,most of the current methods treat all relations independently,without taking advantage of correlations among relations and the semantic information in the definition of relations,which severely limits the overall performance of the model,especially the extraction performance of long-tail relations which lacked training data.This paper proposes an entity relation attention networks(ERAN)model for distant relation extraction task,aiming at the problem in the application of current attention mechanism that the lack of hierarchical structure and semantic information of relations in query vectors of bag level attention calculation,which makes the model unable to learn reasonable attention weight.The ERAN model uses bidirectional GRU(Gated Recurrent Unit,GRU)to encode relation definition to generate context representations of relations,and uses custom dot-product attention mechanism to calculate the attention score,which can make full use of all hidden states obtained by GRU.In this way,our ERAN model can learn how to distribute the attention weights reasonably.In addition,in order to enhance the ability to describe the relationship between entities and the text contexts of the model,the ERAN model also defines the length of the shortest path between entities and words in the dependency tree as the semantic distance.On this basis,the input vectors of the model are weighted with semantic weights to make input vectors more relevant to relations,which helps the model to reduce the influence of noise text segments unrelated to relations,and further improves the relation extraction effect of the model.Experiments show that our ERAN model outperforms previous state-of-art methods on the widely used New York Times relation extraction corpus,and the attention mechanism of ERAN model significantly improves the effect of extracting long-tail relations.The AUC(Area Under Curve,AUC)of the ERAN model reaches 0.460 by using the area of the PR curve AUC as evaluation index,which is 0.044 higher than the best RESIDE model recorded in the NLPprogress,and 0.037 higher than the PCNN+HATT model which also uses the definition of relations as the external supervision.