Research On Deep Recurrent Neural Network For Relation Extraction

Information extraction mainly studies how to extract useful information from massive data effectively and automatically,which has great significance in knowledge service.As one of the important components of information extraction,relation extraction has always been a research hotspot in the field of natural language processing.The main purpose of the relation extraction task is to convert semi-structured or unstructured natural language text into structured text,and then find the semantic relation between the two identified entities in a sentence,which is the basis of intelligent applications such as knowledge map and question answering system.In the past,most of the research on relation extraction was based on traditional machine learning methods,which usually relied on artificially produced features.However,feature engineering is a labor-intensive task that requires a lot of manpower and time.With deep learning methods are successfully applied in the field of natural language processing,a large number of scholars began to use deep learning methods to study the relation extraction task,which weakened the need of extracting features artificially.However,these methods have two problems: one is ignoring the interaction information between entities and contexts;the other is that the long-distance context dependency cannot be well obtained.Focusing on these issues,this paper is based on long short time memory networks for relation extraction,and conducts research from the following three aspects:(1)Aiming at the problem of "ignoring the interaction information between entities and contexts",a model named entity-dependent long-short time memory network for relation extraction is proposed.Firstly,in order to save the contexts information around entities,this paper uses two bidirectional long-short term memory networks to encode the context text in the front and rear directions into their semantic representations.Then,the relevance between entities and their contexts is modeled through the idea of entity dependency,and the semantic relation between entities is inferred by selecting relevant parts of contexts.Finally,this paper uses the SemEval-2010 Task 8 dataset to train this model and selects eight better relation extraction methods are compared with this model.The experimental results show that the comprehensive evaluation index F1 value of this model is 85.6%,which is about 0.5% to 6.8% higher than other methods,effectively improving the performance of entity relation extraction.(2)Aiming at the problem of "not able to obtain long-distance context dependencies",a long-short term memory network model based on self-attention for relation extraction is proposed.The neural networks currently used to extract the relations between entities,such as the convolutional neural network and the recurrent neural networks,do not well obtain long-distance context dependencies.This paper believes that the full use of long-distance context dependencies will help models to extract the semantic relation of entities more accurately.In order to solve this problem,this paper proposes a model named long-short term memory network based on self-attention for relation extraction,which can learn the potential dependence information between each word,and can analyze the relevant contexts more comprehensively and get more useful information.This model first uses a bidirectional long-short term memory network to encode entities and contexts as these feature representations,then inputs the resulting representation matrix into the self-attention module to obtain a multi-layered attention representation of entities and contexts,and finally joins a classification layer classify semantic relation.The comparative experiments with the above eight relation extraction methods show that this model can obtain 85.2% F1 value,which is about 0.1% to 6.4% higher than other methods and better solves the problem that the existing relation extraction models cannot fully obtain long-distance context dependencies.(3)In order to further explore a method of improving the accuracy of semantic relation classification,this paper combines the entity-dependent long-short term memory network with the self-attention mechanism and proposes a model named joint self-attention entity-dependent long-short term memory network for relation extraction.This model combines the advantages of the two models mentioned above,so that the interaction information between entities and relevant contexts as well as the long-distance context dependencies can be used together as the basis for the classification of entity relations,so as to make full use of the contexts information around entities.This paper uses the SemEval-2010 Task 8 dataset as the training data,and compares the results with other existing eight excellent models and the above two models.The experimental results show that this model can better combines the entity-dependent long-short term memory network and self-attention mechanism,and its F1 value reaches 85.9%,which further improves the accuracy of relation extraction.