Research On The Distantly Supervised Relation Extraction On Deep Learning

The purpose of relation extraction is to extract the relationships between entities from unstructured text,so that they can be used for downstream natural language processing tasks.Based on whether the labels of the training datasets are manually annotated,relation extraction methods can be divided into two major categories: supervised and distant supervised.(1)The supervised relation extraction methods typically require a large annotated datasets for model training in order to automatically identify and extract relationships between entities.The supervised methods have higher accuracy and more reliable model results.However,because the datasets requires manual annotation,it can be costly in terms of labor,money,and time.(2)Distantly Supervised Relation Extraction(DSRE)combines the advantages of semi-supervised and unsupervised learning.It uses existing structured data to automatically label the datasets for model training,thereby reducing the burden of manual datasets construction,lowering costs,and facilitating automation.Although distantly supervised relation extraction can automatically generate a large number of labeled training samples,the automatic labeling process inevitably introduces a certain amount of noisy data,and the generated data suffer from mislabeling problems;in addition,The commonly used public datasets for DSRE tasks,NYT10,contains a long-tail phenomenon where there is a relatively small proportion of examples for certain relationships.This phenomenon has a negative impact on the performance of the relationship extraction model.Therefore,how to reduce data annotation errors and alleviate the long-tail distribution of data are two main problems faced by distantly supervised relation extraction.To address the above problems,the following two works are carried out in this thesis.(1)This thesis proposes a Dual Attention Mechanism with Entity-Aware Enhancement(EA-DAM)for distantly supervised relation extraction models,which addresses the issues of ignoring entity embedding representations and the inability of single-sentence bags to effectively distinguish noisy instance data.(1)The input layer obtains the initialized embedding vectors of words,locations and entities by Glo Ve algorithm.(2)The entity-aware representation layer integrates the embedding vectors using a position gate.(3)The encoding layer obtains sentence feature representations through Piecewise Convolutional Neural Networks(PCNN).(4)The dual attention layer uses a selective attention mechanism to obtain feature representations of sentence bags and uses an attention mechanism to obtain grouplevel representations from package-level combinations.(5)The softmax function is used to calculate the relationship classification results.This thesis evaluates the proposed EA-DAM model on the public datasets NYT10 and achieves better performance on the P-R curve than the baseline model,with a 1.4% lead in the AUC metric.(2)To address the problem that existing distantly supervised relation extraction models ignore the interaction and long-tail distribution at different levels,this thesis proposes a distantly supervised relation extraction model(Multi-Level Interactive Attention,MLIA)based on multi-level interactive attention.(1)The sentence representation layer gets the hidden representation of the input text by BERT pre-trained language models,and the embedded representation of the sentence is calculated from the hidden representation of head and tail entities and “CLS” tags.(2)The packet representation layer uses the selective attention mechanism to generate the packet representation,and finally the relationship type of the packet is predicted to get the loss of DSRE task.(3)The multi-level interaction layer uses the multi-headed self-attention mechanism to fuse the interactions between entity-level,sentencelevel and packet-level to obtain the loss of each level separately.(4)The output layer uses the sum of the loss of the DSRE task and the loss of the three levels to finally obtain the total loss of the objective function to evaluate the model performance.Experimentally evaluated on the publicly available datasets NYT10,the MLIA model proposed in this thesis achieves better performance than the benchmark model on the P-R curve and a 2.1% lead on the AUC metric.