Distant Supervision Neural Network Relation Extraction Based On Noisy Observation

The traditional relation extraction needs a lot of human and material resources to label data,and the researchers are afraid of the high cost.So distance supervision relation extraction method emerges as the times require.Its great advantage is to generate labeled data automatically through knowledge base and natural language text.This simple automatic alignment mechanism liberates people from heavy labeling work,but meanwhile inevitably produces various incorrect labeled data,which would have an influential effect on the construction of high-quality relation extraction models.This paper presents two models to handle noise labels in the distant supervision relation extraction.On the one hand,the prior information of noise distribution is not taken into account in the existing distance supervision relation extraction methods.For this problem,we assume that the final label of sentence is based on noisy observations generated by some unknown factors.Based on this assumption,a new relation extraction model is constructed,which consists of encoder layer,attention based on noise distribution layer,real label output layer and noisy observation layer.First,we enrich the semantic information of the model through part of speech tagging,so as to avoid the impact of ambiguity caused by indiscriminate polysemy on the performance of the model.Second,in the training phase,transformation probabilities are learned from real label to noisy label by using automatically labeled data,and in the testing phase,we obtain the real label through the real label output layer.Finally,we research to combine the noise observation model with deep neural network.We focus on the attention mechanism of noise distribution based on deep neural network,and denoise the unbalanced samples under the framework of deep neural network,aiming to further improve the performance of distant supervision relation extraction based on noisy observation.On the other hand,the influence of the inherent noise robustness of the loss function on the task of distance supervision relation extraction is not considered.In this paper,according to the inherent characteristics of the loss function to alleviate the problem of noise tag.First of all,we analyze the anti noise and precision of cross entropy in the existing distance supervision relation extraction model.The analysis results show that the model using cross entropy loss function has higher precision but is more sensitive to noise.We also analyze the mean absolute error loss function in the neural network model.The model using the mean absolute error loss function has slower convergence and lower precision but stronger noise robustness.In order to maintain the accuracy and improve the noise robustness of the model,this paper uses negative Box-Cox transform to combine the advantages of the two loss functions as much as possible,and constructs a noise robust distance supervision relation extraction model,which has further improved the relation extraction result.The main contributions of this paper are as follows:(1)Propose the CBOW model based on semantic features,which is used to extract the semantic features of the model input from the perspective of polysemy,so that the model is more robust;(2)Considering the prior information of the noise distribution in the data,which is used to model the probability of the real label transformed to noise label.And we reduce the noise locally under the imbalance distribution of the data;(3)For the first time,the inherent noise robustness of the loss function is considered in the task of distance supervision relation extraction.The negative box Cox is introduced,which combines the advantages of the average absolute error loss function and the cross entropy loss function.Through the inherent anti noise property of the loss function,the performance of distance supervision relation extraction is improved.This paper experiment the noisy observation model under the public data set and the same parameter settings.By analyzing the distribution of sample noise,the performance of the noisy observation model under various sample noise distribution is evaluated and compared with the existing baseline methods.At the same time,under the same data set and parameter settings,the performance of the noise robustness model is analyzed and compared.The results show that the noise observation model proposed in this paper is about 6% higher than the latest baseline,and the noise robust model is about 5% higher than the classic baseline.