Research On The Method Of Lifelong Relation Extraction

Relation extraction aims to identify the relational facts of pairs of entities in text,and is widely used in the field of knowledge graph construction and natural language processing.Compared with the traditional methods that focus on artificial design features,the relation extraction methods based on deep learning have achieved remarkable results.However,the existing relation extraction methods usually assume a closed set of relations,without considering the dynamic change of requirements,so they are not suitable for practical application scenarios.This leads to the study of lifelong relation extraction,that is,the set of relations that need to be predicted can be changed or enlarged over time,and we cannot revisit all previous data at each stage.This article will study how to extract relations in the context of lifelong learning.The main difficulty is that the neural model will suffer catastrophic forgetting,that is,the model will often encounter a significant drop in the performance of the old task when learning a new task.To this end,different methods were proposed to reduce forgetting,which can be roughly divided into three categories: the first is to use regularization terms to prevent sharp changes in important parameter values;the second is to use memory modules to store a small amount of old task data which is trained with new tasks for experience replay;the third is to adopt a dynamic architecture to learn knowledge by retaining the original network structure and adding layers or nodes.These methods have achieved considerable performance improvements on simple image classification data sets,but it turns out that they perform poorly on natural language processing tasks.In the current research,there are only a few literature discussing the method of lifelong relation extraction.Through the analysis and summary of the existing methods,it is found that there are still three problems in the existing research: first,adopting the dynamic architecture or adding the new network layer for training will introduce more parameters into the over-parameterized relation extraction model,resulting in the doubling of the supervision signals,memory and computing resources required by the model.The second is that when saving a small number of old task samples for learning,the phenomenon of over-fitting to these samples is ignored.The third is that the regularization methods use fixed strength and lack flexibility,so it is difficult to achieve a good balance between new task learning and old task preservation.In view of the above three issues,this paper proposes a lifelong relation extraction model based on dynamic regularization(DR-EMR)and a lifelong relation extraction model(FT-MR and LM-MR)that integrates category differentiation.The main work of this paper is as follows:First,we propose a dynamic regularization method for lifelong relation extraction.In this method,relation extraction is modeled as a sentence relation matching problem,the input is a sentence relation pair,and the output is the corresponding matching score.In order to retain the learned knowledge,two regularization terms are used,and a dynamic balance strategy is designed to adaptively adjust the strength of regularizers with respect to the change of training losses,so as to achieve a good trade-off between current task learning and old knowledge preservation.Better results are achieved without introducing more parameters.Second,we propose a method for lifelong relation extraction that integrates category differentiation.We adjust the training process of the model,first learn new tasks,and then use the memory data to train the model.The sample distribution of each task in the memory is balanced,which helps the model learn new and old knowledge and alleviate the problem of over-fitting.In order to maintain the distinction between relation categories,we propose two methods: one is to fine-tune the model in the last stage to adapt to the goal of category distinction;the other is to use the label mapping algorithm to generate the representation vector with a certain degree of differentiation for each relation before training.The paper conducts experiments on two public datasets.Experimental results show that,compared with the most advanced method,the average accuracy of this model on the Few Rel dataset and Simple Questions dataset has increased by 4.9% and 0.3%,and the overall accuracy has increased by 1.4% and 2.8%.Experimental results show that the proposed models are effective for the task of lifelong relation extraction.