Research On Deep Continual Relation Extraction Algorithms

The relation extraction algorithms based on deep learning show high performance,but the existing algorithms usually assume that the relations to be predicted are a fixed set.After training the model with a fixed dataset,the model is used for prediction.However,in real applications,new samples and relations often appear continuously,resulting in the existing relation extraction algorithms facing the problem of catastrophic forgetting which means that the model can only identify the relations contained in the learning samples and forget the knowledge learned from the previous tasks,which eventually leads to the serious decline of the performance of the model on previous data.Therefore,how to improve the practicability of relation extraction algorithms has become an urgent problem to be solved.To solve this problem,researchers begin to study on continual relation extraction and propose some effective methods.But these algorithms have the following problems: 1)Most of the existing algorithms use replay mechanism,but the selection of old samples does not consider the similarity between the old and new relations,resulting in the low effectiveness of replay.2)In the continual learning scenario,the classifier tends to classify the old and new samples into the new relations,which has not been solved by the existing methods.3)The existing prototype learning has the problem of error accumulation when calculating the relation prototype.4)The existing algorithms are based on supervised scenario and need many labeled data for training,which limits the practicability of these algorithms.In order to solve the above faults,the author deeply studies the existing relation extraction algorithms and continual learning algorithms,fully analyzes the existing continual relation extraction algorithms,and proposes a more effective continual supervised relation extraction algorithm and an efficient continual unsupervised relation extraction algorithm.The main work of this paper is as follows:1)This paper proposes a continual supervised relation extraction algorithm.Specifically,firstly,aiming at the problem that the existing algorithms don’t consider the similarity between the old and new relations during replay,a sample selector based on the sentence’s semantic similarity is proposed to choose old samples,which makes the old relations that are more disturbed in the learning process of new relations have a greater proportion of replay.Then,learning and memory activation stage is proposed,which uses knowledge alignment to activate the model’s memory on the old relations.Specifically,model alignment is used to reduce the forgetting of model on old relations while learning new relations,and weight alignment is proposed to solve the trouble that the classifier tends to classify the old and new data into the new relations.In addition,aiming at the problem of error accumulation in the existing prototype learning,an iterative method to calculate the relation prototype is proposed,which retains the feature of the real relation prototype to the greatest extent and avoids the error accumulation of the relation prototype.Based on the above ideas,a more effective continual supervised relation extraction algorithm DRKA is proposed.2)In view of the poor practicability of the algorithm due to the large amount of labeled data in supervised scenes,this paper introduces continual learning into the field of unsupervised relation extraction for the first time.Firstly,the regularization strategy is used to reduce the forgetting of the model.Specifically,add a regularization term to the loss function by calculating the importance of the parameters to every task,and slow down the update speed of the parameters important to previous tasks by this regularization during the learning of new task.Then,due to the problem of accumulation of minor update of important parameters in the regularization stage,memory consolidation stage is proposed,which further consolidates the memory of the model on relations of previous tasks.At the same time,the sample collector is proposed to obtain and expand the old samples,so as to decrease the forgetting of the model and reduce the possibility of overfitting.Based on the above ideas,an efficient continual unsupervised relation extraction algorithm CURE is proposed.3)Large quantities of experiments are carried out on multiple common datasets to evaluate the effectiveness of the two algorithms proposed in this paper and compare with the latest relevant algorithms.The experimental results show that the algorithms proposed in this paper outperform the state-of-the-art algorithms.Although the two algorithms can effectively alleviate the forgetting of the model,the model adopted by the first algorithm is relatively simple,which leads to the poor expression ability of the model,and the stability of the training process of the second algorithm needs to be further improve.Therefore,how to extract continual relation on the model with strong expression ability and improve the stability of the training process of continual unsupervised relation extraction are the future research directions of this paper.