Research On The Implicit Discourse Relation Classificatio Using Memory Encoding And Data Augmentation

Discourse relation recognition is a basic research task of natural language processing,which processes two adjacent arguments(such as clause,sentence or paragraph,collectively known as argument pairs)in batches and automatically determines the semantic relation between each argument pair.Discourse relation recognition provides one of the most important support for downstream tasks,such as reading comprehension and text summarization.Implicit discourse relation recognition is an subtask of discourse analysis.It is challenging to straight determine implicit relation types due to the lack of connectives.For example,although the connective word like "therefore" can serve as an explicit and disambiguous word-level clue for relation determination,the ommission of the word makes it hard to implement the determination.This dissertation concentrates on the issue of implicit relation classification,and makes great efforts to solve the problems of the utilization of implied relevant information between arguments,as well as the sparsity of observable temporal relational samples.To overcome the problems,we develop and verify the cross-memory encoding based implicit relation recognition model,as well as the optimization method of temporal relation identification using data augmentation.In details,this dissertation will discuss on the following three aspects,two of which are of research content,while the rest is regarding the system development:(1)Implicit Discourse Relation Recognition Combined with Cross Memory and Interactive AttentionExisting researches usually build complex neural network models to improve the performance of implicit discourse relation recognition,but predecessors usually only use the interactive information among arguments,or only focus on the key information of arguments themselves,and do not dig the information of arguments themselves and each other from a deep and multi-angle.To solve this problem,this dissertation proposes a method to simulate cross memory,which affects the encoding memory of one argument by understanding the semantics of another argument.In this dissertation,we implement the cross memory method by modifying the traditional BiLSTM model,and embed it into a set of multi-layer interactive neural network architecture.In this dissertation,we test this method on PDTB V2.0,and The experimental results show that this method achieves higher performance than The baseline model on different relation types,and shows competitive performance compared with The state-of-the-Art.(2)Data augmentation based implicit temporal relation recognitionTemporal relation is one of the four types of primary discourse relationships in PDTB.Recognizing temporal relations,i.e.,determining whether or not a pair of arguments holds the temporal relation(binary classification),is challenging.Nowadays,binary temporal relation classification achieves much worse performance than that for other three primary relation types.This results from the extreme sparsity of labeled temporally-related argument pairs(such samples occupy about 7%in all).In other word,the models most probably suffer from the low-resource scenario.In this scenario,the small number of labeled samples fail to direct the neural network models to obtain reliable feature perception capacity during the process of supervised learning.Therefore,this dissertation proposes and develops the data augmentation based implicit temporal relation classification model.It utilizes variable autoencoder(VAE)and contrastive learning mechanism(CL)to generate variants,i.e.,the ones which have consistent semantics with the original sample though dissimilar presentations.On the basis,it utilizes the variants to strengthen the learning capacity of neural models during training.Experimental results show that the proposed data augmentation method substantially improve the performance of the baseline model,a RoBERTa-based temporal relation classification model.(3)Implicit Discourse Relation Recognition SystemUsing the cross-memory based implicit discourse relation recognition model and data augmentation based temporality identification model respectively,we construct an online discourse analysis system.The system is deployed with the front-end framework Vue and Bootstrap,as well as the back-end framework Tornado builds the implicit discourse relation recognition model.The user enters the related pair of arguments,and the system returns the classification results between the two arguments based on the cross-memory and crossattention models.At the same time,when the user enters a text,the system will automatically identify the temporal relationship.This system demonstrates the research work in this dissertation and can also be used as an aid to other natural language processing works.