Event Reasoning Based On Causalit

Causality inference has attracted extensive attention in the field of artificial intelligence in recent years,and has become one of the hot issues in natural language processing.The research of causal inference between events in natural language texts mainly consists of three parts.The first part is the causal relation identification and extraction from text data,which aims at forming causal relation entities in semantics.The second part is causal inference between events in natural language texts,which aims at generalizing specific causal relations and inferring unseen causal relations based on the existing ones.The third part is the application based on causal relations in natural language texts,which aims at benefiting prediction by making use of causal relations.It remains a challenge to infer causal relation between events in natural language texts.Many scholars have applied causal inference to dialogue generation,QA system,common sense inference and other tasks.However,these existing research is mostly limited to causal relationships in certain specific events,lacking of general causal laws,resulting in insufficient generalization ability of causal relationships.It is also hard to benefiting other applications based on generalized causal relations from these studies.In addition,there are also certain difficulties in integrating causal reasoning knowledge from different forms in the text to enhance causal reasoning ability.Therefore,based on a comprehensive analysis of existing literature and deep learning methods,this thesis covers three aspects:the identification and extraction of causal relationships in text,causal reasoning between events,and the application of causal relationships.The work and innovation of this thesis are as follows:(1)An experimental framework of causal networks constructing and representation learning is proposed.This framework consists of three parts,namely causal relation extraction and generalization,causal network representation learning models,and a causal evaluation framework.It is aimed to assist researchers in better evaluating the performance of different causal network models or algorithms and provide improvement suggestions.Different methods of causal relation extraction and generalization,as well as causal network representation learning models,also impact the construction of causal networks and the accuracy of downstream task.(2)An event inference model based on attention mechanism is put forward.In the stage of attention,the predicates are entered in order that the multi-head attention mechanism based on Bert model focus on the causal event features,and the feature vectors generated by the generalized causal network are used as the event embedding vectors,which can capture the semantic of verb phrases and their tenses in the causal network,improve the accuracy of event reasoning and achieve the accuracy close to SOTA on the open event inference dataset MCNC.(3)An experimental framework of language models for event inference is proposed.Previous experiments have shown that the language model using large-scale unsupervised corpus,captures a large number of semantics in the pre-training stage.This thesis focuses on fine-tuning language models based on causal relationships,common sense knowledge,etc.In the experimental process,multiple language models(BERT,RoBERTa,ALBERT)were fine-tuned separately,and the fine-tuned language models were then used for script event reasoning tasks.The experiments demonstrated that this approach achieved higher accuracy than SOTA.(4)An enhanced language model based on generalized causal network is put forward.The model adjusts the pre-training mode of the language model,and integrates the generalized casual network knowledge in the pre-training stage of the language model.The advantages of feature-based model and language model complement each other and can be maximized.Furthermore,this paper applies the model to the stock forecasting experiment.Compared with the pre-training model without fusion of event reasoning and common sense information,this method improves the accuracy of event prediction to a certain extent.This study shows that,according to the characteristics of causality in text,comprehensive use and improvement of natural language processing technologies such as text classification,causality correlation,attention mechanism and pre-training language model can improve the accuracy of causal inference;the causality network constructed can improve the effect of event inference task.