Causality Mining In Natural Language Text Using Deep Neural Networks

Causality mining is a challenging yet very crucial task in natural language understanding.It plays a significant role in several daily life applications such as decision-making,adverse drug reactions,business risk management,question answering,future event prediction,scenario generation,information retrieval,medical text mining,textual entailment,and discourse comprehension.Many studies have been conducted to handle this task in both formal datasets(data from books,journals,stories,and newspapers)and informal datasets(web corpora,video,images,and audios),beginning with traditional and machine learning(ML)feature-based methods and progressing to the current deep learning(DL)approaches.The majority of ML approaches use automated techniques,and features are created through sophisticated feature engineering using large corpora with label datasets.This automatically mines explicit causality and ignores complex implicit and ambiguous causality.Deep learning has been successfully applied to causality mining using automatic feature engineering techniques,and it has performed well in showcasing hidden causal knowledge in source sentences.However,previous studies have relied on rule-based and machine-based techniques,which are typically inexpensive in terms of cost and performance.Despite significant progress and accomplishments,current approaches are ineffective due to the following issues: First,the majority of machine learning and deep learning attempts are automated and concentrate on explicit causality,neglecting the implicit and ambiguous causality,which is usually found in web corpora.Second,they aim to determine whether sentences are causal or not and rarely focus on extracting causal relationships and their directions.Third,most approaches are domainspecific and rely on small datasets while ignoring informal web corpora.Fourth,most approaches lack multi-level feature networks,which are necessary for effective relational reasoning.As a result,these methods may lead to performance degradation.In order to address the challenge of extracting implicit and ambiguous causality from large domain-independent web corpora,this dissertation employs sophisticated approaches to learning rich contextual and semantic information using efficient multi-level deep neural networks,contextual word extension method,and multi-channel knowledge-oriented deep neural network.The primary goal of this thesis is to solve the problem of massive implicit,ambiguous,and domain-independent causality extraction from a web corpus.Web corpora can contain a variety of data sources,such as online drivers,sensors,online transactions,phones,data stores,social media platforms,and cloud computing services.The main work and contributions of this thesis are as follows:To address the issues that simple traditional,machine learning,and single-level deep neural approaches are unable to mine a large number of implicit,ambiguous,and domainindependent causal relationships in web corpus.We propose a novel deep multi-feature BERT+MFN(Bi-directional Encoder Representation from Transformers + Multi-level Feature Network)model to address the problem of implicit causality at token(word)and event(segment)levels without feature engineering.To overcome the limitations of tokenlevel feature engineering,we use BERT to integrate basic features to capture long-range dependencies and local context in text.This leads to word-level semantic representations.To handle segment-level features,we use a unique MFN(TC-KN,bi-LSTM,and RN)to collect segment-level basic features and employ a novel approach to automatically build novel convolutional word filter from the knowledge bases instead of a trained typical convolutional filter.BERT+MFN has an efficient inference capability to identify causal relationships.Experimental results on publicly available datasets show that the proposed model outperforms the baseline approach on various evaluation metrics.Causality mining has been a challenging task for many traditional ML and DL methods because,in the absence of any explicit signals,causality is usually existing in an informal,implicit,and ambiguous form.To mine such causal relationships,multi-level DL networks are required.In this paper,we propose a unique approach called CMCE+BK(Causality Mining by Candidate Event Extension and Prior Background Knowledge),which combines two modules for efficient causality mining.These modules concentrate to mine extended segment/event and connective level elements in source sentences,where extended segments and connectives can be characterized using causal terms extracted from contextual vocabulary to describe the extended nature of candidate segments and connectives.The use of the a priori BK(Background Knowledge)module further strengthens the key features of causality at the segment and connective level,thus improving the model ability to perceive the BK of causal links in a sentence.Experiments and ablation studies in the publicly accessible Alt Lexes corpus have shown that CMCE+BK requires a certain amount of contextual word expansion and multi-level background knowledge to achieve optimal performance over state-of-the-art causal and text mining techniques.To address the problem of implicit causality in web corpora,we propose an innovative MCKN(Multi-Column Knowledge Oriented Neural Network)model.The proposed MCKN is unique in that it uses multiple KCs(Knowledge-oriented Channels)and a novel word filtering technique for the first time,which significantly reduces the dimensionality of the model.This is the first attempt to target segmental and connective level features in the target sentence using convolutional "wf" for all channels instead of predefined convolutional filters.The proposed work is unique in that it utilizes a web corpus to target causality in sentences,which includes noisy and domain-independent data.Experimental results on the AltLexes(Alternative Lexicalization)dataset show that MCKN achieved the best performance on different metrics.