Research On Machine Reading Comprehension Based On Deep Learning

Machine Reading Comprehension(MRC)is a subtask of Natural Language Processing(NLP).It aims to allow machines to read,understand text and answer questions.It is one of the most difficult challenges in Natural Language Processing(NLP)and Artificial Intelligence(AI).Recently,with the release of large-scale MRC datasets and the rapid development of Deep Learning(DL),research on MRC has made remarkable progress.However,there are still many issues to address.Our thesis focuses on the research of MRC based on deep learning.The main works are as follows:First,we propose a novel hierarchical attention model for extractive MRC,aiming to answer questions for a given passage.In our proposed DIM,attention and fusion are conducted full attention and bidirectional attention across layers at different levels of granularity between question and passage.Specifically,it first encodes the question and passage with fine-grained language embeddings,to better capture the respective representations at semantic level.Then it proposes a multi-granularity fusion approach to fully fuse information from both global and attended representations.Finally,it introduces a hierarchical attention network to focus on the answer span progressively with multi-level soft alignment.Extensive experiments on three large-scale extractive MRC datasets validate the effectiveness of the proposed method.Second,we address the multiple-choice Machine Reading Comprehension(MRC)problem.While existing approaches for MRC are usually designed for general cases,we specially develop a novel method for solving the multiple-choice MRC problem.We take the inspiration from Generative Adversarial Nets(GANs)and firstly propose an adversarial framework for multiple-choice oriented MRC,named McGAN.Specifically,our approach is designed as a generative adversarial network-based method that unifies both generative and discriminative MRC models.Working together,the generative model focuses on predicting relevant answers given a passage(text)and a question;the discriminative model focuses on predicting their relevancy given an answer-passage-question set.Based on the competition via adversarial training in a Minimize-Maximize game,the proposed method takes advantages from both models.To evaluate the performance,we tested our McGAN model on three well-known datasets for multiple-choice MRC.Our results show that McGAN can achieve a significant increase in accuracy comparing existing models based on all three datasets,and it consistently outperforms all tested baselines including the state-of-the-arts techniques.Third,there is growing interest in the tasks of financial text mining.Over the past few years,the progress of Natural Language Processing(NLP)based on deep learning advanced rapidly.Significant progress has been made with deep learning showing promising results on financial text mining models.However,as NLP models require large amounts of labeled training data,applying deep learning to financial text mining is often unsuccessful due to the lack of labeled training data in financial fields.To address this issue,we present FinBERT(BERT for Financial Text Mining)that is a domain specific language model pre-trained on large-scale financial corpora.In FinBERT,different from BERT,we construct six pre-training tasks covering more knowledge,simultaneously trained on general corpora and financial domain corpora,which can enable FinBERT model better to capture language knowledge and semantic information.The results show that our FinBERT outperforms all current state-of-the-art models.Extensive experimental results demonstrate the effectiveness and robustness of FinBERT.The source code and pre-trained models of FinBERT are available online.Fourth,it is hard to obtain large scale labeled data for MRC.To address the issues,we propose a novel Reinforcement Learning(RL)based Semantics-Reinforced architecture,named SiriQG,for QG task.In SiriQG,we propose a hierarchical attention fusion network,for better modeling of both answer information and passage information by integrating explicit syntactic constraints into attention mechanism,and for better understanding the internal structure of the passage and the connection between answer,which makes it better to fuse different levels of granularity(i.e.,passages and questions).Last,we also introduce a hybrid evaluator using a mixed objective that combines both RL loss and cross-entropy loss to ensure the generation of semantically and syntactically question.To evaluate the performance,we tested our SiriQG model on well-known datasets for QG.Extensive experimental results demonstrated that proposed SiriQG can obtain a significant increase in accuracy comparing existing models based on public dataset,and it consistently outperformed all tested baseline models including the state-of-the-arts(SOTA)techniques.