Research On Key Issues In Machine Reading Comprehension Models

AI constantly brings about a great upsurge in several fields,among the various fields of AI,NLP(Natural Language Processing)requires AI capable of processing,understanding and utilization of human language,which can be treated as the true sense of ‘artificial intelligence',thus is significant for AI.However,machine reading comprehension(MRC)is a key step to realize language intelligence,which mainly involves fields like deep learning,NLP and information retrieval etc.Through which,the computer can help human to find the desired answer among massive information,while saving time and costs.In this paper,we conduct a research on key issues in MRC models from three aspects: exploring the modeling capacity of the existing MRC models,improvements on incorporating external knowledge into DL(deep learning)models and DL models' own framework on attention mechanism,and improvements from two aspects: answer pre-filtering and model improving on the baseline system of the large-scale Chinese MRC dataset Du Reader.The main contributions of this paper are shown in the following aspects:Firstly,we conducted an analysis and comparison of the DL models of MRC.We firstly pick one perform well model from the two type of MRC models of making answer selection and answer span locating respectively,and we pick the QA-LSTM with attention model and Bi DAF model.After that,we give a detailed introduction to the model framework and the inner attention mechanism,then carry out comparison from several aspects.And finally,we conduct an experiment to evaluate and analysis the performance of the two models.Secondly,improvements to the DL models of MRC in two aspects.On the one hand,we incorporate prior knowledge into DL model.We firstly comb language knowledge of various aspects,such as morphology,syntax and semantics,then give the exploration of ways to incorporate prior knowledge into various network layers in DL framework.We pick the representative knowledge from each aspect,conduct the experiment of incorporating prior knowledge into DL framework.On the other hand,we try to do an exploration to the granularity of attention in MRC DL model.We tried to incorporate three more question to passage sentences attention into the Bi DAF model and made comprehensive analysis in experiments.Thirdly,based on the Chinese MRC dataset Du Reader,we made improvements from two aspects to the baseline system released by Bai Du: answer pre-filtering and model improving.Among which,answer pre-filtering consist of two parts: answer paragraphs selecting and answer span locating.For improving on answer pre-filtering,we try to select answer paragraphs and locate answer span globally,and select answer paragraphs based on logistic regression.As for DL model,we add self-attention mechanism to the Bi DAF,and update the training objective as shared-normalization form.And finally,we get a promotion at BLEU-4 by 6.25,and ROUGE-L by 4.79.