Chinese Machine Reading Comprehension Based On Improved R-NET

The goal of natural language processing is to enable computers to process text on the basis of understanding its inherent meaning.Machine reading comprehension refers to a machine that answers questions based on the content of a given article,which is similar to human reading comprehension tests.In recent years,with the development of deep learning and the rise of artificial intelligence,machine reading comprehension,as a representative of intellectualization in the field of natural language processing,has attracted extensive attention from industry and scholars at home and abroad.The development of Chinese machine reading comprehension lags behind that of foreign countries because of its late start.The appearance of CMRC 2018 data set fills in the blank of Chinese span-extraction machine reading comprehension data set.Therefore,this thesis attempts to apply the R-NET model,which is excellent in the English segment extraction machine reading comprehension data set,to the CMRC 2018 Chinese data set and get the Chinese machine reading comprehension benchmark model R-NET(Skip-gram).The objective is to improve the EM value and F1 value of R-NET(Skip-gram)on CMRC 2018 data set.Firstly,the thesis introduces the research background,significance and research status at home and abroad.,and explains the related concepts used in R-NET model,including recurrent neural network,word vector,attention mechanism and Pointer Network.Next,the SQuAD data set used by R-NET and CMRC 2018 data set used in this thesis,as well as their same evaluation of EM and F1,are described respectively.After introducing the four layers of R-NET in detail,R-NET is applied to CMRC 2018 data set.The Skip-gram are selected after comparison,and the benchmark model R-NET(Skip-gram)is obtained.Then,on the basis of R-NET(Skip-gram),the thesis proposes three improvements from word vector,gate function and check layer: 1.Directional Skip-gram word vector is introduced into R-NET model for the first time to generate R-NET(Directional Skip-gram),which strengthens the feature extraction of articles and problems.Directional Skip-gram strengthens feature extraction of articles and questions,and increases EM and F1 values on dev set by 2.050% and 1.518% respectively;2.Try to change the gate function of R-NET(Skip-gram)to tanh and get R-NET(tanh).As gate function,tanh strengthens the negative eigenvalues and improves the circulation of useful information among models.The EM and F1 values on dev set are increased by 2.795% and 2.895% respectively;3.Referring to human's habit of checking when doing reading comprehension problems,the thesis adds check layer to R-NET(Skip-gram)and obtains R-NET(check layer).The experimental results show that the check layer emphasizes the importance of problem features and compensates for the loss of problem features in the process of information transmission.The EM and F1 values on dev set are increased by 1.926% and 1.833%,respectively.At the same time,the check layer increases the complexity of the model and deepens the over-fitting.Finally,the P-Reader model is proposed by combining R-NET(Directional Skip-gram),R-NET(tanh)and R-NET(check layer).Compared with the R-NET(Skip-gram)benchmark model,the model complexity of P-Reader is higher,but the EM and F1 values on dev set are increased by 6.337% and 5.126%,respectively.In order to overcome the over-fitting caused by the small training set of CMRC 2018,the processed DRCD data set is added to the training set,which further improves the performance of P-Reader.And on the Open Challenge Leaderboard,P-Reader ranked third in the test set(single model)and first in the challenge set(single model).In order to show the effect of P-Reader more intuitively,a simple simulation interface is also designed in this thesis.