Research On Chinese Machine Reading Comprehension Based On Pre-trained Model

Machine reading comprehension is an extremely challenging task in the field of natural language processing,which aims to answer relevant questions given a given passage.And it has a wide range of applications in search engines,intelligent education,and other fields.Compared to the English language domain,Chinese reading comprehension has more ambiguity and polysemy,which leads to poor performance of pre-trained language models in the Chinese context and makes it difficult to learn deep semantic information.Currently,there are three problems with the application of pre-trained language models in the field of machine reading comprehension.First,in the pre-trained stage,the traditional paradigm of masked training leads to insufficient word coverage of the training data,making it difficult for the model to learn global contextual information.Second,pre-trained language models also have problems such as input length limitations and noise interference.Finally,ineffective semantic matching networks make it difficult to support pre-trained language models to exert more powerful performance,which can even lead to a decrease in performance.In response to these problems,the thesis mainly focuses on research on Chinese machine reading comprehension based on pre-trained models,and the main content is as follows:(1)We use the powerful context encoder Mac BERT to build a Chinese multi-choice machine reading comprehension model.Current research shows that machine reading comprehension models have difficulty capturing deeper contextual and synonym information.And it cannot capture the deep interactive information between passage,question,and options.Mac BERT,based on a multi-layer Transformer architecture,can capture contextual information of long-distance text.It also adopts a pre-trained method of synonym replacement,which effectively solves the problem of inconsistency between upstream and downstream tasks in traditional paradigms,making it more suitable for machine reading comprehension tasks.Finally,the correct answer is output through the multi-layer perceptron structure of the decoder.Compared with traditional models,Mac BERT demonstrates better performance.(2)An unsupervised passage-sentence quality evaluation method is proposed.The emergence of pre-trained language models has brought significant breakthroughs to the field of machine reading comprehension.However,pre-trained models with fixed input length limits cannot handle long text and may introduce noise.Our method uses an unsupervised algorithm to obtain the support degree of different passage-sentence pairs for a given question.Then choose the highest scoring sentences as inputs.This algorithm is computationally efficient and effectively extracts information.Finally,the generalization performance of the machine reading comprehension model is further improved on top of the original basis.And a system framework is constructed based on a passage-sentence quality evaluation.(3)We propose a Chinese multi-information co-matching model for machine reading comprehension.To effectively capture the interaction information among passage,question,and options,we design an efficient semantic matching network that leverages the context language model and exhibits stronger performance with the support of the semantic matching network.Firstly,based on the reading strategy of simulating human reading comprehension,bilinear matching interaction and gate-control fusion mechanism are employed to explore the contrastive information among options.Furthermore,we adopt a multi-head attention mechanism to consider the focus information from both the passage and the question.This approach improves the reasoning efficiency and better simulates the human answering process.Experimental results demonstrate the effectiveness of various learning strategies,and the final model shows the best performance.