| With the development of information technology and Internet,the scale of text is developing explosively.The automation and intelligence brought by machine reading comprehension(MRC)coincide with the time and play a role in many fields,such as intelligent customer service,intelligent law,intelligent education,intelligent retrieval and so on.Machine reading comprehension is a technology that uses artificial intelligence algorithm to make computer understand text semantics and answer relevant questions.It gives computer the ability to read and analyze text.At the same time,in natural language processing,machine reading comprehension is an important and challenging task,which is often used to measure the ability of computer to understand natural language.The proposal of deep learning model and the emergence of large-scale machine reading comprehension datasets make it possible to train neural network model through deep learning method.The general form of these datasets is that given a text and some questions,the computer is required to solve these questions according to the given text.However,in practical application,it is often difficult to get an accurate answer only by a given text.Some common sense or factual knowledge can help computers better understand the question.Therefore,a new trend in the field of machine reading comprehension is to use knowledge outside the dataset to assist the computer in completing the task of machine reading comprehension.This paper also follows this trend to study Chinese machine reading comprehension based on external knowledge.This work faces two challenges:1)the basis of this work is to get external knowledge related to the target dataset,and it is difficult to retrieve the relevant external knowledge.As far as the Chinese field is concerned,there are few open source knowledge sources and can’t be directly used as relevant external knowledge sources.For example,the two knowledge graphs of OwnThink and CN-DBpedia have large data scale,but most of the data are irrelevant to the target dataset.2)The deep learning model used in this work needs to take the vectorized data as the input,which requires encoding the external knowledge and encoding the structured knowledge into a vectorized form suitable for the deep learning model.In order to solve the above problems,this paper has carried out the following work:Firstly,in order to solve the retrieval problem of relevant external knowledge,based on the open source knowledge graphs,this paper constructs an external knowledge source KSCMRC2018 which is strongly related to the Chinese reading comprehension dataset CMRC2018。In order to ensure the strong correlation between the constructed external knowledge source and the target MRC dataset,this paper defines the concept of "related entity" from the perspective of semantic importance,and identifies all related entity words through entity linking technology;In order to ensure the high quality of the constructed external knowledge source,this paper defines the concept of "invalid entities" from the perspectives of semantics,vocabulary commonality and word frequency,and designs manual rules to filter these invalid entities.Finally,the external knowledge source KSCMRC2018 is constructed contains about 650,000 triples of items.Secondly,in order to solve the problem of external knowledge encoding and fusion,this paper proposes an external knowledge fusion method based on knowledge representation vector,and designs and implements a machine reading comprehension model KR-net(Network with Knowledge Representation vector)based on this method,which improves the effect of machine reading comprehension task.At the same time,in order to ensure the acquisition of knowledge representation vector,this paper also designs a translation model Trans-BiLSTM based on the idea of knowledge representation learning and the BiLSTM network structure,which is used to extract knowledge from the external knowledge source KSCMRC2018 to train knowledge representation vector.Finally,considering the unstable storage space occupied by the external knowledge fusion method based on knowledge representation vector,this paper proposes an external knowledge fusion method based on the language model containing knowledge,and designs a "twin-tower" fusion model structure to complete the task of machine reading and understanding.In this method,a negative sample generation strategy is proposed to help train a language model Mk with specific knowledge.It is integrated with the general pre-trained language model to obtain the results of machine reading comprehension.Finally,the feasibility and effectiveness of this method are verified by experiments. |
