Research On Answer Selection Model Based On BERT And External Knowledge

The answer selection is an important task in natural language processing,which can effectively help machine understand natural language and provide fundamental support for artificial intelligence applications such as question answering system and intelligent dialogue system.The emergence of BERT has had a profound impact on the research of answer selection,and its powerful general language capability provides strong support for the construction of answer selection model.This thesis aims to study and solve the problems of absence of features,lack of knowledge and high cost encountered in the application of BERT to build answer selection model,and realize highperformance answer selection model.This thesis focuses on both text feature mining and knowledge enhancement to improve answer selection model and explore efficient fine-tuning method for the application of BERT in downstream tasks.The specific main work is as follows:(1)To solve the problem that the text feature vector generated by parameter-frozen BERT lacks sentence-level semantic features and question-answer pair interaction features,this thesis proposes a feature-enhanced answer selection model based on multilayer perceptron and semantic matrix.The model contains two network modules for mining different features,including the semantic understanding module and the semantic interaction module.Among them,the semantic understanding module uses a fully connected layer to obtain the semantic understanding matrix and embeds it into the multilayer perceptron for extracting the sentence-level semantic features of the text,while the semantic interaction module uses a two-way attention mechanism to compute the semantic interaction matrix and embeds it into the multilayer perceptron to capture the interaction features between question-answer pairs.After experimental verification,the answer selection model is lightweight.Compared with other similar models based on text feature extraction,the model performs well and improves in terms of accuracy.(2)To solve the problems that some knowledge associated in the knowledge base may not be applicable to some specific question-answer pair instances when external knowledge is used to enhance the semantic comprehension of the model,and that the feature representation of knowledge does not take into account contextual information,this thesis proposes a knowledge-enhanced answer selection model based on context awareness.The knowledge enhancement approach in this model consists of two important components,which are the knowledge filtering layer and the knowledge representation layer.The knowledge screening layer dynamically screens knowledge based on its confidence level and the degree of matching between knowledge and contextual environment to alleviate the interference generated by invalid knowledge to the model;the knowledge representation layer uses attention mechanism to adapt knowledge features to the applied context,enhance the role of contextual information in knowledge representation.It also uses a convolutional network to compress the representation of knowledge features.The experimental results demonstrate that,compared to other knowledge augmentation models,this model achieves higher accuracy.(3)To solve the problem of high training and storage costs of BERT fine-tuning methods in practical applications,this thesis proposes an efficient fine-tuning method for BERT models based on cue vectors.The specific implementation of this method adds cue vectors to each Transformer coding layer of BERT,and conduct the model to produce specific output.When training for downstream tasks,it only needs to adjust these cue vectors and bias term parameters to be able to adapt to the needs of downstream tasks.The experimental results show that this method can significantly reduce the number of parameters specific to answer selection during training and deployment of BERT compared to fine-tuning methods,and also has greater advantages in terms of accuracy and generalization capability.Compared with other efficient fine-tuning methods,this method performs better in terms of accuracy and shows better overall performance.