Research On Answer Selection Based On Pre-training Model

The massive data on the Internet brings some challenges and it is difficult for users to obtain knowledge from the massive data accurately and quickly.The emergence of QA solves these challenges by providing users with accurate information.Answer selection is an important part of QA,improving its accuracy and supporting the development of QA.In order to solve the problem of insufficient extraction of text Semantic information in the model,this thesis investigates the answer selection task based on pre-training as follows.The current mainstream models cannot adequately represent the semantics of question-answer pairs,do not fully consider the relationships between the topic information of question and answer pairs,and the activation function has the problem of soft saturation.To solve these problems,an answer selection model based on pooling and feature combination enhanced BERT was proposed.Firstly,pooling operation was introduced to represent the semantics of question and answer pairs based on the pre-training model BERT.Secondly,the relationships between topic information of question and answer pairs were strengthened by the feature combination of topic information,and then the semantics of question and answer pairs and the feature combination of topic information were spliced.Finally,the activation function in the hidden layer was improved,and the splicing vector was used to complete the answer selection task through the hidden layer and classifier.The model has higher accuracy and F1 value than the baseline model on Sem Eval-2016 CQA dataset and Sem Eval-2017 CQA dataset.In response to the lack of knowledge of BERT and the average pooling operation ignoring the global semantic features of question-answer pairs,this thesis optimizes the previously proposed model based on the previous research.To solve these problem,an answer selection based on knowledge base and convolutional neural network enhanced BERT was proposed.Firstly,synonym knowledge was introduced to the underlying multi-headed attention mechanism of BERT,the BERT model learned the synonym knowledge and then better extracted the semantic features of the question-answer pairs.Secondly,adversarial samples was introduced on the original input of BERT-Word Net to enhance the robustness of the model.Then,a convolutional neural network approach was introduced to extracted the global semantic information of the QA pairs.Finally,the combination of the global semantic features of the question-answer pair and the topic information features is combined to complete the answer prediction by the classifier.The model was performed the comparison model on datasets Sem Eval-2016 CQA and Sem Eval-2017 CQA.