| In recent years,the rapid development of internet technology has led to exponential growth in various types of data information.Meanwhile,ways of information acquisition have also shown a trend of diversification and convenience.In this context,the application demand based on information retrieval,intelligent question answering and other realistic scenarios is gradually expanding.How to promote question answering systems to serve multiple fields more efficiently has become a current research hotspot,which makes machine reading comprehension technology,a mainstream implementation method,has significant research value.At present,the pre-trained language models in deep learning algorithms have achieved excellent performance on some public reading comprehension datasets due to their excellent text representation ability.However,when faced with interference from adversarial bias samples,the textual reasoning ability of existing models is easily damaged,thus affecting the expected predictive performance.Therefore,this paper studies the de-biasing algorithm based on pre-trained models in reading comprehension,which aims to optimize the performance of the model when facing non-directional data bias in real scenarios.The main work of the paper is as follows:(1)Traditional pre-trained models are prone to make shortcut predictions based on position information of the answer when facing training data bias,resulting in the decrease of text reasoning ability.In response to this problem,this paper proposes a de-biasing algorithm named R-PD based on the specific structure of the BERT model.Firstly,regularization technology is introduced into the initialized word embedding to generalize the over-dependence of the model on the location distribution information of answers during the training process.Then,in order to further maintain the predictive performance of the model,the KL divergence distribution term is introduced to the loss function,which constrains the influence of regularization randomness on the training stability.The experimental results show that R-PD algorithm can effectively enhance the semantic reasoning ability of the model and improve its prediction accuracy on biased datasets.(2)Aiming at the problem that the encoding modules of the pre-trained model are stacked layer upon layer,which is easy to lose low-level phrase features,this paper proposes an improved algorithm named R-PD-HC based on semantic information of combinatorial hierarchical encoding modules.The idea of this algorithm is to make full use of the features learned at different encoding layers,introduce pooling module and Sigmoid activation function to assign the weight of hierarchical features,and obtain the classified output of the text features fused at different levels by means of weighted combination,so as to comprehensively considering the surface position features and deep semantic features.The experimental results indicate that R-PD-HC algorithm promotes the model to learn the diverse semantic features and further improves the prediction accuracy.(3)In order to further optimize the predictive performance of the model,this paper increases the diversity feature extraction of the model based on data elements,and proposes a data enhanced R-PD-HC de-biasing algorithm.Considering the unique format of reading comprehension samples,this paper introduces two adaptive sample enhancement strategies for problem text: the first method is the EDA data enhancement algorithm based on sample replacement,deletion,exchange and insertion transformation;the second method is the BERT-QG question generation algorithm based on language training.The experimental results show that the ensemble de-biasing model with data augmentation can better capture position bias,thus further optimizing the de-biasing performance of R-PD-HC algorithm,and ultimately has better prediction performance than the original model. |
