Research On Factual Consistency Evaluation Method Of Abstractive Summarization

In today’s information society,with the rapid development of the Internet,information is growing exponentially.People can easily obtain various types of text information,which requires more time to filter,digest,and understand.Therefore,text summarization technology can meet the demand for quickly understanding and analyzing text information.As a key research direction in natural language processing,text summarization technology has a wide range of practical applications and research value.Text summarization technology is usually divided into extractive summarization and abstractive summarization.Compared with extractive summarization,which constructs summaries by concatenating phrases and sentences from the original text,abstractive summarization that involves understanding the original text and rephrasing it better meets the demand for human reading.However,current abstractive summarization models and systems still face difficulties and challenges,among which the "factual inconsistency problem " issue is of particular concern.Factual inconsistency problem refers to the inconsistency between the factual information in the summaries generated by abstractive summarization models and the factual information described in the source documents input to the model.Such a deficiency hinders the accuracy of obtaining text information and can result in misinformation,seriously affecting the credibility and reliability of abstractive summarization models and systems.Evaluation methods for factual consistency in abstractive summarization can help the models detect summaries with factual errors,and promote the development and application of text summarization technology.In recent years,it has become a hot research topic in the field of natural language processing.This paper conducts research on evaluation methods for factual consistency in abstractive summarization,including the following main work:(1)A factual consistency evaluation model with multi-attention mechanism for abstractive summarization is proposed in this paper.To address the issues of insufficient reasoning information and simple reasoning process in current factual consistency evaluation methods based on natural language inference,the proposed model can obtain key reasoning information through similarity calculation of word vectors between sentences,and obtain contextual representations of summary sentences and key information through pre-trained language model’s contextual encoding.The model strengthens the reasoning process through local information interaction,and further enhances information communication during the reasoning process using graph attention mechanism.In addition,considering the limited amount of data samples in current datasets for fact consistency in abstractive summarization,a training dataset is constructed based on common types of fact errors to help the model distinguish factual consistency errors.Experiments show that the proposed method has significant improvements compared to current mainstream factual consistency evaluation methods on commonly used metrics,demonstrating the superiority of the proposed method.(2)A prompt learning method for factual consistency evaluation in abstractive summarization is proposed in this paper.Considering the issue of limited labeled samples in the field of factual consistency evaluation methods for abstractive summarization,prompt learning can transform the downstream task into a cloze task that is the same as the pre-training process of the language model,alleviating the task discrepancy between the two and achieving good results with limited samples,to some extent,solving the low-resource scenario in factual consistency evaluation.This paper first attempts to analyze the impact of manual templates on factual consistency tasks,and then designs a way to learn continuous prompt templates,combining learnable vectorized templates with source documents and summary sentences,inputting them into a BERT-based masked language model to elicit latent knowledge in the model to assist in judgment.Experiments show that the proposed method is not as good as the best supervised methods on the human-annotated CNN/DM dataset,but it outperforms other methods in the human-annotated XSUM dataset with limited samples,indicating the effectiveness of the proposed method.