The Research On Text Summarization Based On Pre-trained Models

In the fast-paced information age,high efficiency means high value and high profits for people and enterprises.In particular,efficient work and life allows individuals or companies to obtain more core information one step ahead of others.However,under the background of facing thousands or even tens of thousands of new knowledge or information every day,how to efficiently obtain useful information is a topic that needs to be solved and a problem tha t needs to be continuously optimized.In the field of natural language processing,the demand for text summarization tasks has emerged as the times require,which aims to help people quickly obtain key information from massive knowledge information,so tha t they can quickly filter out the necessary information and then read the full text.Therefore,for ordinary users,text summarization technology can help them quickly acquire and filter knowledge in a limited time,and then expand the learning,living and working sequence based on these knowledge information.At the same time,for enterprises,text summaries can help enterprises to directly show users the core points that users want to care about most,which is conducive to retaining users and attracting t raffic,and finally achieves the purpose of commercial profit.To sum up,it can be found that text summarization technology has great industrial value,but it is still not fully explored in academia and industry.Technically,text summarization technology refers to obtaining key information from a long text through computer means and algorithms to form a relatively short text,which can be obtained by reorganization or directly extracted from the words and sentences in the original document.Among them,th e method of extracting words and sentences from the original document to form a summary is also called an extractive text summarization.Since the quality of the extracted feature vectors of words and sentences will directly affect the quality of the final summary text,it has high requirements for the model to understand the semantic features.On the other hand,even after obtaining high-quality embedding features,the model must ensure that there will be no "model collapse" and "exposure bias" phenomena.The "model collapse" means that the extracted features are not separable in the feature space,but make the semantics more ambiguous,and the "exposure bias" refers to the problem of differences between the model in the training phase and the inference pha se.In addition,how "exposure bias" performs an evaluation of compressed text at both sentence level and summary level is also a key issue.Faced with the above problems,this paper carefully studies the relevant literature on extractive text summarization models,large-scale pre-training models,contrastive learning,etc.,and proposes an extractive text summarization method based on pre-training framework and introducing contrastive learning.Specifically,the research point in this paper mainly focuses on: 1)Using the pre-training model to capture text semantics and ease the training difficulty.2)Use the contrastive learning to solve the exposure bias problem caused by the difference between the model in the training phase and the inference phase.3)Reduce the difficulty of designing and training the multi-stage summarization framework through multi-task learning.4)In addition,further optimization of this model is also discussed in depth,mainly from two aspects of text semantic representation and comparative learning representation.In the experimental section,this paper implements sufficient experiments on a public Chinese corpus.This section mainly starts from the detailed experimental settings,model comparison experiments and analysis,ablati on experiments and analysis,summarization framework optimization,visual analysis and display experiments,etc.A large number of experimental results demonstrate the effectiveness and feasibility of the framework proposed in this paper from multiple perspectives.Finally,in order to further demonstrate the practical value and application potential,this paper builds an text summarization platform system that can be used by users.This system can not only complete the operation logic of this model,but also support the expansion in various scenarios such as short text,long text and super long text.Finally,system tests and case studies demonstrate its ease of use and the rationality of summarization results.