Research On Key Technologies Of Generative Model Based On Abstracts Of Chinese Scientific Papers

Scientific and technological theses are crucial media for showcasing scientific and technological achievements,and their numbers have been increasing rapidly year by year,especially in the Chinese language field.However,writing scientific and technological theses is a research process based on rich academic accumulation,requiring researchers to clearly and explicitly describe the research process,results,and conclusions based on extensive literature and innovative points.As an indispensable part of a thesis,a text abstract is a highly condensed and summarized version of the entire scientific thesis content.Through the thesis’ s abstract,readers can quickly understand the research content,methods,and results of the article,as well as reflect the author’s core views and insights on the research content.In recent years,natural language generation technology has made rapid progress in theory and application,and text summary generation is a typical subtask in the text generation scenario.Therefore,based on the largest Chinese scientific literature dataset CSL,this thesis applies text summary generation technology to Chinese scientific thesis abstract generation,with the aim of providing ideas and guidance for scientific researchers’ thesis writing and improving research efficiency.This thesis adopts a generative summary method to generate long text abstracts of scientific theses using the titles of Chinese scientific literature,mainly focusing on how to solve the five problems faced by generative summary methods in generating long text content: out-of-vocabulary(OOV)problems,poor readability of summaries,repeated generation,unsatisfactory effects of short text generating long summaries,and inconsistency between model training objectives and evaluation metrics.In response to these problems,this thesis conducts in-depth research focusing on the Seq-to-Seq model based on relevant literature references.The main research content and innovative contributions of this thesis are as follows:(1)In response to the lack of specialized solutions for generating long scientific thesis abstracts in current Chinese abstract models,this thesis proposes a Chinese scientific thesis abstract generation model based on an improved T5-PEGASUS model-T5-PEGASUS＿CSLLS.This model further optimizes the T5-PEGASUS model based on the pre-training knowledge learned by the text generation pre-training model.Firstly,key information of CSL data is used for knowledge enhancement,then the attention mechanism of the key information is utilized in the encoder module to enable the model to obtain more key information and the ability to capture fine-grained semantic information.Then,the copy mechanism and coverage penalty mechanism of the pointer generation network in the encoder module are used to reduce the OOV phenomenon,and to address the problems of poor readability and repeated generation of generated summaries.(2)In response to the poor performance of T5-PEGASUS＿CSLLS in generating summaries and the exposure bias problem in the generation model itself,this study proposes a scientific thesis long text summary generation algorithm that combines HNSW recall and target loss fused with the ROUGE metric.The algorithm introduces the HNSW＿CSLLS algorithm module to trace back high-similarity thesis abstract content,greatly improving the quality of generated summaries.By adopting the strategy of incorporating evaluation metrics into the target loss function,this algorithm solves the problem of inconsistency between model training objectives and task evaluation metrics.While ensuring the model’s good performance,this thesis algorithm avoids the evaluation problem that metrics like ROUGE cannot directly act on the loss function.This thesis makes improvements and innovations in the model architecture and generation algorithm performance for the task of generating Chinese scientific thesis abstracts.Through the analysis and experiments on the T5-PEGASUS＿CSLLS model and algorithm,and the ablation study of the factors that affect the model’s performance,the experimental results show that the related modules proposed in this thesis have achieved good effects in solving a series of problems encountered by the model and improving performance,with certain theoretical and practical value.