Research And Implementation Of Text Generation Technology Based On Prompt

The text generation task generates the output automatically according to the given input.The typical text generation tasks include machine translation,text summarization,dialog task,etc.At present,the mature scheme to implement these tasks is to fine-tune the downstream tasks on the pre-trained model,which has achieved good results,and also has a very mature application on these tasks.However,with the increasing number of parameters in the pre-training model,this method has the problem of high resource consumption.Another problem of this method is that the pretraining task and the fine-tuning task cannot be aligned,cannot make full use of the knowledge that the model learned in the pre-training phase,and may even lead to catastrophic forgetting of the model.The learning method based on prompt can solve the above problems.Prompt-based fine-tuning method can reduce the training parameters and the amount of training data,but also achieve good results,and gradually become a new fine-tuning paradigm.This paper mainly explores the research and application of Prompt technology in the field of text generation.The main contents of this paper are as follows:a prompt-based parameter initialization method is proposed to solve the problem of slow or non-convergence of prefix-tuning in the case of low-data setting;The structure of prompt,which combines discrete prompt with continuous prompt,combines prompt method with reinforcement learning,and aims at the problem that the scoring model in reinforcement learning is easy to score highly for the repetitive abstract of the original text,a data construction method is proposed.The experimental results show that the proposed methods can effectively improve the performance of the model in low-data settings.Finally,the above technologies are integrated,and a prompt-based text generation system is designed.The system realize content security generation and text summary functions.A sensitive text detection module is added to the system to detect abnormal text and filter unsafe text content.