Research On Key Technologies Of Pre-Training Based Open-domain Persona Dialogue Generation

Among the various natural language processing tasks,the human-computer dialogue task has been a hot issue that has received extensive attention from academia and industry.Various statistical models and deep learning models for solving human-computer dialogue tasks are also proliferating.In the field of human-computer dialogue,open domain persona dialogue generation is a recent and popular research problem in the industry,focusing on how to enables the bot to generate responses that are both persona-and context-consistent,given pre-defined information about the persona.In recent years,pre-trained language models based on the Transformer structure have been used for various natural language processing tasks remarkable results have been achieved.In the field of dialogue generation,pre-trained unidirectional language models,represented by GPT,are gradually beginning to be used for open domain dialogue generation tasks,and achieved better results than the traditional RNN-based dialogue model.However,this common structure of the pre-trained Transformer suffers from a variety of problems when directly modeling role-based dialogue generation tasks.In order to make it possible to better model role-based dialogue generation tasks,we conducted the following parts of the study:(1)Using an additional memory module to encode personalized information independently.In order to reduce the noise caused by stitching together persona information and conversation history during encoding,we explored two ways of encoding persona information,the encoding method using an independent encoder and the encoding method using a memory network,respectively.(2)Adding a copy mechanism to the decoder side of the Transformer.Since a large part of the current modeling work for persona-based dialogues is still represented by the way the persona-based information is represented by some key in the reply characterized vocabulary implementations,so we explored how to increase the probability of generating these words in responses using a copy mechanism to generate more consistent responses to personified messages.(3)The fusion of a memory module on the encoder side and a copy mechanism on the decoder side.These two components have their own advantages and disadvantages,so we went on to explore how to fuse the two parts of the characterization encoding approach and add a copy mechanism for decoding,and an automatic scoring approach for role-based consistency is proposed.We further explored how to better model the role-based dialogue generation task using the pre-trained language model of Transformer through the above sections,and evaluated each section of the model using both automatic and subjective evaluation metrics,and the final experimental results validated the validity of our various proposed approaches to modeling role-based response generation.