Research On Task-Aware Q&A Generation Model

With the increasing maturity of neural network technology,more and more text generation tasks are replaced by network models.Generation tasks are an important type of task in natural language processing,and their main role is to generate text with natural language forms,such as automatic generation of text summaries,machine translation,question and answer systems,dialogue systems,etc.Generative tasks usually involve processing and transforming the input information into a form that matches human understanding.As a subtask of the generation task,question and answer generation can generate corresponding questions or answers by analyzing the semantic logic in the passage.Compared with traditional methods,pre-trained models can learn the intrinsic logic of language through large samples and thus improve the quality of question and answer generation.There are still some shortcomings in the current domestic and international research methods.First,the existing pre-training models are not targeted for specific downstream tasks,and some key information is often overlooked in a specific task quotation.In addition,the single mask mechanism leads to the limitation of the information obtained by the model from the text segment during the pre-training process,resulting in the model over-relying on the encoder in the generation process.In order to get better performance of the pre-trained model ERNIE(Enhanced Representation through k Nowledge Int Egration)in the Q&A generation task,the following specific optimization schemes are proposed in this paper:(1)To better meet the requirements of downstream tasks,a task-aware and multi-mask pre-training-based question and answer generation model is proposed by constructing different structures in the pre-training and fine-tuning phases.First,a multi-mask pre-training strategy with three mask rules is used to ensure that the model can get rid of the dependence on bidirectional information during pre-training,adapt to the generation rules,and improve the decoder capability.Secondly,the Q&A information in the Q&A dataset is used as the input sequence and substituted into the model for fine-tuning training to highlight the important role of key semantics in the Q&A generation.(2)In order to ensure the complementarity of semantic features between Chinese segments and Q&As in the fine-tuning process,a gated network is used to measure the contribution of different semantics to the final generated sequences of the model,and a method based on gated network is proposed to realize the semantic fusion of Q&As.At the same time,based on the pre-training model ERNIE-GEN,the padding generation mechanism is organically combined with the multi-mask pre-training.Finally,a question and answer generation model based on gated perception and multi-mask noise padding generation is proposed.Comparative experiments are conducted on the datasets Fairytale QA and SQu AD 1.1,respectively,and the model is able to achieve satisfactory results in cross-sectional comparison with other models.On the other hand,the effects of gating-awareness and multi-mask noise filling generation on the model performance are explored through ablation experiments.