Research On Diversity And Exposure Bias Alleviation Methods For Question Generation

Question Generation(QG)is a challenging and innovative task in the field of Natural Language Processing(NLP)that aims to automatically generate relevant questions based on given contexts.QG has diverse applications such as data augmentation for questionanswering models,guiding chatbot conversations,and assisting intelligent teaching systems.However,current QG models lack the ability to generate diverse questions.Thus,further research is needed to develop models that can generate questions that are not only relevant but also varied.Additionally,a significant gap exists between the quality of samples encountered during training and inference.This gap leads to the problem of exposure bias,which occurs because the model learns in "teacher-forcing" mode during training by learning standard questions,whereas it can only rely on generated tokens for autoregressive generation(i.e.,"free-running" mode)during inference.This paper focuses on two specific areas of research related to QG.1)Firstly,the paper addresses the issue of achieving diversity in QG,which currently relies mainly on adjusting the sampling strategy of the decoding stage.However,such methods may reduce the accuracy of models.To address this,the paper introduces the conditional variational autoencoder framework that uses the pre-trained language model UniLMv2 for encoding and decoding.Experimental results demonstrate that this method effectively improves both the accuracy and diversity of QG.2)Secondly,the paper explores the issue of exposure bias in QG.The paper upgrades the SeqGAN model by using UniLMv2 as both the generator and discriminator,and by conceiving a discriminator structure that directly generates token-level rewards,avoiding the need for Monte Carlo search used in SeqGAN.This upgrade effectively improves model performance and reduces training costs.3)However,optimizing the objective function of the adversarial framework can be difficult,and multi-round confrontation is timeconsuming.Therefore,we propose two modules to address exposure bias in QG.First,we suggest using the moment matching network to narrow the moment information between generated samples in binary channels(training and inference modes).Second,we propose a data augmentation strategy called "assistant-cooperating" to weaken the constraints of "teacherforcing" training and reduce the gap with "free-running" generation,improving the model’s performance in real-world scenarios.This paper presents a novel diversity generation method based on the conditional variational autoencoder that effectively improves the accuracy and diversity of QG tasks.In addition,this paper proposes two methods to alleviate exposure bias and compares them with current state-of-the-art QG models and exposure bias alleviation strategies,demonstrating their effectiveness.The results show that our proposed methods outperform existing approaches.