Deep Learning Based Machine Automatic Question Answering

Machine automatic question answering has been regarded as a measure of computer intelligence since its conception,which aims to rely on different information sources to answer human natural language questions.It is one of the most difficult challenges in the field of natural language processing(NLP).Nevertheless,limited by the machine learning theory and computing capacity,prior studies in automatic question answering fail to match human’s expectation.In the 21 st century,the advancement of deep learning techniques and large-scale datasets fuel the evolution of machine learning based question answering.By arousing the interest of the NLP community,the relevant research acquires impressive progress.Leading by the innovation of computer science,cognitive science,and sociology,the potential of automatic question answering also increases,while new challenges emerge,i.e.,interpretability of results,the complexity of applications,awareness of the ability’s constraints,and cross-documents information aggregation.Thus,how to realize the synergistic effect of human creativity and high-tech information technology on the machine to achieve intelligent automatic question answering has undoubtedly become an urgent problem to be solved.This thesis,therefore,aims at raising the upper threshold of automatic question answering,focusing on the challenges from knowledge graph reasoning,single document multiple-choice type,single document extractive reading comprehension,and multiple documents reading comprehension.We follow the learning patterns of deep learning and apply a series of state-of-the-art theories,including reinforcement learning,prompt learning,contrastive learning,and graph representation learning,to motivate novel designs.The main research contents and contributions are summarized as follows:(1)Revised Reinforcement Learning Formulation for Multi-Hop Knowledge Graph ReasoningTraditional knowledge graph reasoning methods cannot explicitly generate the reasoning path,resulting in prediction results with low interpretability,and previous reinforcement learning based method cannot explore the reasoning path of any length.Therefore,this thesis innovatively proposes a revised reinforcement learning formulation for multi-hop knowledge graph reasoning,which can explore reasoning paths of any length,thus improving the reasoning accuracy and explicitly displaying the corresponding reasoning process.Specifically,first,we design a stop signal to enlighten the agent to know when it is supposed to finish the reasoning and stay in the current entity via a special action ”STOP”and the path punishment award mechanism.After that,a worth trying signal is devised to exploit valuable patterns from wrong reasoning paths that fail to reach the correct entity in the end.Last but not least,extensive experiments on multi-hop knowledge graph reasoning prove that the proposed method can actively explore the short but correct reasoning paths and precisely arrive at the ideal target.(2)Prompt Tuning for Attributing Unanswerable QuestionsTraditional single-document extraction based reading comprehension was mostly built upon the assumption that the target answer must exist in the relevant text,which does not conform to the actual question-and-answer scenario.Although follow-up studies have explored the answerability of the question,explanations for why the question is unanswerable have been ignored.Therefore,this thesis innovatively proposes a prompt tuning-based framework for attributing unanswerable questions,so as to explore a machine question answering system with self-awareness.Specifically,first,we introduce common features from training samples of different attribution classes into the continuous template to acquire the cause-oriented templates in the vector space.After that,the label semantic of each cause performs as the guidance of the learning direction of the model to make it understand complicated information inside attributing reasons.In addition,we incorporate two modules to the masked language model,which converts the text classification into the cloze test form.Last but not least,extensive experiments on reading comprehension containing unanswerable questions validate that the proposed method can not only determine the unanswerability but predict the proper cause.(3)Unified Machine Reading Comprehension Framework via Recombination and Adversarial LearningSingle-document based multiple-choice reading comprehension often assumes that the type of problem involved is single and the options are fixed,which is set to facilitate model optimization and improvement,but it fails to satisfy the practical application of multi-type problems and unfixed options due to the oversimplification.Therefore,this thesis innovatively proposes a unified machine reading comprehension framework via recombination and adversarial learning,so as to meet the increasingly complex real needs of users.Specifically,first,we reformulate the candidates and questions to unify different classes into true or false determination.After that,we model the interaction between generated candidates and the relevant document through information fusion and attention mechanism.In addition,inspired by adversarial learning,we add the stochastic disturbance to the training process to improve the model’s robustness.Last but not least,extensive experiments on high-school students’ tests validate that the proposed method can flexibly and correctly tackle various questions.(4)Contrastive Heterogeneous Graphs Learning for Multi-Hop Machine Reading ComprehensionThe biggest challenge in multi-document reading comprehension comes from the information loss caused by inaccurate positioning of key information during cross-document information interaction.Existing studies avoid the loss of relevant text through entitylevel and document-level content interaction,while the performance is not satisfactory due to the noise information.Therefore,this thesis innovatively proposes a contrastive heterogeneous graph learning method,so as to improve the accuracy and efficiency of multi-document information interaction.Specifically,first,we select sentences directly and indirectly relevant to the question from all documents and combine them with documents and entities to construct a heterogeneous graph that helps perform the reasoning.After that,to enlarge the dissimilarity of representations in the graph,we design two strategies,i.e.,structure drop and representation drop,to build the contrastive loss function.In addition,we build a multitask learning framework to combine these modules into a single training procedure.Last but not least,extensive experiments on open multi-hop reading comprehension validate that the proposed method can accurately locate the key information of the documents and discriminate confusing candidates’ representations.