Deep Sequence Learning Based Open-Domain Dialogue Generation

Open-domain dialogue system can naturally talk with humans on various topics,whose purpose is to maximize user engagement through human-like interactions,thereby building long-term and trustable relationships with users.With the development of deep learning technology and the increasing availability of large-scale conversational corpora,dialogue generation based on deep sequence learning has become the most popular way to build open-domain dialogue systems.It can automatically generate responses for user’s input by learning human conversation patterns from the training on large amounts of conversational data.This thesis aims to improve the semantic and interactiveness of neural dialogue generation and endow dialog systems with better abilities of human-like conversing.Specifically,we conduct researches from the perspectives of dialog history representation,context modeling,topic management,question generation and adaptive response generation,proposing five novel dialogue generation models whose effectiveness have been verified by extensive experiments on large-scale conversation datasets.The proposed models can support the construction of intelligent dialogue systems at the levels of dialog understanding,dialog skills and practical applications.The main contributions are summarized as follows:(1)We propose a dialogue generation method based on neighbouring semantic enhanced dialogue history representationIn this thesis,we consider the explict responding relations between adjacent contextual utterances and propose a dialogue generation method based on neighbouring semantic enhanced dialogue history representation.This method represents dialogue history from the perspectives of the contextual background and the ongoing focus.Specifically,the proposed Keep module re-encodes each contextual utterance by introducing relevance semantics from its neighbouring utterances,resulting in a neighbour-aware context representation;and the Select module regards the latest utterance as the current dialogue focus and enriches it with the relevant information of context,which eventually produces a representation of dialogue focus.Extensive experiments on two public datasets show that our proposal can effectively improve the quality of dialogue generation.(2)We propose a dialogue generation method based on hard-style selective context utilizationIn this thesis,we propose a dialogue generation method based on hard-style selective context utilization.This method first chooses “whether to utilize context” based on the information density of user’s latest utterance(i.e.,query),and then chooses “which context to utilize” by the interaction between query and context.Specifically,we propose two metrics to calculate the information density of query.For a strong query,our method directly inputs it into a Seq2 Seq framework to generate a response.For a weak query,we design a dialogue generation model based on semantic interaction,where context information is selectively introduced to enrich the query.We also extract the co-reference relations existing in dialogue histories as distanced supervision labels and use a multi-task learning framework to promote the prediction accuracy on context selection,thereby improving the quality of dialogue generation.Extensive experiments on two public datasets verify that our proposal can effectively filter out the noises contained in dialogue history and improve the quality of dialogue generation.(3)We propose a context-controlled and topic-aware dialogue generation methodIn this thesis,we propose a context-controlled and topic-aware dialogue generation method,which uses topics to alleviate the semantic sparsity of dialogue texts,meanwhile conducts contextual controlling on topic representation and transition to make dialogue more focusing.We design a context-dependent topic representation module to capture the semantic relationship of “topic-topic” and “context-topic” through multiple attention mechanisms.We also propose a context-controlled topic transition module to obtain coherent and meaningful transition words.Extensive experiments on two public datasets demonstrate that our proposal can achieve comprehensively balanced improvement on dialogue generation in terms of relevance,informativeness and diversity.(4)We propose a dialogue generation method toward proactively asking questionsIn order to make open-domain dialogue system learn to ask,we propose a dialogue generation method specializing on asking questions.We design two parallel mechanisms to generate the question content.The Review mechanism focuses on selecting keywords that are worthy being asked from the dialogue history,which aims to control the relevance of question;the Transit mechanism uses point-wise mutual information to collect transition candidates based on contextual keywords,and filters out irrelevant ones by measuring their coherence to dialogue context.It aims at enhancing the informativeness and diversity of question.Furthermore,to alleviate the data-sparsity issue of question generation on chat corpora,we obtain labelled data by a self-supervising manner and use it to enhance question generation process in a multi-task learning framework.Extensive experiments on two public datasets show that our proposal can generate semantically relevant and informative questions,effectively improving dialogue interactiveness.(5)We propose a context-adaptive dialogue generation model based on dual readersIn this thesis,we propose a context-adaptive dialogue generation model based on dual readers,which can choose appropriate dialogue generation models according to the specific textual features of context.Specifically,we design a sequential readers(SR)specializes on short dialogues,which concatenates historical utterances according to their temporal orders and make the prior utterances be the context of the posterior ones.SR generates a response by a RNN-based Seq2 Seq model with taking the concatenated sequence as input.The jumping reader(JR)focuses on long dialogues,which regards the latest utterance as an anchor and using it to integrate or absorb context.JR generates a response by taking the context-enriched anchor representation.Extensive experiments on two public datasets show that the model can outperform various competing baseline models in terms of response relevance,informativeness and diversity,validating the effectiveness of our proposal.