Research And Application Of Commonsense Knowledge Enhanced Natural Language Generation

Natural language generation aims to generate appropriate results based on the given context.It has great application value and significant research significance due to its versatility in various downstream tasks.However,the model hardly comprehends the underlying implicit information and generates contextually relevant content based on the text’s surface information.At the same time,commonsense knowledge can be used to explore the hidden logicality,emotions,and semantics behind the text.Therefore we focus on commonsense knowledge enhanced natural language generation,which utilizes various commonsense knowledge to assist the model in generating results.Firstly,we use event-level commonsense knowledge to enhance the causal logic of the context and generate appropriate story endings.Secondly,multi-granularity commonsense knowledge analysis is used to analyze the conversation’s logicality,emotions,and semantics,generating more humanized,diversified,and contextually appropriate responses.Finally,an emotional dialogue system is developed based on commonsense reasoning to respond with greater empathy.Concretely,we conducted the following research on the above issues:(1)Causal Commonsense Enhanced Joint Model for Story Ending Generation:The story ending generation does not follow the logical development of the story due to the lack of commonsense events.To address this problem,We propose Causal Common-sense Enhanced Joint Model for Story Ending Generation(CEG).Firstly,we propose the commonsense event generation model to transform GLUCOSE into a generation model.It uses prompts to generate commonsense events of stories as pseudo-labels to train CEG.CEG generates story endings through two tasks,commonsense events inferences and story endings generation.The former infers commonsense events for each sentence,providing long-distance information to understand the logic better.The latter uses commonsense events to generate contextually appropriate endings.We jointly train CEG on two tasks to make commonsense events contain more information beneficial for story endings.Ex-perimental results on the ROCStories demonstrate the effectiveness of CEG.(2)Multi-granularity Commonsense Reasoning Enhanced Emotional Dialogue Generation Model: The sentences of dialogue are colloquial and sketchy,causing a lack of commonsense information.Furthermore,it makes the difficulties of semantic,emotional,and logical understanding.To address this problem,we propose a Multi-granularity Commonsense Reasoning Enhanced Emotional Dialogue Generation Model(MCG).Firstly,we use Concept Net to extract subgraphs based on dialogue keywords,providing entity-level commonsense information.Meanwhile,we use COMET to infer commonsense events,providing causal inference at the event level.The model uses graph neural networks and Transformer encoders to encode different knowledge,providing in-formation on logicality,emotion,and semantics.During the emotion-intent prediction,the model improves the Transformer encoder by utilizing parallel multi-head attention mech-anisms to integrate multi-granularity commonsense knowledge and predict emotions from different dimensions.To generate the response,the model uses response emotion-intent and commonsense knowledge as prompts to guide the generation,making the results more contextually relevant and diverse.The model achieves state-of-the-art performance on the Empathetic Dialogues dataset,demonstrating that MCG can accurately predict emotions and intents of users and generate reasonable and diverse responses.(3)Emotional Dialogue System based on Commonsense Inference: Based on pre-vious research,this paper proposes an emotional dialogue system based on commonsense reasoning.The system encapsulates Concept Net and COMET to perform real-time infer-ence on user statements,assisting the system in generating responses.At the same time,the model encapsulates MCG to understand the user’s intention and emotion during the dialogue process,dynamically adjusting strategies to provide the most appropriate emo-tional response to the user.Finally,the model implements a backend network interface and a frontend web client to provide users with real-time communication with the system.