Research On Out-of-domain Detection Of Task-oriented Dialogue System

In recent years,with the rapid development of conversational artificial intelligence applications,people are increasingly accustomed to using natural language to communicate or send instructions to conversation systems.Systems provide corresponding responses or services to help users complete their work by understanding their words and converting them into machine language.Task-oriented conversation systems focus on tasks in specific domains,and users have clear conversation intentions and slot types,such as restaurant reservations,scheduling,and restaurant names.Due to its complex structure and wide application requirements,task-oriented dialogue systems have always been a research hotspot and have difficulty in dialogue systems.When constructing a task-oriented dialogue system,it is usually necessary to define the dialogue intent and key slots in advance.However,as the environment facing the dialogue system becomes increasingly open,there are more and more conversations with unknown or out-of-domain intentions and slots,and the dialogue system does not know how to handle them.The purpose of the external domain detection task in a task-oriented dialog system is to correctly classify data within the domain,while detecting unknown or unfamiliar data outside the domain,to avoid erroneous operations,tap into the potential needs of users,and establish a more reliable dialog system.Early extraterritorial detection efforts used it as an N+1 classification task,where category N+1 represents labeled extraterritorial data,which relies on additional large-scale labeled extraterritorial samples,and the artificially constructed object-oriented samples give artificial induction bias,which cannot cover all open classes in the actual environment,with significant limitations.With the rapid development of deep learning,more research work has begun to explore unsupervised out-of-domain detection methods.Although current out-of-domain detection algorithms have made significant performance breakthroughs,there are still many difficulties:1.Semantic false correlation:Some similar high-frequency semantic information in the data can be incorrectly bound to label discrimination,that is,semantic false correlation.The model is confused or misjudged on similar categories,further leading to a decline in the performance of extraterritorial detection.2.The problem of uncertainty in the distribution outside the domain.Due to a lack of relevant knowledge outside the domain,the model is not familiar with unseen data outside the domain,and cannot confidently make predictions.It presents the characteristics of planar distribution on a simplex.The prediction probability distribution types of unknown samples outside the domain have randomness,making it difficult for neural networks to accurately distinguish them.3.Prediction overconfidence problem:Neural networks based on softmax classification are prone to generate excessively high prediction probability scores,and the probability distribution predicted by real out-ofdomain samples does not conform to the assumption of ideal uniform distribution,which seriously affects the performance of out-of-domain detection.However,out-of-domain detection methods such as distancebased methods,while improving overconfidence problems,introduce a large amount of additional post-processing work.How to do this without adding additional post-processing conditions,Mitigating model overconfidence is a tricky issue.4.The current research on fine-grained extraterritorial detection is mainly focused on sentence-level extraterritorial intent detection tasks.There are few studies on more fine-grained token-level extraterritorial detection,lacking relevant standard definitions,and facing more challenges such as insufficient context information,misleading functional words,and entity dependency,which need to be addressed urgently.Intention recognition and slot filling are two key sub-tasks with different granularity in task-oriented dialogue systems.In response to the above issues,this paper takes the task of out-of-domain intention detection and out-of-domain slot detection as the entry point,and makes the following research results and work content:1.Propose an external domain detection model based on reassignment comparative learning.Aiming at the problem of semantic false correlation in external domain detection tasks,provide solutions from the perspective of comparative learning,help the model learn more discriminative intent representations,to more effectively distinguish between easily confused internal intentions,and design adaptive class thresholds to more specifically distinguish easily confused external and internal intentions,to alleviate the problem of semantic false correlation.2.Propose an external detection model based on Bayesian estimation calibration.Aiming at the uncertainty of external distribution in external detection tasks,first analyze it from a theoretical perspective,and design a Bayesian estimation calibration strategy to calibrate the distribution of data within and outside the domain to alleviate the uncertainty of external data.3.Propose an external domain detection model based on energy learning,analyze the reasons for overconfidence in external domain detection methods based on softmax probability from a mathematical perspective,and combine energy theory to propose an energy function that is more suitable for external domain detection tasks.Without adding additional post-processing,fully model the intradomain and external distribution,and further propose an energy-based target function to widen the distribution differences of data within and outside the domain.4.Propose an external domain detection model for fine-grained tasks.Aiming at token-level key subtasks in the conversation system,propose a new slot detection task,including task definition,dataset construction strategy,and related evaluation benchmarks.Design a model structure and experimental benchmarks based on multiple algorithm strategies such as distance,and explore the respective advantages and key challenges of different strategies in the new slot detection task,laying a foundation for future research work.