Text Semantic Matching Method With Hybrid Strategies

In recent years,enterprises have increasingly stronger demand for Automatic Client Service(ACS).In traditional ACS systems,customers need to work with multiple departments inefficiently and time-consuming.In the future,interactions of customers can be filtered quickly and accurately through ACS systems,thereby reducing the reliance on labor and cost-saving.Nowadays,Retrieval-based Question Answering(rQA)is the most popular technique applied by commercially deployed automatic client service(ACS)system.Usually,an rQA consists of three parts: query preprocessing module,retrieval module,matching and re-rank module.In an rQA system,the sentence semantic equivalent matching(SSEI)plays the key role in matching user's questions with questions in the database and may affect user experience.However,for the domain-specific ACS system,on the one hand,the traditional text semantic matching technology is often based on feature engineering,and it is difficult to migrate to other fields;On the other hand,how to improve the accuracy of intent recognition in the case of the lacking of various question samples for the same answer in the database at the stage of cold starting,which has become a hot topic of current research and is also the main research topic of this paper.The deep neural networks trained on text semantic matching task can be divided into two categories: sentence encoding-based model and joint feature models which use the cross sentence feature or attention from one sentence to another.Based on these two methods,in this paper,we first propose a multi-layer text semantic matching model based on gate mechanism and residual connection named Denselyconnected Fusion Attentive Network(DFAN)to enhance the capability of representing complicate semantic relation between matching questions.By referring to the idea of Long Short-Term Memory(LSTM)gate mechanism,two gates are introduced before the sentence interaction,which solves the problem that previous methods does not explicitly model word relationship in a single sentence.Inspired by Deep Residual Network(ResNet),we repeat some substructures in the network and use residual connection to enable the model to interact at different granularities.Besides,by keeping the scale of hidden nodes in each layer smaller than existing methods,the total scale of the model is not increased,which keeps the efficient of model training and running.We conducted experiments and analysis on three public English datasets of different tasks,with state-of-the-art 88.8% and 89.51% accuracy on the SNLI,Quora Question Pairs corpora,respectively.To further deal with the corpus lacking issue in cold start stage,the transfer learning and multi-task learning strategies are utilized.In the former strategy,we tried four different methods,i.e.method based on source-task pre-training to target task fine-tuning,method based on source-domain pre-training to target domain finetuning,feature transferring and adversarial domain adaptation.In this paper,we conducted the offline test and online evaluation with limited training examples from the people's livelihood field.The experiment showed that the text semantic matching model combined with a variety of strategy training could reach the best,and the McNemar test with P value of 0.037 proved that the results are significantly improved compared with the previous methods without the use of strategies.Finally,we also show that the proposed text semantic matching model can be applied to online real ACS products.