| With the rapid development of marketization and the continuous improvement of people’s living standards,people’s material and spiritual needs are expanding and the masses have higher and higher expectations of the government’s administrative capacity.In providing public services,government administration departments need to listen more and better to the voices of the public and provide timely feedback and communication.Customer service staff are the main service providers in the work of the government hotline,and their emotional state will have a certain impact on the effectiveness of the service.If negative emotions arise in the dialogue with the public,it will not only hinder communication and reduce the efficiency of the service but also reduce the satisfaction of the public and affect the image of the government.In order to obtain the emotional state of customer service in a timely manner,this paper takes the voice conversation information of customer service in the government hotline as the research object.Customer service emotions are analyzed and classified,and an emotion recognition model based on speech-text bimodality is established to predict the emotions of customer service agents.An emotion alert system is designed and implemented through relevant technologies to monitor the emotional state of customer service during hotline conversations.The main tasks include:（1）Analysing and classifying customer service emotions on government hotlines.A sample of customer service calls on the government hotline was listened to and the emotional characteristics of the customer service work were analyzed.Based on the six basic emotions defined by Ekman’s discrete emotion theory and the emotional characteristics of customer service on the government hotline,the customer service emotions were classified into four categories:enthusiasm,calm,disgust,and anger.（2）Building a bimodal Chinese customer service emotion dataset CSR-Emo.This paper builds a speech-text bimodal Chinese customer service emotion dataset based on the classified customer service emotions of the government hotline.The data from four sources,THCHS-30,Aidatatang＿200zh,ST-CMDS,and the government hotline,were collected and pre-processed by filtering,cutting,and transcription to obtain short audio and transcribed text.The data were sentiment-labeled by the annotation team,and the final labels were obtained by majority voting method.（3）Building a bimodal-based deep learning multiclassification model BIERM.This paper converts the emotion recognition problem into a deep learning multiclassification task and constructs a speech-text bimodal emotion recognition model.The model contains CNN,Transformer,and other modules.MFCC and its first-order and second-order differentiation are selected for speech features,word embeddings are extracted by BERT for text features,and decision fusion is used to obtain emotion recognition results.Experimental evaluation was conducted on the CSR-Emo dataset,and the overall accuracy of the model was 68.8%,with the F1 score of 68%.After the ablation experiments,it can be seen that BIERM outperformed the model using only unimodality as well as other related speech emotion recognition models.（4）Design and implementation of a customer service emotion alert system.This paper analyses the operation mode of the government hotline center in terms of service architecture and service process,and designs and implements a customer service emotion monitoring and early warning system.The early warning system provides early warning functions in a comprehensive manner through both real-time detection and periodic reporting.The research in this paper provides a set of intelligent customer service emotion warning systems for the government service hotline center,which achieves a certain degree of early warning effect and support.The research results of this paper also provide a reference for research and application in the field of emotion recognition. |
