| With the development of deep learning technology,researchers hope to build a more anthropomorphic human-computer interaction system.And emotion recognition is one of the key technologies that endow artificial intelligence with anthropomorphic performance.It’s of great help to the fields of online education,psychotherapy,traffic warning and more with accurate emotion recognition in speech.However,due to the ambiguity of emotion definition and the lack of emotion datasets,there is still a large research space for speech emotion recognition.The traditional emotion recognition methods classified the speech emotion by extracting the manual acoustic features in the audio,and modeling the relationship between the feature and emotion through statistical methods.In recent years,various neural network systems have been applied in the field of emotion recognition to extract high-dimensional features so that the accuracy of emotion recognition has been greatly improved.However,most current emotion recognition methods focus on extracting sentence-level emotion representations.And those methods ignore the physical fact that emotion is not immutable in sentences.At the same time,the scale of the dataset also greatly limits the further development of emotion recognition.Therefore,this thesis proposes an acoustic segmentation modeling method to segment emotional speech and perform semantic analysis on acoustic unit sequence information,so as to obtain emotional features that are closer to human emotional descriptions.On the other hand,the sparse problem of the dataset is alleviated by fusing multimodal information,which further improves the recognition performance.This thesis explores the following research directions:First of all,this thesis proposes an emotion recognition system that combines acoustic segment units and latent semantic analysis.GMM-HMM is used to model the acoustic information extracted from speech frames by frames,then the the same state frames are merged as a segment.The acoustic segment units are analogized to the term in the text.Through statistical processing of terms by using latent semantic analysis method,the relationship between segment sequences and emotion description is obtained.Finally,emotion recognition is performed by the deep neural network classifier.The segment modeling method is more similar to the physical phenomenon of emotion in speech,so it can get higher accuracy in emotion recognition task.Secondly,this thesis proposes an emotion recognition system by combining acoustic segment units and deep semantic analysis.Specifically,after using acoustic segment modeling for audio,word embedding method is used to convert the sequences into word vectors.Deep encoding is performed through a neural network to obtain the temporal connection of the sequence context.At the same time,the attention mechanism is used to obtain the statistical information of the acoustic segment units.Then combine the deep encoding features and the statistical information to get the deep semantic emotion representation.Finally,a multi-level data augmentation method is adopted to alleviate the data sparsity caused by complex models.Through deep semantic analysis,more effective emotion features are extracted.As a reason,the accuracy of the system is improved.Finally,this thesis proposes an emotion recognition system that combines acoustic segment model and text semantic analysis.By introducing text information,the problem of alleviating difficulties caused by insufficient data volume is solved.We extract text semantic features by using a pre-trained network,and extract audio semantic features by using an acoustic segment model.Then feature fusion is performed on the acquired text features and audio features.Since the two modalities describe emotion representations from different perspectives,they are complementary to emotion recognition.Therefore,the joint-training is used to adjust the network parameters,while the bilinear pooling method combines the two semantic features to interact with the information between the modalities,which further improves the performance of the emotion recognition system. |
