Research On Mongolian Music Emotion Recognition Based On Multi-modal Fusion

Music is the product of consciousness and emotion,which is closely related to human life.With the development of science and technology,music information retrieval under big data has attracted extensive attention.Emotion is one of the main semantic information contained in music.Classification based on emotion can explore music categories from multiple angles and improve the efficiency of music retrieval.In recent years,it has gradually become a research focus of music information retrieval.Music emotion has a strong subjectivity,rich style,the characteristics of the Mongolian music,as one of the distinctive national music style,because of cultural and artistic value,historical value and commercial value,gradually attention by the researchers,how to better inheritance,propagation and development of Mongolian music culture needs to be more diversified way.At present,the emotional research on Mongolian music is still in its infancy,facing many problems: First,there is no publicly available emotional data set of Mongolian music,resulting in a lack of strong data support for subsequent research,and the general emotional model is not suitable for Mongolian music.Second,the bottom layer of the Mongolian music audio features and high-level audio,lack of emotional connection between the feature extraction method based on the different have different effects,from multi-level incomplete analysis of the emotional tendency of Mongolian music,and most of the current research has focused on a single audio modal or lyric text modal,a single mode of emotion classification can’t both signal features and semantic features.Finally,due to the small scale of Mongolian music data,only deep learning method can not guarantee the effect of feature extraction and take into account the classification accuracy.In view of the above problems,this thesis identifies and classifies Mongolian music emotions based on the fusion modes of audio and lyrics,and carries out the following work respectively:（1）According to the structure of Thayer’s emotional model,with reference to TWC model and Hevner model,and combined with the emotional style characteristics of Mongolian music,an emotional model consistent with the emotional characteristics of Mongolian music is established;Collect and sort out Mongolian music data,and establish Mongolian music database containing four emotional categories.（2）To explore the correlation of audio signal and music characteristic,according to the characteristics of the Mongolian music,extract audio features,bottom on the portfolio selection,it and the integration of audio features based on the deep learning,through comparing with deep learning methods,different learning method combines the characteristics of the classification accuracy of emotion,the overall average accuracy is 61.5%,That’s 7.5% more accurate than the single machine learning method.（3）Establish a multi-modal music sentiment classification model based on the fusion of support vector machine and convolutional neural network.Using convolution neural network audio and lyrics of text feature extracting,the two modal characteristics of the data fusion,the fusion of multiple modal characteristics of the data as the input into the support vector machine（SVM）,the convolutional neural network last two output characteristics of the connection layer,as the input of the SVM classifier,and helped to create a new classification model,To realize the emotional classification of Mongolian music.Compared with the traditional classification method and single mode feature classification,the average classification accuracy of fusion feature and model proposed in this thesis can reach 72.2%,and the accuracy is improved by 8.4% compared with single mode feature classification.