Research On Multi-modal Learning Sentiment Analysis For Online Learning

With the continuous development of Internet technology,online learning has become one of the mainstream learning methods,especially during the period of COVID-19,the number of users of online learning has exploded.Because it is not limited by the time and space of traditional classrooms,and has rich educational video resources,many learners are attracted to learn online independently.Although this new way of education brings great convenience to teachers and learners,it hinders the emotional communication between them.Teachers and students make eye contact in the actual teaching scene,and learners’ sitting posture,head posture and subtle facial expression changes can all express students’ learning emotions and states.In the remote education system,not only through the body posture and facial expression recognition to synchronously analysis learning emotion can help to improve learners’ learning efficiency by timely emotional communication between teachers and learners,it can also help the teacher to improve the teaching content,so as to enhance the mutual action between teachers and students.By using various emotion recognition and analysis methods to complement each other,the results of emotion analysis can be more accurate and objective.In this paper,the status quo and key technologies of multi-modal sentiment recognition and learning sentiment analysis have been studied and analyzed,and the main problems to be solved in this paper are identified in two aspects:1)There are few researches on multi-modal emotion recognition algorithms in the field of education.2)There is a lack of Online learning emotion analysis visualization system.In view of the problems existing in the two aspects,this paper studies the following two parts:1)Design and research the multi-modal sentiment recognition framework based on feature fusion.2)Design and implement a sentiment analysis system for online learning.The main work contents are as follows:(1)In order to improve the recognition accuracy of students’ learning emotions during online learning,this paper proposes a multi-modal emotion computing model based on feature fusion.Firstly,the Deep Convolution Generation Adversation Network method has been used to enhance and expand the self-built learning emotion data set,so as to obtain more training data.Secondly,in view of the problem that the traditional feature extraction algorithm is susceptible to noise,occlusion and other factors,this paper adopts the deep convolutional neural network method to extract the features of facial expression data and attitude data,and then carry out feature layer fusion of the two modes.The weight vectors of different features are calculated by the optimized DS evidence theory,which can not only indirectly improve the accuracy of multi-mode feature fusion,but also effectively suppress the relatively inefficient features that are easy to interfere with the recognition results.Finally,learning emotion will be classified and recognized by DenseNet.By using DenseNet as the infrastructure,the structure of the multi-modal affective computing model is designed.Finally,the model was validated in the self-built bimodal learning sentiment dataset,and the results show that the proposed multi-modal sentiment computing model based on feature fusion has a good accuracy.(2)In order to facilitate the teacher to understand the student’s learning emotion in online learning situations,a multi-modal sentiment analysis system for online learning has been designed and developed on the basis of the multi-modal sentiment computing model based on feature fusion.The system is a learning emotional feedback platform for students and teachers.As a student user,you can learn the course through this platform.The system will record the learning process of students.After learning,the emotion generated in the learning process of students is identified through the multi-modal emotion recognition framework and visualized.As a teacher user,you can upload and manage courses through this platform,understand the overall learning situation of the course and the emotional feedback of individual students,so as to improve the quality of the course.Through this system,teachers can observe the distribution of students’ learning emotion types in different course periods,and teachers can improve teaching methods and strategies in different time periods by analyzing the overall changing process of learning emotion.Finally,this paper also verified the effectiveness of the system by comparing and analyzing the results of manual statistics and system identification.