Research On Emotion Recognition Based On Deep Convolutional Network And Its Application In Online Class

Affected by the epidemic,online classes have become one of the indispensable ways for teachers and students to learn.In online classes,it’s often difficult for teachers to communicate face-to-face with students,teacher tends to pay more attention to teaching contents rather than focusing on student’s facial expression and other information which contains students’ learning state.To a certain extent,the emotional states of students in online classes reflects their acceptance of knowledge points,which further plays a role in the feedback of online classes teaching effect.At present,more and more emotion recognition tasks introduce deep convolutional networks.The emotion recognition technology of static face images based on deep convolutional networks has the advantages of high classification accuracy and strong generalization ability.It shows excellent accuracy and robustness in complex natural scenes such as online classes.In emotional recognition tasks,the deep features extracted by deep convolutional networks are crucial.Therefore,the DenseNet and ResNet network models for facial image emotion recognition based on deep convolutional networks are studied,which helps to compare the two network models of the emotion recognition performance and analyze their respective applicable scenarios.It is applied to online classroom teaching effect feedback,so as to provide technical support for online class teaching effect feedback.The research work is as follows:1、A multi-level supervised fusion ResNet-ASE network model is proposed.At the same time,it supervises the shallow,middle and deep features,integrates the complementary layer features,helps the network model to learn the emotional features,extracts the deep emotional features with the help of asymmetric convolution,and reduces the parameters of ResNet-ASE model.With the help of the deep emotional features extracted by the deep convolutional network,the accuracy of the facial emotion recognition task is improved.Combined with the scene of online class emotion recognition of students’ face images,a deeper convolutional neural network is trained.A multi-level supervision ResNet-ASE network model is proposed,which makes it easier to train the deep convolutional networks;a supervisory information fusion structure is proposed to fuse the emotional characteristics of different depths.The amount of parameters learned by the model has reached ten million,but the training takes up less proportion of memory.The data-enhanced FER2013+ and RAF datasets get the accuracy rates of 95.61% and 92.98% respectively.Due to the ResNet network model needs to learn a great number of parameters,it is suitable for emotion recognition scenarios with sufficient computing resources and lack of memory resources.2、A DenseNet-MSE network model of multi-scale atrous convolution fusion is proposed.DenseNet-MSE can supervise the shallow,middle and deep dense blocks at the same time.For the ResNet network model,due to its huge amount of parameters,it takes huge computational resource overhead.A dense convolutional neural network model with multi-scale atrous convolution module to extract features of different spatial scales is proposed to realize face image emotion recognition under low computational cost.The model is mainly composed of two sub-networks: multi-scale atrous convolution and DenseNet neural network fused with multi-level supervision.The multi-scale atrous convolution extracts different scales features from four-branch networks with different atrous rates;Multi level supervision and fusion to extract emotional features in different depths.Finally,the Adam optimizer and the center loss function are combined in the DenseNet-MSE network.The recognition accuracy of the model for FER2013+ and RAF data sets after data enhancement can reach 93.99% and 88.23%respectively.Compared with ResNet-ASE,the DenseNet-MSE network model has fewer parameters to learn,but the recognition accuracy of DenseNet-MSE is slightly worse than ResNet-ASE.Therefore it is suitable for the emotions recognition scenarios with insufficient computing resources,sufficient memory resources and low accuracy requirements.3、Build a microservice to realize and deploy the teaching effect feedback system of facial emotion recognition in online class.Due to sufficient computing resources and limited video memory resources,the trained ResNet-ASE network model is deployed on the server to intercept the students’ class images in Tencent conference,the face images are intercepted using Dlib’s face detection,and the ResNet-ASE network model outputs the learning status,send the learning status to teachers and students through email in time,design the database and frontend pages,and complete the implementation,test and analysis of the system.