Research And Application Of Facial Expression Recognition In Online Education Based On Realsens

The importance of online education has been increasingly highlighted with the continuous development of remote real-time video technology and the outbreak of the COVID-19 pandemic.Although the epidemic situation in our country has stabilized,this sudden event has also sounded the alarm for the world,pushing the development of online education onto the "expressway".However,the lack of emotional communication between teachers and students due to the temporal and spatial separation of online education has always been a problem in this field that needs to be urgently solved.This paper,using Realsense depth camera,aims to help teachers obtain real-time facial expressions and postures of students in online education,analyze their learning status,and thus improve the quality of teaching.The main content and innovation points of this paper include:(1)Having analyzed common learning expressions of online education students,a learning expression dataset was self-made using a Realsense depth camera,which not only includes RGB images and corresponding depth images of each frame but also comprises poses of different expressions.This article subsequently conducted various preprocessing methods for RGB-D images and selected the most suitable method for the dataset through comparative experiments.(2)A dual-channel and multi-task DCMT-Res Net34 network model was proposed to address the impact of factors such as lighting and head posture on online education facial expression recognition.The RGB images and depth images were separately input for feature extraction,and the task of head posture,which is highly correlated,was added as an auxiliary guidance for joint training,aiming to guide the model to learn features and improve efficiency.Finally,the extracted features were fused,and Softmax was utilized for facial expression recognition,while fully connected layers were utilized for estimating head posture.(3)To address the issue of the DCMT-Res Net34 network model being too large,which affects both speed and accuracy,dilated convolutions and spatial attention mechanisms were integrated into the entire network to enlarge the model’s field of view and further explore features while ensuring speed.Furthermore,a loss function was designed by merging cross-entropy and Euclidean distance to enhance the accuracy of feature extraction for both tasks.Experimental results demonstrate that the proposed method achieved a speed of 0.584 seconds,a facial expression recognition accuracy of88.04%,and a head posture recognition accuracy of 97.97%.(4)A Realsense-based online education facial expression recognition system was designed in this study to perform real-time recognition and overall analysis of the learning status of individual and multiple students during Tencent Meetings,and the feasibility of the system was validated and practicality.