Estimate Of Student Engagement Based On Multiple Visual Features

Students’ classroom engagement refers to the time and energy that students put into effective classroom teaching activities,which directly reflects the classroom teaching effect and students’ learning gains.Accurate estimate of students’ participation in class is an important basis for teaching quality monitoring.At present,the evaluation of teachers’ offline classroom teaching quality in colleges and universities mainly depends on students’ performance,prearrangement and evaluate of teaching,spot checking of normal teaching and students’ evaluation,which is difficult to reflect the actual situation comprehensively,accurately,scientifically and objectively,and is lack of real-time.At the same time,with the development of online teaching,we need to use new technology to track students’ engagement in real time.Therefore,the objective and quantitative real-time estimating of students’ classroom engagement is of great significance to strengthen the fine management of teaching process and promote the teaching reform and quality improvement.Based on one-dimensional convolutional neural network,a method for estimating student engagement in Multi-Instance learning(MIL1DCNN)is proposed.According to the composition of the concept of student engagement,the head posture,eye sight,eye opening and closing state and the most commonly used 17 kinds of facial action units are regarded as visual features.In feature extraction,based on the video feature files extracted by Open Face tool set,a method of relatively variable feature extraction is proposed.The standard deviation of the distance between adjacent multi frames relative center points of three kinds of visual features is taken as the relatively variable feature of the video.The relatively variable feature sequence is regarded as a package in Multi-Instance learning.The sliding window model is used to divide the sequence into multiple feature subsequence segments,and the subsequence segments are regarded as instances.Considering the low relative position correlation of the features in instances,one-dimensional convolution neural network(1DCNN)is used to analyze the instances,and the student engagement of the instance level is obtained.The MIL-Pooling layer is used to infer the student engagement of video from the engagements of instance.Based on the "engagement in the wild" data set,the MSE is used as the estimate standard.Based on all visual features,the mean square error of MIL1 DCNN is 0.075.Compared with MIL method based on LSTM,the mean and variance of MSE of MIL1 DCNN are smaller when the instance length is 10,40,70,100,130,160 and 190,which reflects that the average estimating effect of MIL1 DCNNis better and more stable.The methods of extracting relatively variable feature and regarding 1DCNN as instance level regression in MIL1 DCNN provide a new idea for the quantitative estimate of student engagement based on video.