The Research And Applications Of The E-blackboard Based On Facial Expression Identification

The importance information technology is becoming vital in all kinds of fields,and education has also made great achievements in the technology change of our society.The fast development of “computer touch technology+ education” has created a great number of media teaching products,such as double screen electronic blackboard,Nano blackboard and electronic whiteboard,etc.The survey in the blackboard promotion reveals that in a class,teachers interact with students mainly through observation and asking questions.However,due to their different personal experiences,teachers might get different learning feedbacks and learning state from students and fail to achieve supposed teaching effects.To solve this problem,deep learning technology is applied on the basis of dual-screen electronic blackboard to creatively apply facial expression recognition to the dual-screen blackboard,which can not only improve education efficiency and convenience and teaching efficiency,but also help teachers know student’s real performance.The specific steps of applying facial expression recognition to the dual-screen electronic blackboards are: face detection,basic facial expression recognition,the determination rules of facial expression and recognition,and the presentation of interactive results.Focusing on the application of basic facial expression in dual-screen electronic blackboards,this research studies facial expression methods based on the hybrid model of transfer learning and the face detection method based on YOLOV3.The main work is as follows:1.In practical applications,the scale of images of students in class is relatively small,and a small amount of training data cannot support the establishment of the model.In response to these problems,this thesis uses transfer leaning to extract features,transfer the parameters of the trained model to the new model,and lock all convolutional layers.In constant training,it is found that the facial expression recognition algorithm can only achieve limited recognition if it only uses facial feature points,so it is often necessary to extract more complex features.This thesis proposes a convolutional neural network hybrid model based on the combination of a restricted Boltzmann machine and a multi-layer fully connected layer,which combines transfer leaning with a trained model.This new model combines the feature leaning capabilities of these two models and classifies the structural high-order statistical features of the image.2.The number of students in the classroom is large but the space is limited.Therefore,the student images collected by the electronic blackboard might affect the judgment of the detection model due to the camera angle problem,the casual sitting posture of the students in class,and the block of the students in front of the class,resulting in the loss of internal data in the image and spatial hierarchical information.These problems bring difficulties to the accuracy of face detection.Therefore,this thesis adopts the hole convolution,and adds the ECANet attention mechanism on the basis of YOLOV3.3.Based on the above-mentioned expression recognition algorithm and face detection algorithm,this thesis designs a student expression recognition and analysis system which can be applied in dual-screen electronic blackboard.The system realizes the real-time collection and detection of student expressions,and uses the method of expression recognition to add weight to the facial expressions of students to analyze the students’ class performance.It visually displays the students’ class concentration,attendance information and student information,and adds intelligent interactive functions to the smart blackboard,and provides more data visualization support for teachers’ teaching.