Learning State Recognition Under Smart Classroom Environment

Facial expression is an important part of communication in our daily life.We can perceive people’s state,feelings and emotions through the change of facial expression.With the rapid development of computer vision,image processing,machine learning and artificial intelligence,the ability of computer to understand image and video is getting stronger and stronger.From the massive image and video data,the computer obtains valuable information to improve production and life is becoming possible.At the same time,with the increasing importance of education and the development of online education,the lack of an effective assessment system of the status of students’ attention to lectures and lectures quality has become increasingly prominent.In this context,this paper presents a method to automatically analyze the facial images of students in class based on classroom video data,and use depth learning and computer vision technology to automatically recognize the learning state.Essentially speaking,facial learning state recognition is also a facial emotion recognition,but these three states in daily life is not as common as the prototype expression,there is a large difference of expression intensity and facial muscle movement between learning state and prototype expression.The main contribution of this paper are as follows:Relying on laboratory projects,this paper collected a large-scale class videos from multiple subjects and multiple grade.In order to organize and mark these class videos efficiently and carefully,this paper designs and implements a complete semiautomatic learning state annotation tool.On the basis of the video data and marking tools,this paper builds a multi-person data annotation system and establish the facial recognition state database.This database provides training data and testing data for this paper,and also lay the foundation for further research in this field.Combining the research experience in the field of traditional prototype expression recognition and the recent research on deep learning,this paper proposes a scene-targeted class-based learning state recognition technique based on deep convolution neural network,which makes the recognition accuracy more accurate and stable.Based on the situation that sufficient training data can not be obtained,the face standardization process is introduced into the algorithm structure,and the network structure is designed according to the requirement,which greatly reduces the requirement of training data size.For the distracted state and other special learning state,flexible use of computer vision technology helps to accurately identify them.