Research On Student Behavior And Concentration Recognition Based On Classroom Video

The classroom is an important place for students to receive education and learn knowledge.The rapid development of artificial intelligence technology promotes the reform of education.It is more and more urgent to make classroom teaching quality information and intelligent analysis of classroom teaching quality.In the classroom teaching activity,students are the main individuals in classroom learning activity.Their behavior state is the direct embodiment of teaching activities,and students’ classroom behavior reflects students’ study attentiveness.In the traditional observation methods,researchers obtain the information of students’ classroom behavior through classroom observation and questionnaire survey.However,this method is difficult to avoid subjectivity,and there are some problems such as low efficiency and incomplete observation,which cannot be extended to intelligent information classroom teaching.The method of classroom behavior recognition based on deep learning overcomes a series of shortcomings of traditional methods,but it also has some data and technical problems.For example,there is no dataset for students’ classroom goal detection and behavior,and deep learning algorithms in real classroom environment are easily influenced by light,occlusion,camera angle and other factors.In view of the above question,this paper mainly carries out the following work:(1)We construct student classroom dataset.In this paper,the video data of different courses and different classrooms are collected to construct the students’ classroom dataset.The constructed dataset consists of the detection part,the pose part and the behavior part.The detection section consisted of 5,000 samples,each samples containing between 8 and66 students.The pose estimation section includes 2,832 samples,each samples contains keypoints and object labels.There were 3,173 samples in the behavior section,which included reading or writing,looking at the blackboard,playing with the phone,looking left and right,standing up,raising the hand,and lying down.(2)We design a component attention-based classroom attention evaluation algorithm,which obtains students’ attention by detecting their classroom behavior.Firstly,the classroom teaching video is sampled and detected to get the student’s position information.Secondly,the multi-object video is transformed into the students’ single-object ID sequence by the method of tracking assignment.Finally,the single sequence of students is input into the behavior recognition network based on component attention to get the single student’s attention score.We used weighted integration to get focus scores for all students and the classroom.In the training stage,the algorithm uses transfer learning to solve the problem of insufficient samples.After several experiments,we show that this method can quickly detect the whole attention of students and the classroom,and the accuracy of behavior recognition algorithm based on component attention is more than 85%.(3)We design a multi-task classroom behavior recognition algorithm that combines human pose estimation and object detection.First,the object detector extracts the individual region from the keyframe as the network’s input.Then,the multi-task heatmap network(MTHN)module extracts the intermediate heat map of multi-scale feature association.The attitude estimation and object detection tasks are constructed by mapping relations to obtain the keypoints and object position information.Finally,the keypoints behavior vector and the metric vector are used to model the behavior,and a classroom behavior detection algorithm based on the fully connected network is designed.Also,we created a classroom dataset with pose estimation,objects,and behavior labels.Meanwhile,transfer learning is used to solve the problem of insufficient sample size.After several experiments,we show that the detection accuracy of the proposed multi-task learning based student behavior recognition algorithm reaches more than 90%.With improved algorithm accuracy,the curves of student and the classroom concentration scores over time showed a smooth trend.