Research On Classroom Student Action Recongnition Algorithm Based On Pose Tracking

With the popularization of campus digital construction,through deep learning methods,classroom videos can be fully excavated,providing students with class status information,helping teachers improve work efficiency,and urging students to study seriously,but it cannot meet the changes of different scenes,therefore,it is still challenging to apply action recognition to classroom videos.By analyzing classroom videos,aiming at the problems of dense students,similar action and serious occlusion,a pose tracking method is designed to extract the student’s single-person pose flow,and a model combining graph convolutional neural network and self-attention mechanism is constructed to further improve the accuracy of student action recognition.The main work of this dissertation includes:Firstly,in view of the problem that students cannot locate and extract pose flow due to crowded students in classroom,a classroom student pose tracking model based on FFL-Track is proposed.The model converts the skeleton information obtained by the pose estimation algorithm into the human pose expression in the classroom,filters the illegal poses and redundant poses in order to obtain the correct skeleton information,calculates the similarity of students’ pose in the first frame and the next frame,the latest frame and the next frame respectively,and matches the students with dual information to obtain a single-person pose flow,in order to deal with the problem of student ID switching caused by occlusion and missing.Secondly,in view of the problem that the similar poses of students in classroom videos cannot effectively extract features and classify them,a classroom student action recognition model based on SGCTSA is proposed.The model constructs the student’s spatiotemporal graph structure according to the characteristics of human skeleton,adds skeleton centering and time position embedding to the student graph structure,so as to retain the sequence relationship and action characteristics;uses the spatial graph convolutional neural network and the temporal self-attention mechanism to extract the temporal and spatial characteristics of students’ action,in order to improve the expression ability of the model.Finally,the two models are fused to achieve end-to-end multi-person pose tracking and action recognition in classroom videos.In order to verify the effectiveness of the model,datasets of classroom students’ pose tracking and action recognition are established,ablation experiments are performed to verify the rationality of each component,and quantitative researches and qualitative analyses show the practicability of the model in classroom videos.