Class Naming And Listening Status Detection Based On Video Frames

Class naming and listening status detection are important components of classroom teaching management.In order to solve the problems such as the low efficiency of traditional class naming and listening status,this thesis studies and implements a system of class naming and listening status detection based on video images.Class naming part mainly adopts face recognition algorithm based on deep learning.Listening status detection part uses face detection algorithm to record listening time,and analyzes students’ classroom statuses based on key points detection algorithm of human body.The main work of this thesis is as follows:1.Create an image dataset for class naming and a video dataset for listening status detection.There are 200 images in the image dataset,with a resolution of 1280×1024.There are a total of 10 video clips in the video dataset.Each video has a duration of 5 minutes,with a resolution of 1280x1024.Both images and video clips are collected in real classroom environments.2.Design and implement class auto-naming function with SeetaFace face recognition engine based on deep learning.①Detect faces based on Funnel-Structured Cascade,and crop to obtain face images.②Implement face alignment based on Deep Auto-encoder Networks.③Match the face image with all face images in the face template gallery,then count the number of identified people to get the result of class naming.The accuracy of class naming based on deep learning is about 98.9%,better than the algorithm with artificial extraction features.3.Design and implement listening status detection function.①Record listening time of each classmate with SeetaFace face detection algorithm,and the average accuracy reaches 96.4%.②Analyze three classroom statuses(head-down,body-leaning,hands-up)with coordinates obtained from OpenPose keypoint detection of human body.The average accuracies reach 90.1%,84.7%,and 92.0%,respectively.4.Design and implement a system of class naming and listening status detection based on Android.①For the class naming part,the code is transplanted from Windows to Android.The average time to detect an image on Android is about 25s.②For the listening status detection part,Android Studio and Beego development platform are used to realize the asynchronous communication between the mobile phone and the server.It costs about 50s for a 5-minute video clip from the transmission to the return detection.