Research On An Integrated Classroom Management System Based On Non-Intrusive Perception

Based on non-intrusive visual perception,this thesis uses a global gaze camera to detect and locate and identify students in classrooms,and to achieve functions such as Spatio-temporal trajectory query of classroom student locations,classroom energy management,and automatic classroom attendance.The main research elements are as follows.1.To address the problems of dense seating,mutual occlusion,and variable head postures of students in large spacious classrooms,this thesis proposes the DFV-YOLOv5 algorithm by improving the network model in three aspects: anchor mechanism selection,loss function design,and decoupling of prediction layers,using the YOLOv5 algorithm as a baseline.Experiments show that the improved algorithm improves the m AP（mean Average Precision）metric by 4.3% on the public dataset VOC2012 and 3.3% on the self-built dataset while maintaining the volume and inference speed of the model.The improved algorithm was then used to detect and locate students’ bodies in the panoramic image of the classroom and fuse the face detection results to obtain more accurate location coordinates of students in the classroom.Finally,the student location coordinates are perspective transformed and mapped to a two-dimensional seating chart.2.To address the problems of large seat depth in classrooms,varying face imaging size in panoramic images,low resolution,and motion blur,this paper uses linear data fitting and artificial neural networks,respectively,to design a target-driven automatic face capture method for PTZ gaze cameras with high definition,establish a global camera and PTZ gaze camera mapping model,control the PTZ head to focus the gaze camera on the spatial location of students’ faces in order to capture moderately sharp face images at each location,and carry out qualitative and quantitative analysis of these two master-slave calibration models.3.To ensure that the clear image of a face captured by the gaze camera is consistent with the identity of the identified target before and after the panoramic image,the SIFT+KNN algorithm is used to match the clear face image with the features of the target area in the panoramic image,and the consistency verification of the target is achieved by the result of feature matching.The face Net algorithm is then used to identify the clear face image after the verification is passed.4.In response to the lack of student occupancy status datasets in classrooms,this thesis constructs a classroom occupancy status dataset using image data captured by multiple viewpoint cameras installed in classrooms and uses this dataset to evaluate the performance of each model.5.Finally,from the actual requirements of the integrated classroom management system,the software platform for the integrated classroom management system was developed using the Spring-MVC,Springboot,and Mybatis-plus frameworks,storing data in a My SQL relational database and caching data in a Redis non-relational database,and implementing the expected features.