Effective Yawn Detection Method Based On Deep Learning And Model Compression In Classroom

People are likely to yawn when they are bored,drowsy.The yawning of the students in classroom can reflect their distraction during courses.Yawn detection of students in classroom can be used as an auxiliary means for teachers to know the learning state of students and improve teaching quality.Traditional yawn detection methods are mainly used in driver fatigue detection systems,which are generally implemented by three steps: face location,mouth feature extraction and classification.However,these methods are not suitable for complex classroom scenarios.Firstly,the number of students in class surveillance video is usually about 30.The background differences of different surveillance videos increase the complexity of the scene.Secondly,the face resolution of students who sit in the back rows is about 30×30pixels.It is very difficult to locate the faces in images because of the low resolution.Thirdly,during courses,students often shake their heads left and right and talk to others sitting near them,resulting in the change of face orientation.The facial features are similar to yawning when students are talking or reciting,so extracting mouth features of yawn is very difficult.Due to the limitation of traditional methods,we dexterously utilize and improve object detection algorithms of deep learning,regarding the yawn detection as object detection problem to solve.In the classroom scene,we introduce feature pyramid method to solve low resolution and multi-scale problem.Yawn detection is just one of the tasks in the teaching evaluation system.In addition to that there are also other tasks including hand detection,sleep detection,etc.One detection model should not consume too much memory and computing resources so as to run multiple models parallelly in the system.We use network pruning to reduce the size of the model and improve the speed so that it can be deployed to applications.We collect more than 300 class surveillance videos of primary and middle schools,and make a yawn dataset with these videos.We compare our proposed method with state-of-the-art algorithms on this dataset,proving that our method outperforms others in accuracy and speed.