Research On Automatic Recognition Of Classroom Teacher Behavior Based On Multimodal Fusion

Education is a hundred-year plan.At present,most of the research on classroom teaching behaviors at home and abroad is mainly based on MOOC,SPOC and other methods,and offline as a supplementary mode.While the current mainstream education model is still offline,the evaluation of offline classroom teaching has always remained at the stage of the original expert review,while classroom teaching analysis and evaluation based on new technologies such as artificial intelligence and big data Always stay at the theoretical level.The evaluation of teacher's teaching behavior is undoubtedly one of the main contents of classroom evaluation,but there is no standard coding method and data set available for the recognition of teacher's teaching behavior.Many technical bottlenecks.In response to these technical bottlenecks and huge challenges,this paper proposes an automatic recognition model of classroom teacher teaching behavior based on multi-modal fusion,thereby solving the above problems and taking an important step for large-scale,high-precision,high-performance AI intelligent teaching evaluation,and The following results were achieved:(1)Established teacher behavior videos,audio data sets and corresponding coding standards in the field of classroom teachingIn the traditional classroom teaching field,Flanders Interactive Analysis Method(FIAS)is often used to analyze classroom interaction behavior.This study takes teacher teaching behavior in classroom instruction monitoring video as the research object,and collects Huazhong Normal University No.8 The 120-hour teaching video of 12 courses in the smart classroom of the teaching building,and the first large-scale classroom teaching behavior video and audio data set in China according to the professional data set production process,with a data volume of 56,000 segments,named CCNU-10,to solve the situation that no standard data set is available for classroom teaching behavior.(2)Propose a video classification model based on deep RNNTraditional video classification methods such as C3D,Two-Stream and other methods are often not ideal for long-term sequence modeling.This paper proposes a RNN structure that deepens the network depth by means of jump connections.By longitudinally deepening the network,it can ensure modeling At the same time,the network is still performing well for a long time sequence,so as to solve the problem of low recognition accuracy caused by long classroom teaching behavior videos.(3)Use voice MFCC combined with deep learning to detect voice scenesThe graphic appearance information in a single video has obvious limitations on the recognition of teaching behaviors.For example,the question link is mainly based on teacher-student interaction,so the voice mode can not be ignored for the recognition of teaching behaviors.It can supplement the lack of video Audio information.(4)Propose a new method for multi-modal teacher's classroom teaching behavior recognition based on voice and videoThe accuracy of single-modality recognition is still relatively limited.Multi-modality is introduced to fuse video,audio,etc.to solve the problem of low accuracy of single-modality recognition.