Research On Learning Evaluation Method Based On Multi-modal Information Fusion From The Perspective Of Smart Classroom

In traditional learning contexts,teachers mainly evaluate the behavior and emotional changes of students and the completion of homework to ensure teaching quality.At present,there are still the following problems in the evaluation of students.First,due to the limited energy of teachers,there is a lack of comprehensive and timely evaluation of students.Second,the evaluation angle is single,and it is not possible to comprehensively consider the factors that affect learning evaluation.Third,the correlation between evaluation indicators is not strong,leading to poor evaluation results,which has a certain impact on the development of students.In recent years,with the accelerated development and widespread application of artificial intelligence and information technology,the era of smart classroom has arrived,and new technologies such as image processing and artificial intelligence have provided opportunities for personalized support services and improved teaching quality.The main research contents of this paper are as follows:(1)Building a multi-modal learning evaluation modelIn order to solve the problem that single dimensional information evaluates incompletely,a multi-dimensional learning evaluation model is designed to calculate the scores of students’ learning evaluation results.The model is based on constructivism and multiple intelligences,and combines three dimensional information: students’ cognitive attention,emotional attitude,and course acceptance,with three attributes of student behavior,facial expressions,and answer information.The model describes the learning status of students in the smart classroom from multiple dimensions,and makes a complete assessment of students.(2)A multi-modal classroom information collection modelThe multi-source information includes student behavior,student expression and class answer information,of which the first two need to complete student human body detection.Firstly,a multi-scale attention module MAM is designed to detect students’ positions,and a Multi-task convolutional neural network(MTCNN)model is introduced to complete the detection of students’ faces.Secondly,based on the detected student individuals and faces,,the Open Pose model is used to obtain the skeletal joint points of students,to realize the state recognition of students’ behavior.Complete students’ expression recognition through the mini＿Xception＿ECA model;Finally,answer information is obtained through the classroom interactive platform.(3)A learning evaluation model for multi-modal information fusionOn the basis of learning evaluation model,each dimension in the model is quantitatively calculated,and a weight analysis method for multi-dimensional information fusion is studied.Firstly,the image information collected in the classroom is automatically recognized to determine the threshold of the three attributes of student behavior,facial expression and answering situation.Secondly,the weight values of the three dimensions of cognitive attention,emotional attitude and course acceptance and their corresponding attributes were determined.Finally,the learning evaluation score is calculated by fusing the information of these three dimensions through the fusion strategy,which provides an objective basis for the learning evaluation.This paper aims to automatically detect and recognize students’ behavior and face by using machine learning algorithms,study the division of students’ classroom behavior status and facial expressions,and obtain students’ answer data information through the classroom interactive platform,and then fuse and analyze these data,and finally calculate the students’ learning evaluation results.