Research On Learning Attention Recognition Based On Multimodal Data

Learning attention is an important factor affecting learning efficiency.In recent years,more and more attention has been paid to massive open online courses.MOOC learning,as a ubiquitous learning method,allows learners to learn anytime and anywhere without the limitation of time and space.But this kind of learning mode will also lead to the learners’ complex learning environment and difficulty in focusing.Recognizing learning attention in real time and effectively as the basis of adjusting teaching content and teaching strategies plays an important role in providing personalized learning feedback and improving learning efficiency.This study proposes a learning attention recognition method based on multi-modal features.This method uses physiological and image signals collected by sensor devices,and analyzes them by constructing CNN-LSTM hybrid network model.It can alleviate the problem of learning attention recognition accuracy in complex learning environment,and enhance the robustness and generalization ability of attention recognition method.The main research results are as followsFirstly,a feature extraction method based on the pulse wave signal of photo capacitor is proposed and attention recognition is carried out.The wearable sensor device is used to collect the learner’s optical capacitance pulse wave signal.Aiming at the high-dimensional,multi feature,complex redundancy and other characteristics of the signal,the time-domain,frequency-domain and nonlinear features are extracted,and the random forest is used for feature screening to accurately determine the relationship between data and features.The effectiveness and practicability of the method are verified by empirical experiments.Secondly,an attention recognition algorithm based on CNN-LSTM hybrid network is proposed.The deep learning algorithm is mainly composed of convolutional neural network,feature fusion layer and long-term and short-term memory network module.The CNN used has 5 convolution layers,2 pooling layers and 1 fully connected layer.The over fitting is suppressed by using dropout and batch normalization,which increases the training speed and generalization ability of the model.Then,the fused multimodal features are introduced into the long-term and short-term memory network module to enhance the recognition effect of the algorithm.Finally,an experiment of learning attention recognition based on multimodal data is designed for empirical verification.The depth camera,head mounted EEG sensor and smart wrist watch are used to noninvasively collect the depth image data,color image data,EEG data and photo capacitance pulse wave data of learners in MOOC learning.The image data of Yolov3 algorithm is used for target detection to remove the background to reduce the influence of complex image background on the attention recognition model.The CNNLSTM is used for depth detection The learning model trains the model with multi-modal data,and the prediction accuracy of the attention recognition model is 77.29%.Compared with the results based on single-mode features and different classification algorithms,the results show that the hybrid network model constructed in this study can effectively improve the accuracy of learning attention recognition.