Flow Experience Identification In Online Instructional Video Learning

Currently,online video teaching has become one of the main instruction methods,the researchers are paying more attention to students’mental states and behaviors during video learning.Among the different mental states,flow experience is what students aim to achieve during the process of online learning.It is a complex,multidimensional,reflective psychological construct characterized by a pleasant state of concentration during challenging activities.During video learning,the flow experience can lead to deeper learning and a higher level of personal satisfaction for students.This means that effectively identifying the flow experience of students during video learning is especially important for revealing students’mental state in specific learning activities as well as helping teaching staffs design better learning materials and instruction methods.However,the evaluation of flow experience in previous studies mainly relied on the subjective methods such as scale measurements and interviews,and they have not explored the use of artificial intelligence technology to automatically identify the existence of flow during interaction between learners and online video teaching.In this study,we used multimodal learning analysis（MMLA）to identify and predict students’flow experience in video learning,by collecting synchronized multi-stream data（i.e.,behavioral,physiological and expression features）and using machine learning methods to fuse multimodal data.This paper includes two researches.The research 1（i.e.,Experiment 1）induced high and low flow experience levels by watching learning videos of different popularity,established classification models based on physiological indicators and posture indicators,and used meaningful objective metrics to predict the students’two different levels of flow experience during video learning.In view of the fact that the binary classification of flow experience could not completely match the learning states in actual instructional scenario,the research 2 established a three-classification scenario of learning states in video learning（i.e.,boredom,anxiety and flow experience）according to the challenge-skill balance and the four-channel model of flow theory,and identified these three learning states.The research 2 included two experiments（i.e.,Experiment 2 and Experiment 3）.Experiment 2 established video learning materials that successfully induced boredom,anxiety and flow experience through video preliminary screening,expert evaluation,screening index calculation and participant pre-evaluation.Experiment 3 collected multimodal data（physiological,posture,facial and speech signals）during students’video learning,extracted relevant multimodal features,made classification recognition of the three learning states and regression prediction of the flow experience scores through applying data mining and multimodal fusion technology.Results:（1）Students accompanied by higher time-domain heart rate variability,lower high-frequency heart rate variability and less degree of head shaking in the high flow experience.Based on the physiological and posture data,the recognition accuracy of the model can reach up to 81.6%.（2）This study proved that the decision-level fusion of multimodal data could significantly improve the recognition ability of the three learning states,and the recognition accuracy of boredom,flow experience and anxiety states could reach47.48%,80.89%and 47.41%,respectively.（3）While the learning state of students changed from boredom to flow experience to anxiety,the physiological indicators SDNN,HF and speech features MFCC₁₅ and MFCC₅ changed the trend of approximately the same.In summary,this study successfully identified students’mental states during online video learning by collecting student-generated multimodal data（physiological,posture,facial and speech signals）.The results of this study contributed meaningful objective indicators of multimodal data to the automatic identification of teaching-related mental states,and made a preliminary exploration for the future intelligent understanding and recognition of students’flow experience in specific learning activities.