Research On The Key Technologies Of Digital Performance Evaluation

Digital performance evaluation,which uses digital multimedia and intelligent human computer interaction(HCI)technologies to establish novel,effective evaluation methods in performing arts,is a rising cross-disciplinary research field in recent years.According to the diversity of targeted subjects and the interactive models between them,current studies about digital performance evaluation usually lack systematized architecture and evaluation rules.Therefore,our research focuse on establishing affective recognition methods,detecting models,and evaluation feedback mechanisms,which had become the key points of digital performance evaluation.But,it also brings great challenges at the same time.From the perspective of cognitive psychology and intelligent interaction,this paper systematically studies:1)Evaluation methods based on different subjects in performing space;2)Affective detecting and evaluation methods for audience oriented in specific performing scenarios.This dissertation presents our study from the following four aspects:We construct an evaluation and feedback oriented Intelligent Performance Space(IPS).From the cognitive perspective,we divided IPS into three levels,"Behavior Level","Visceral Level",and "Empathy Level".On behavior level,we present a performer self-assessment method based on mixed reality.By using motion capture technology,the system tracks performer's motion image and overlap on real-world scenario,the results show that our method performed better especially on the aspect of rehearsal.On visceral level,we present an affective feedback mechanism based on audience physiological signals.Informantion transferring and performance content feedback are achieved through pattern recognition of physiological signals in different affective states.On empathy level,we present an inter-feedback mechanism,which is to determine the afrective correlation between audience and performer,and provide relevant feedback to improve affective similarity.We present an audience oriented affective recognition in response to performance videos based on "Arousal-Valence" emotional space.We create a multimodal performance database and employ an affective hightlighting algorithm to select performance videos with different affective grades as test materials.We collect EEG and peripheral physiological signals from audience and their subjective assessment.The two-dimensional affects are classified by usingGaussian Naive Bayes for single trail classification.Later,we use decision fusion to general new F1-scores.The results show that EEG frequency are significant correlate to subjective assessment,EEG and ECG achieve the best classification result on "Arousal" and "Valence"level,respectively.Also,decision fusion indicate better performance compared with single modality.We present an engagement detecting and evaluation method in live performance based on audience EEG signals.In this work,we first define audience live experience as engagement,which represents participation levels.We compare three EEG engagement indices and choose the best one.In order to establish the relation between audience engagement level and their affects,we divided the performance content into two stages.We calculate their "Arousal-Valence" values and generate an Engagement-Arousal Evaluation Rule(EAER).The results show that the engagement index could accurately separate audience experience according to the different performing stages.We found association between arousal level and engagement level,however there were no significant difference on valence level between the two stagesWe present an audience oriented engagement detecting and evaluation method in digital performance.In this work,we first bring a new question-Adaptive feedback in performing arts.We establish a Brain-Adaptive Digital Performance(BADP)system based on digital performance simulation platform.We present an Adaptive Thresholds Engagement Detection(ATED)algorithm to determine significant decreases on audience engagement level and employ performing cues as feedback.The results show that the ATED algorithm could successfully determine whether the audience was in "Less Interests" or "Stable Engagement" state.The performing cues provide positive effects,and audience in Multiple Performing Cues(MPC)condition have better recall ability than those in Single Performing Cues(SPC)condition.