The Effect Of EEG Artifacts Caused By Micro-expression On Its Recognition Performance

Micro-expressions,as a brief emotional expression produced in an excitation-inhibition antagonistic state,can reflect a person’s genuine emotional state that he or she is trying to suppress or hide.Recently,along with the advancement of technology in computer vision,micro-expression recognition is already applied extensively in lie detection,justice and national security.Currently,research on micro-expression recognition based on electroencephalography(EEG)signals has also attracted extensive attention.However,in the micro-expression recognition task based on EEG signals,artifacts such as facial muscle activity can interfere with EEG signals and even mask the power spectrum information of EEG signals,increasing the difficulty of micro-expression recognition tasks based on EEG signals.Thus,Thus,it is very important to evaluate the effect of artifacts caused by micro-expressions on the EEG signal and micro-expression recognition performance.At present,most existing studies have found that the artifacts have a significant effect on the EEG signal.For example,in the study of macro-expressions,it was found that artifacts significantly affect EEG signals,and consequently affect the emotion recognition performance.Compared with macro-expressions,micro-expressions are weak and short-term.Therefore,it is speculated that there may be differences in the effects of artifacts caused by micro-expressions on EEG signals compared to macro-expressions.However,the effects of artifacts caused by micro-expressions on EEG signals are still unknown in micro-expression studies,and no studies have yet investigate the effect of EEG artifacts caused by micro-expressions on recognition performance in micro-expression studies.Therefore,in order to investigate the effect of artifacts caused by micro-expressions on EEG signals,and whether the micro-expression recognition performance based on EEG signals can be improved when artifacts are removed,the paper combined facial images and EEG signals,and used statistical analysis and machine learning methods to investigated the effect of EEG artifacts caused by micro-expression on recognition performance.The major contents and work of this paper are listed below:1.The micro-expression elicitation experiment paradigm was designed,and the Southwest University Micro-expression Database(SWUME)was established.First,a total of 68 right-handed healthy participants were recruited for this experiment,and 7highly entertaining emotional video clips were selected to evoke micro-expressions,and the EEG signals and facial image data were collected.Then,the collected signals were preprocessed,and constructed the SWUME dataset.The dataset consists of 393macro-expression samples and 806 micro-expression samples,which provided data support for the subsequent study.2.The effect of artifacts caused by micro-expression on EEG signal is studied.Specifically,it contains: analyzed the correlation between facial micro-expressions and EEG signals,explored the degree and mode of effect of artifacts caused by micro-expressions on EEG signals,and compared the difference of effect on EEG signals before and after artifact removed.The process is divided into two stages: feature extraction and statistical analysis.Firstly,the feature extraction stage is performed on the pre-processed facial micro-expression sequences and EEG signals(without artifact removed and with artifact removed),respectively.Second,in the statistical analysis stage,the effects of artifacts on EEG signals were investigated using multiple linear regression analysis methods and Granger causality tests.The results showed that the average percentage of EEG signal affected by artifacts caused by micro-expressions was11.5% when artifacts were not removed from the EEG signal.Specifically,compared to other brain regions,the frontal and temporal lobes were significantly affected.Compared to other facial regions,muscle activity in the eyebrow and mouth regions had a stronger effect on the EEG signal,with up to 15%.Finally,when artifacts were removed from the EEG signal,the effect of artifacts on the EEG signal was able to be significantly reduced,with the average percentage of this effect decreased to 3.7% and a total decrease of 7.8%,with only the frontal regions being slightly affected.Furthermore,we validated the method by analyzing the correlation between macro-expressions and EEG signals.3.A micro-expression recognition model is constructed to explore whether the performance of micro-expression recognition based on EEG signal can be improved after artifact removed.The EEG features without artifact removed and the EEG features with artifact removed were used as the input of three classifier models,Support Vector Machine(SVM),K-Nearest Neighbor(KNN)and Decision Tree(DT),respectively,and the classification tests were performed using a 10-fold cross-validation method.The comparison of experimental results showed that the recognition accuracy of micro-expression samples after artifact removed were all significantly improved compared with that without artifact removed.Among them,the average recognition accuracy improved by 4.17% for the SVM classifier,3.32% for the DT classifier,4.31%for the KNN classifier,and the highest recognition accuracy(84.26%)was obtained by using the KNN classifier.In summary,the paper firstly designed the micro-expression elicitation experimental paradigm to elicit participants’ micro-expressions under positive emotions and established the SWUME dataset.Then,the effect of EEG artifacts caused by micro-expression on recognition performance was taken as a starting point to systematically analyzed the effect of artifacts caused by micro-expressions on EEG signals,compared the effect results without artifact removed and with artifact removed to verify the stability of micro-expression recognition results based on EEG signals.Finally,the micro-expression recognition model was constructed on the basis of Work 2.It is found that the artifacts caused by micro-expressions have significant effects on the EEG signal and recognition performance,and artifact removed can significantly improve the signal-to-noise ratio of EEG signal and micro-expression recognition performance.This indicates that in the process of micro-expression recognition based on EEG signals,EEG signals used after artifact removed are mainly generated by brain neural activity rather than artifact signals,and EEG signals are minimally affected by artifacts.The research results of the paper demonstrate the reliability of the research on micro-expression recognition based on EEG signals,and provide important reliable evidence support for future research on micro-expression recognition based on EEG signals.