Research And Application Of Confidence Evaluation Method Of Psychological Scale Assessment Results Based On Multi-modal Physiological Signals

With the quickening pace of social life,mental health becomes one of the key issues.Rapid and accurate screening for mental illness is a prerequisite for solving this problem.Due to the advantages of low cost and high portability,psychological scale has become the most common screening means at the present.However,as an objective evaluation method,the polygraph mechanism still has shortcomings such as poor applicability,weak effectiveness,and long evaluation time.To solve this problem,this study proposes a confidence evaluation method of psychological scale assessment results based on multi-modal physiological signals which uses physiological signals(one or more of EEG,ECG,Eye movement data)to effectively detect lying answer behavior in the process of psychological scale evaluation,and finally generates more objective and accurate confidence of psychological scale assessment results.The research and implementation process consists of three parts,including dataset construction,research of key technology and system integration.The specific work is as follows:(1)To solve the problem of lack of relevant public dataset,we independently designed and constructed datasets for psychological scale assessment results confidence evaluation.We first designed and developed a physical and mental signal hardware and software acquisition framework before the data collection and we selected the test method for hidden information according to the weird ball paradigm theory,and then designed the psychological scale evaluation experiment.During the experiment,we strictly implemented the standardized procedure of cognitive experiment in psychology and recorded the experimental data in detail.Finally,we collected a total of 42 subjects in this dataset.After screening,the total number of physical and mental health data scale samples reached 320,equivalent to 12160 question samples,which met the requirements of datasets in this fields.(2)In order to verify the feasibility of each model physiological signal in the psychological scale assessment results evaluation environment,this thesis conducted a research on the question confidence evaluation based on the single model physiological signal.In the pre-processing stage,the comparison effect of each modal data before and after is obvious,and the noise processing work reaches the experimental expectation.During the feature extraction process,P300 components was observed on more than 95% EEG electrodes,and the availability of ECG signals and eye movement data was verified.In the comparison of single experimental features,the validity of the data set is proved and end-to-end method is determined to be the subsequent model construction method through the engineering comparison of feature.Finally,the optimal model was determined by comparing the six common machine learning models.The EEG signal had the best effect when using TSception deep learning framework to detect lying behavior,and the accuracy rate was up to 92.67%.(3)Research on confidence evaluation method of multimodal scale assessment results.In order to obtain higher confidence accuracy of question assessment results and robustness of the method,we analyzed the influence degree of each mode by data missing comparison experiment and proposed a Multimodal fusion in stages method from the perspective of fusion of feature layer and decision layer.In the feature layer fusion,we proposed neural network named Polygraph Net with high efficiency detection of lying signal fluctuation and attention mechanism fusion for ECG signal and eye movement data with low influence of missing.In decision level fusion,we used a method called “Trusted Multi-View Classification”(TMC)for EEG signals with significant influence of missing.Finally,the ablation experiment proved the effectiveness of Polygraph Net neural network and TMC method,and the accuracy of the Multimodal fusion in stages method could reach 96.32%while ensuring high noise resistance.The prediction results of the question assessment results confidence and the score of the lying detection factor of the psychological scale were combined for modeling.The experiment proved that the accuracy of the logistic regression classifier in the combined scale data could reach 93.59%.(4)Design and development of psychological scale assessment results confidence evaluation system.The whole work is divided into three parts: requirement analysis,system design and function development,and the psychological scale assessment results confidence evaluation method is integrated into it.In terms of function design,the system provides some information management services such as mental health scale evaluation,multimodal physiological signal detection,management of scale confidence assessment results enhancement report,and could generates a understandable psychological scale assessment results confidence enhancement report after the psychological evaluation.Through a large number of comparative experiments,this study proves that the confidence evaluation method of the psychological scale assessment results can replace the polygraph mechanism and help psychological practitioners obtain more comprehensive and accurate evaluation results.In addition,the design of this method fully considered the portability,cost,performance and other factors of physiological signal acquisition equipment,and provides a reference for the future development of psychological scale evaluation system through experiments.