Recognition Of Instantaneous Cognitive Task-Load Based On Physiological Features And Support Vector Machines

In many complex command and control systems, human operators and a machine (or a computer) are often integrated to accomplish a series of predefined tasks, which constitutes human-machine systems. However, the degradation of operator performance can significantly increase the risk of the occurrence of catastrophic accidents for safety-critical tasks. Due to technical limitations the complete lack of human control and/or supervision is still impractical. Hence, objective and accurate assessment of operator Cognitive Task-Load (CTL) is becoming the basis to avoid abnormal high operator effort and low vigilance level. Adaptive automation systems can be thus facilitated based on dynamical CTL level variation to allocate proper type and amount of tasks between the operator and machine and thus the operator performance can be optimized. Sine physiological measures provide continuous and objective assessment of both cognitive and functional state of operator, this study is aiming at using operator physiological data and pattern recognition methodology to automatically classify operator temporal CTL state into several discrete levels. The main contributions of the paper are listed as follows:(1) The experimental human-machine system for physiological data collection was built based on automation-enhanced Cabin Air Management System (aCAMS) software and a set of complex and safety-critical process control tasks were simulated. Two different experimental paradigms were designed with 13 healthy subjects participated. Subjects’ physiological data, task performance and subjective ratings were simoultanously collected during the tasks. The raw physiological data were properly preprocessed by using low-pass filter, power spectrum analysis, and artifact removal techniques. The multi-dimentional physiological features, performance indices, and subjective ratings were computed and the methods for automatic determination of CTL class labels were explored. All these data provide the basis for designing and evaluating the proposed CTL recognition techniques.(2) The recursive feature elimination (RFE) with least square support vector machine (LSSVM) are combined and used for binary and 3-class physiological feature selection and CTL classification. Besides typical binary LSSVM classifiers for two-class CTL assessment, the multiclass classifier based on 3-class LSSVM-RFE and decision directed acyclic graph (DDAG) scheme was developed. Feature selection results have revealed that different dimensions of CTL can be characterized by specific subset of physiological features. Performance comparison studies show that reasonable high and stable classification accuracy of the proposed classification framework can be achieved if the RFE procedure is properly implemented and utilized. This work has demonstrated the feasibility of using physiological features to indicate 2 or 3 level of CTL.(3) A 3-4 class CTL assessment technique with high accuracy based on adaptive exponential smoothing (AES) and adaptive bounded support vector machine (ABSVM) is proposed. Bassed on the previous work, the data smooth and adaptive classifier were used to improve the multiclass CTL classification performance. The AES technique is used to smooth the physiological data and to remove the strong artifacts without requiring their templates. The locality preservation projection (LPP) technique is utilized to derive the low-dimensional salient physiological features. By combining the AES-LPP and BSVM methods, the accuracy of the 3-class coarse-grained CTL classification was significantly improved by 11-13%. On the other hand, to perform the CTL classification under higher temporal resolution and with higher cross-subject and cross-session generalizability, finer-grained data analysis is conducted to recognize 3 or 4 levels of CTL based on a combination of Adaptive BSVM (ABSVM) and AES techniques. In comparison with the use of BSVM algorithm alone, significant performance improvement (by 10-20%) of the finer-grained CTL classification is achieved. This work demonstrated the effectiveness of adaptive physiological feature smoothing and classification scheme on the improvement of multi-class CTL recognition performance.(4) A 4-5 class ensemble CTL classifier is developed by combining Laplacian Eigenmap and heterogeneous ensemble support vector machines. Based on the previous work, the ensemble learning is adopted to enhance the subject-specific CTL recognition accuracy. The aim of this work is to find the relationship between the operator task performance and physiological features based on a combination of data clustering, feature reduction, and ensemble classification techniques. Five or four target levels of CTL are first automatically determined by using three different performance indices and a Gaussian mixture model. By using Laplacian-eigenmap-based feature reduction technique, a few most representative EEG features are extracted and combined with heart rate as the input features of the CTL classifier. Then, multiple support vector machines with heterogeneous structures are aggregated to form a classifier ensemble to recognize the CTL levels via majority voting approach. Finally, the results indicates the SVM ensemble with multi-kernels can improve the subject-specific CTL classification performance.(5) A simulation framework of adaptive human-machine system based on CTL dynamic classifier is developed. Based on the previous work, dynamic classification approach was used recognize 5-class CTL and its output was adopted as the basis for achieving adaptive human-machine task allocation. The starting point is the accurate recognition of CTL levels by using nonlinear dynamical pattern classifier. The object of the framework is to control the operator CTL and performance around the set point of safe state. In order to enhance the generalization capcacity for online CTL classifier, AES and LPP method were combined again to extract salient EEG features. The LSSVM with multi-kernels was adopted as a basis for building dynamic CTL classifier. The subject-specific dynamic LSSVM model is then used to predict the performance index and thus 5 discrete level of CTL at each time instant can be recognized. The elicited performance index and CTL level is then adopted as the basis for a simulated control system to reallocate the tasks between humanistic and machine agent. It has been validated the promising improvement of the operator performance was achieved when proper human-machine task allocation scheme was employed.