The Outlier Detection Of Chemical Data And Application

In many fields such as chemical engineering, biomedical engineering, and so on, many internal mechanisms of objects subject to research can hardly be recognized. However, observation data, which reflect the values of dependent variables changing with respect to the corresponding values of independent variables, can be obtained through experiments, and then be used to model the involved object, which is qualitatively describe the relationships between the dependent and independent variables. This will serve as another form of representing their internal mechanism. This is a kind of modeling by learning from observation data, which is one of the basic important tasks for researchers.Accordingly, the detection of outlier is very important in the chemical field which emphasizes the experimentation and data collection. In the paper, the theory foundation, need of application, research thinking, key to technology and development of the detection of outlier are analyzed comprehensively, combined with the characteristic of chemical data. According to the linear and non-linear system, it introduces the current detection solution briefly, including detection criterions, the ways of direct and non-direct, single and non-single detection, classic and modern means. The thesis applies the support vector machine (SVM), with the thinking of robust modeling and the characteristic of radical basis function nets, to outlier detection. It is proved that the method performs effectively and it has own advantage. The main contributions of this dissertation are as follows:1) The support vector linear regression is used to detect the outlier in linear modeling. The linear regression is the most commonly used in linear modeling, whose hypostasis is the least square method. However, the least square is very sensitive to the outlier, which can spoil the model severely. The existence of sensitivity area in support vector linear regression makes it feasible to detect the outlier. The work makes use of the method to improve the robustness of linear regression modeling and to detect the outlier effectively.2) The support vector machine (SVM), with radical basis function nets (RBF), is proposed to detect the outlier in the nonlinear chemical modeling. Both SVM and RBF are the powerful tool of the modeling, but they have advantage and disadvantage respectively. SVM can fit the data accurately, and its structure complexity depends on the number of support vectors. Furthermore, its optimization problem can get the global optimization through the Quadratic Programming, which avoids getting into the local optimization. It has huge advantage compared with the other neural networks. But, the choice of parameter of SVM depends on the experience. However, the accuracy of the model is related to the parameter. RBF is also a good method of modeling, which self-regulates the parameters through the learning. But its initial structure is difficult to get. Especially, the number of the nods of latent layer is difficult to decide. The work combines the SVM and RBF .The initial structure is decided by SVM, and then the parameters are self-regulated through the self-learning ability of RBF. Both of them compensate each other, and the method performs effectively.The thesis applies the robust modeling to the learning of RBF mentioned above, which improves the robustness of model to the outlier. The traditional learning of RBF is to attain the net's parameters by the least square, whose target function is the error squared sum. But, the function is very sensitive to the outlier. So, the robust modeling is proposed to get the parameters of RBF.In a word, broad and deep investigation on the detection of outlier in chemical modeling are made in this dissertation, which affords us new approaches to the chemical robust modeling. The conclusions and the further development directions are also given in the last part of the work.