A Study Of Subspace Analysis Approach For Pattern Discrimination

Subspace analysis is an important research direction of pattern Analysis. Subspaceanalysis technique has been successfully applied to computer vision and patternrecognition. According to the different ways of mapping, subspace analysis is dividedinto two major categories: linear and nonlinear. Linear subspace analysis methods havethe advantages of simplicity and high efficiency in computation, which provide a solidfoundation for a variety of non-linear methods. Nonlinear subspace analysis methodsfocus on extracting local structure of data. In recent years, subspace methods have beenwidely used in biometrics and biological data recognition. Currently, subspace analysishas become the key research direction of feature extraction, and it also has a very goodapplication prospects in data mining. In order to extract and effectively utilize validdiscriminative information contained in massive data, there is an urgent demand toresearch and develop new discriminant subspace analysis methods systematically anddeeply. This article started from the small sample size problem of Fisher lineardiscriminant analysis, conducted deep study on the subspace analysis method and madethe following innovative research results:1. A new linear discriminant subspace analysis method—Valid Discriminant NullSpace analysis is presented. The physical meanings of null-spaces of total scatter matrix,between-class scatter matrix, and within-class scatter matrix are explained and provedfor small sample size problem. The intuitive geometrical interpretations of thesenull-spaces are given. The relationship between inter-class scatter matrix on originalspace and inter-class scatter matrices on valid null-space and valid range space isrevealed. It is proved that intra-class distance on valid discriminant null space is zeroand its inter-class distance is greater than zero. The recognition experiments on differentdata testify that the valid discriminant null space analysis is superior to other relativemethods in terms of recognition accuracy, robustness and efficiency.2. A novel approach named pattern discriminant based on class null subspace analysisof range space is proposed. This method not only classifies samples of known classes,but also discovers new pattern class, which is not included in known classes. The natureof range space is studied. According to the function for pattern classification, spacedecomposition is performed on original space and resulted subspaces are analyzed. Theprojection matrices of each class are constructed according to properties of thesesubspaces. The material identification experiment based on THz-TDS signal and objectrecognition experiment based on COIL-100data show the superiority of the class null subspace analysis method.3. A weak space analysis method is proposed for two-class classification. The weakspace method aims to extract richer discriminant features than LDA. It is analyzed anddemonstrated with experiments that the complement space of Fisher’s discriminantspace contains valid discriminant information. The role of small variance components inpattern classification is discussed. It revealed that weak space contains great amount ofdiscriminant information. The method to constructe weak space is presented. Materialidentification experiment based on THz-TDS signal and experiment based on emulationdata validates that recognition rate could be improved on subspaces with rich feature.4. The weak component discriminant analysis is proposed. The physical meaning ofeigenvalue of sample covariance matrix is analyzed and revealed. The algebraicconnection between SNR of sample data and eigenvalues of sample covariance matrix isstudied and deduced. Accordingly, weak component is defined. The geometrical andphysical meaning of Fisher’s linear discriminant analysis is studied, which intuitivelyexplained heteroscedastic problem, seperability problem, and stability issue of LDA.New class discriminant criterion is presented by extending the idea of LDA from datapopulation to subclasses. And the weak component discriminant analysis algorithm(WCDA) is presented and upper bounds on classification error probability of WCDAare derived. WCDA and principal components null space analysis are compared inaspects of application condition, classification error, training data size andcomputational complexity. The proposed WCDA is also experimentally compared withrelated MLDA, PNBDA, and PCNSA on three different kinds of database. Theexperimental results verified that weak component discriminant analysis has goodclassification performance, and it can extract low-dimensional feature with goodconcentration in information space.5. The range space hyperspherical discriminant analysis is proposed. The unithyperspherical model is put forward and introduced into linear discriminant analysis. Itconverts Euclidean distance into length of an arc on unit hypersphere. It also bringsdiscriminant alalysis from Euclidean space onto hypersphere. The range spacediscriminant analysis is extended to nonlinear analysis by introducing kernel functionsand the range space hypersherical kernel discriminant analysis is presented.Classification experiments on different databases confirm the superiority of range spacehyperspherical discriminant analysis beyond classical LDA and its relatedgeneralization algorithms in aspects of training sample size, recognition accuracy, and computational efficiency.The results obtained in this paper have great theoretical significance andapplication value for subspace analysis method study in pattern recognition.