Dimensionality Reduction Method In High-dimensional Data Analysis

In this paper we are concerned with the dimension reduction of high-dimensional data. In the first and second section of Chapter 1 we introduce the model of dimension reduction problem, put forward the concepts of dimension-reduction function and embedding function, and make a classification for the dimension reduction problem; in section 1.3 we discuss "the curse of dimension" and the sparsity of high-dimensional space; in section 1.4 we discuss "intrinsic dimension" and its estimation based on the model of dimension reduction.Chapter 2 is a review of dimension reduction techniques. Firstly we give the linear and nonlinear classification of dimension reduction technique according to the linearity of dimension reduction function, then on the base of the basic model and the classification we discuss some commonly-used dimension reduction techniques such as principle component analysis( PCA), principle curve\surface, Kohonen's self-organized mapping and density network. We give special attention to the virtues and disadvantages of PCA.In Chapter 3 we analyze a useful statistical method-projection pursuit(PP), including its basic theory and application. We put out definitions of projection index and projection pursuit, then prove that PCA is a special case of PP with sample variance as projection index and list some indices often used. In section 3.3 we apply PP with information divergence index to hyperspectral image, and demonstrate the advanced ability of it by the comparison between it and PCA.Chapter 4 is the main part of this paper. In this chapter we have a deep study on a new technique for nonlinear dimension reduction-locally linear embedding(LLE). In section 4.2 we analyze its main idea and algorithm in detail, two relevant theorems included; section 4.3 provides plenty instances so to explain its nonlinear dimension reduction ability, section 4.4 propose a combined method that integrates the advantage of various methods. In section 4.5 we analyze some significant problems in LLE, including the locality of manifold representation, the choice of the neighborhood, the intrinsic dimension estimation and the parametric representation of mapping. In section 4.6 we design an algorithm for estimating the intrinsic dimension in the base of locally linear approximation and discuss the choice of its parameters. In last section we propose a new method for image classification and recognition, and the result of experiment shows that the method is effective with classification accuracy of 96.67%.The main creative points in this paper are: propose the concepts of dimension-reduction function and embedding function, define the projection index in term of linear operator and prove two relevant theorems; design a method to estimate the intrinsic dimension; put forward an classification algorithm based on LLE.