Joint Methods For Large-scale Discriminant Analysis

Linear discriminant analysis(LDA)is a classical supervised dimension reduction method,which has been widely used in machine learning,image retrieval and other fields.Spectral regression discriminant analysis(SRDA)is one of the most effective dimensionality reduction algorithms for large-scale sparse discriminant analysis.It is a stepwise algorithm: first,the response vector is obtained from solving an eigenvalue problem,and then the projection vector is computed by solving a least squares problem for the response vector.But SRDA is solved step by step may lose useful information.In this work,we propose two new models and corresponding fast algorithms.The first is Joint spectral regression discriminant analysis(Joint SRDA).In this algorithm,we propose a unified framework to compute both the response matrix and the projection matrix.By using this method,one can extract the discriminant information of classification tasks effectively.The second is the adaptive parameter Joint spectral regression discriminant analysis(Joint SRDA-PF).In this algorithm,we select the balance parameters adaptively,In addition to Joint SRDA algorithm,we also improve the selection of initial values within the Joint principal component and discriminant analysis algorithm(JPCDA),and propose an Improved joint principal component and discriminant analysis algorithm(IJPCDA).In order to illustrate the effectiveness of the proposed method,we conduct a large number of experiments on ten popular data sets.The experimental results show that the proposed method has better classification performance than the SRDA algorithm.