Research On The Algorithm Of Fisher Linear Discriminant Analysis

Fisher linear discriminant analysis?FLDA?is a classical dimension reduction method,which can be summed up as the solution of generalized eigenvalue problem.However,the complexity of the solution of generalized eigenvalue problem is high and the workload is large.So in order to solve FLDA problem better,this paper proposes two new algorithms for solving FLDA problem and proves their convergence.The experimental results show that the two new algorithms are effective and feasible.The content structure of this paper is as follows:Through machine learning,common"dimension disaster"phenomena and related cases,we review several commonly used dimensionality reduction methods?such as principal component analysis and FLDA?.Finally,we summarize the research status of FLDA and propose a new algorithm for FLDA.By introducing the definition and theorem of eigenvalue and eigenvector,we introduce the related concepts and solution methods of generalized eigenvalue,and the related forms of Rayleigh quotient and fractional programming.Through the related definition of DC?Difference of Convex?function and theorems of DC programming,we introduce the DC algorithm proposed by Tao et al.^[37]Then,combined with DC,a new algorithm was proposed for FLDA,namely FLDA_DC.Through experiments on some data sets,we compare FLDA_DC with principal component analysis and the FLDA solved by generalized eigenvalue algorithm.For ratio optimization?or fractional programming?,we introduce proximal gradient descent?Proximal Gradient Descent,PGD?algorithm proposed by Radu et al.^[54]Then,combined with PGD,a new algorithm was proposed for FLDA,namely FLDA_PGD.Through experiments on some data sets,we compare FLDA_PGD with FLDA_DC,principal component analysis and the FLDA solved by generalized eigenvalue algorithm.We summarize the content of the full text,point out the insufficient aspects,and propose the planning for the next step of research.