Generalized Regularization And Sparse Regression For Image Feature Extraction

Feature extraction is one of the most fundamental problems in pattern recognition tasks,and the principle component analysis（PCA）,linear discriminative analysis（LDA）,ridge regression（RR）and their extensions are simple but effective methods in terms of feature extraction.However,since these methods focus on the global structure of the data sets and their optimal solution（projection）is a linear combination of the data,they do not preserve the local geometric structure of the data.Even though the methods based on the local geometric structure of the data are able to preserve the intrinsic low dimensional manifold embedding of the data,they still have some drawbacks.Therefore,in this paper,we conclude four main problems of the existing methods based on global/local structure information of the data.First,both of RR,LDA and their extensions have the small-class problem,which means that the number of the projections learned by these methods is limited by the number of class.Therefore,they cannot obtain enough projections for effective feature extraction,especially on the occasion when the number of class is very small.Second,even though some of these methods can obtain the sparsity for feature extraction by using L₁-norm penalty on the regularization term,they can neither obtain jointly sparse projections nor provide joint semantic interpretation.Third,these methods using L₂-norm as the basic measurement on the loss function are sensitive to outliers,especially when the images are corrupted by noise or the data are imbalanced.Therefore,they cannot guarantee the robustness for feature extraction.Fourth,since these methods do not consider the generalized orthogonality that related to the locality information of the data,the effectiveness of the learned subspace cannot be guaranteed.To solve the above four problems,in this paper,we propose a set of methods so as to release the small-class problem and to enhance the robustness as well as guarantee the joint sparsity and the generalized orthogonality of the projections.That is,we first propose a method called jointly sparse locality regression（JSLR）to consider the local structure of the data in regression form,and at the same time solve the small-class problem as well as obtain jointly sparse projections for discriminative feature extraction and selection.Second,in order to enhance the robustness,we propose a method called generalized robust regression（GRR）for robust feature extraction.Finally,we consider the generalized orthogonality of the projections and propose a method called robust jointly sparse regression（RJSR）to obtain generalized orthogonal projections for effective feature extraction.These proposed JSLR,GRR and RJSR gradually solve the above four problems so as to release the drawbacks of the existing linear feature extraction methods.This paper also provides the corresponding theoretical analysis of the proposed methods,including the proof of the convergence and the computational complexity.A series of experiments,including experiments on face databases,numerical or character databases and hyperspectral image databases,are performed to evaluate the robustness and effectiveness of the proposed methods in terms of feature extraction and selection,especially in the case when images are corrupted by noisy or occlusion.All the theoretical analysis and experimental results show that the proposed methods not only solve the small-class problem,but also obtain better performance under the large-sample case.In addition,they are more robust than most of the comparative methods.