Collaborative Representation-based Subspace Learning Methods

Dealing with massive high-dimensional data has become a necessary processing in scientific research,due to the rapid development of data collection and storage techniques.Therefore,it is particularly important to learn simple and effective low-dimensional features from complex high-dimensional data.Among many feature extraction methods,subspace learning methods are the most important one whose core is how to design the objective function to obtain the desired feature subspace.After obtaining the lowdimensional features,the selection of classification method is also very important.The reason is that different classification methods have different characteristics,they will show different classification ability when using the same features.In recent years,the representation-based classification methods become one of the research topics in the field of computer vision and pattern recognition.However,the common feature extraction methods are not suitable for these classification methods who have a special assumption on features.Therefore,it is worth to consider how to find more suitable features for these classification methods.In this thesis,we think that feature extraction method and classification method are not independent,but interdependent.Therefore,we regard collaborative representation-based classification(CRC)as the research object,and propose several collaborative representation-based subspace learning algorithms to overcome the shortcomings of existing related methods.The main contributions of this thesis are as follows:(1)In CRC,it is assumed that data satisfies the subspace assumption,which may not be practical in real scenes.To address this problem,we propose a classification method termed multiple kernel and local constraints collaborative representation-based classification method(MKLCRC).In MKLCRC,we utilize local linear constraints to enhance contribution of local samples in representation and improve discriminability of collaborative representation.Then,Inspired by the idea of Multiple Kernel Learning for Dimensionality Reduction,we integrate multiple kernel learning and feature extraction into a unified framework to make data satisfy the subspace assumption in feature subspace.In order to match feature subspace with MKLCRC,a feature extraction method is proposed,named multiple kernel locality-constrained collaborative representation-based discriminant projection method(MKLCR-DP).MKLCR-DP designs the objective function according to the subspace assumption in MKLCRC,so that data has the minimum reconstruction residual within class and the maximum reconstruction residual between classes feature subspace.When the objective function of MKLCR-DP is optimized,a trace difference optimization method is adopted.Experimental results on different face databases verify effectiveness of the proposed method.(2)CRC assumes that training and test samples have the same distribution.When the distribution is inconsistent,classification ability of CRC will be reduced seriously.Considering that the difference of distribution will lead to a weakly correlation among features,we propose a domain adaptation collaborative representation-based classification method(DACRC)to capture the potential information sharing among samples of the same class in different distributions.Therefore,DACRC uses a set of projection matrices to eliminate differences between distributions,then projects features with different distributions into a shared feature subspace where a shared dictionary is used to capture the potential shared information.In order to learn projection matrices and shared dictionary which are most helpful to DACRC,we then propose a feature extraction method called jointly discriminative projection and dictionary learning for domain adaptive collaborative representation-based classification method(JD2-CRC),which designed the objective function according to the class reconstruction residual of DACRC.In JD2-CRC,the optimized variables are decoupled by utilizing the closed form of collaborative representation.Then a gradient descent method is used for optimization.Extensive experiments demonstrate superiority of the proposed method.(3)When calculating projection matrices,the existed methods only consider discriminability of the collaborative representation in CRC,but ignore class specificity in the classification rule of CRC.It is ignored that different classes have different requirements for feature subspace,so the obtained global projection matrix is not optimal for some classes.Hence,we propose a class-specific collaborative representationbased classification method(CCRC).CCRC considers that each class corresponds to a feature subspace,in which it is easy to distinguish whether new samples belong to this class.In order to find a set of projection matrices which are most suitable for CCRC,we propose a feature extraction method termed multiple discriminant analysis for collaborative representation-based classification method(MDA-CRC).MDA-CRC regards calculation of a projection matrix as a binary classification problem,and considers that each projection matrix should only consider the within-class and between-class relationships of the corresponding class.In order to reduce time consuming,an accelerated trace difference optimization algorithm is used to solve the projection matrices.The experimental results verify effectiveness of the multiple subspace learning strategy.(4)we propose a robust margin collaborative representation-based classification method(RMCRC)to solve the problem that the classification performance of the existing related methods is limited due to cannot use all training samples effectively when the training set is large.Inspired by the mechanism of collaborative representation,we find that robust marginal samples play an important role in collaborative representation.Therefore,we use the robust marginal samples instead of the original training set in RMCRC which eliminates the redundancy between samples while maintaining the representation ability of the original training set.Considering that classification ability of RMCRC depends on the quality of robust marginal samples and the subspace assumption,we further propose a margin embedding net(MEN).In MEN,we use a generative model to generate virtual marginal samples,which makes the robust marginal sample set more reliable for RMCRC.In order to ensure that the subspace assumption is satisfied in the feature subspace,an embedded network with triple loss is used with the proposed collaborative representation-based triplet mining method.Experiments on several datasets verify the effectiveness of the proposed method.