Subspace Regression Analysis And Its Algorithmic Applications

Subspace regression leads to a series of associated problems by analyzing the sta-tistical properties of the data.The core purpose of subspace regression is to obtain the corresponding subspace representation of the data via the regression model.Due to the excellent statistical and global characteristics,the subspace regression problem serves as a pivotal role in the fields of machine learning and data mining such as low-rank reconstruc-tion of data,principal component analysis,embedded feature selection,semi-supervised learning,unsupervised learning and so on.Since orthogonal subspace has the property to preserve the manifold structure,this dissertation specially focuses on the orthogonal subspace representation of the data.Al-though the orthogonal subspace performs with manifold structure,it is frequently re-frained from the non-convexity of the orthogonal constraint such that constrained regres-sion problem becomes tricky to solve or to find the corresponding optimal solution.To address the issues previously mentioned,we first propose an adaptive framework by intro-ducing novel adaptive models and algorithms such that robustness,scaling and manifold structure of orthogonal subspace regression models are taken into account.Accordingly,the overall performance of the regression model can be strengthened via the proposed adaptive framework.Equipped with the related improvements,the modified models are further applied to the data reconstruction,classifier,spectral clustering and sparse feature selection,respectively.To sum up,the dissertation focuses on two-dimensional(2D)principal component analysis,subspace reconstruction classifier,subspace clustering problem,graph-based semi-supervised learning and embedded feature selection with putting forward a series of new algorithms:1.To tackle the high-dimensional problem caused by vectorization of the input im-age data,2D principal component analysis(2DPCA)is proposed such that image data can be directly processed instead of conventional vectorization technique.Consequently,the original image can be directly analyzed via 2DPCA so that the existence of high-dimensional vectorization problem can be prevented.In addition to large-scale savings of computational time,the accuracy of data classification is improved.However,as for the noised data,traditional 2DPCA method is sensitive to the outliers.To enhance the robustness of traditional 2DPCA,we utilize the proposed adaptive framework to mod-ify the traditional 2DPCA model.Consequently,two novel robust 2DPCA methods are eventually derived and proposed.2.The data reconstruction is to recover the original data with high quality via low rank subspace.However,utilization rate of reconstruction model is low for dealing with label information.In other words,although the reconstruction problem can be used to optimally approximate to the original data,the reconstruction problem often leads to un-satisfactory classification results due to its inability to utilize label information.To address this problem,a subspace reconstruction classifier is proposed,such that label information and reconstruction problem are combined simultaneously.Accordingly,subspace recon-struction classifier can not only preserve the statistical properties of the data but also utilize the label information for superior classification results.3.Semi-supervised learning is divided into two categories:The first one is the classi-fication problem with the existing labeled data;The second one is the clustering problem with side information.We utilize the proposed adaptive model to modify and improve both two types of semi-supervised problems.As for unsupervised learning,we mainly focus on the clustering problem of data under the guidance of orthogonal subspace.4.Embedded feature selection specifically refers to the case that feature selection is embedded in the subspace problems.This paper proposes two novel adaptive sparse dimensionality reduction models via the adaptive framework and then applies them to addressing the embedded feature selection.