Research On Active Learning Algorithms For Insufficient Labeled High-Dimensional Data

Learning from massive high-dimensional data with scarce labeled samples is a fundamental issue in machine learning.As there exists lots of irrelevant and noisy features in high-dimensional data,feature selection have become an indispensable step in preprocessing,and it aims to select relevant and discriminative features while eliminating noisy and irrelevant features simultaneously from original feature space.Existing feature selection methods more or less ignore some critical discriminative information such as local geometric structure consistency or fail to establish the redundancy between features,thus it is necessary to design effective feature selection strategies to select profitable features.Since the labeled samples are scarce and unlabeled examples are plentiful,and manually labeling a decent training set is extraordinarily labor-intensive,time-consuming and errorprone.Toward this end,active learning has been proposed to alleviate the problem of scarcity of training samples.Active learning is devoted to selecting less but maximallyinformative unlabeled examples from unlabeled pool and querying their ground truth annotations for next round of supervised training,at the same time the annotating cost is saved as much as possible.Current active learning methods are always characterized by “scarce training examples”,“sampling criteria weights are fixed” and “considerable distribution bias between labeled and unlabeled data”.To address such problems,this thesis starts with the issue of data pre-processing,and focus on the two basic topics of feature selection and active learning.The main contributions include:(1)We propose a novel unsupervised feature selection method which combines local geometric structure consistency and redundancy minimization together for feature selection.Existing unsupervised feature selection methods always ignore critical discriminative information such as local geometric structure consistency or fail to formulate the redundancy between features,which may result in redundant feature selection.Toward this end,this thesis integrates two critical discriminative information to perform feature selection:(a)Exploring the local geometric structure consistency between original feature space and selected feature space to select discriminative and relevant features;(b)Formulating the similarities between feature pairs via maximal information coefficient to minimize the redundancy of the selected subset.After that,the above two kinds of discriminative information are integrated into an unified framework to select discriminative and relevant features while eliminating redundant and irrelevant features.Besides,an efficient iterative optimization algorithm is designed to obtain the solution of the unsupervised feature selection framework,and the convergence of the optimization algorithm has been proved theoretically.The features selected by the proposed unsupervised feature selection framework achieve substantial performance improvements on clustering tasks.(2)We propose a novel adaptive criteria weights batch mode example sampling method for batch mode active learning.Current batch mode active learning methods always select samples by assigning a fixed weight to each sampling criterion,and the fluctuations of sampling criteria are ignored during active learning procedure,which may result in suboptimal sample selection.To address this issue,the example sampling criteria including(un)certainty,representativeness and diversity are first constructed,and we integrate these sampling criteria into an adaptive framework,following which the solution is obtained by a Lagrange Multiplier method.The proposed framework dynamically adjusts the importance of(un)certainty,representativeness and diversity to select the most critical unlabeled samples for training a reliable classifier.On one hand,the samples identified by the proposed algorithm will definitely facilitate the learning of the classifier;On the other hand,by leveraging the learning mechanism to update criteria weights adaptively,the proposed algorithm is more suitable for BMAL under dynamic changing batch mode active learning environment.(3)We propose a submodular function to formulate the similarities between the candidate examples,thus the adaptive weight criteria selection algorithm developed in previous chapter is improved and extended.To be specific,the proposed method consists of two critical phases: First,the adaptive criteria weight sampling framework is utilized to select a batch of valuable unlabeled examples.Then,we resort to a submodular function to identify a mini-batch from the selected batch,thereby the redundancy of the mini-batch is explicitly controlled.The adaptive sampling criteria re-weighting mechanism adopted by the proposed method not only makes the sample selection framework more flexible,but also the redundant control function can significantly improve the robustness of the learning model.Furthermore,the improved adaptive batch sample selection algorithm is extended to the essential semi-supervised classification and clustering tasks,and promising experimental results demonstrate that the proposed algorithm achieves the best classification and clustering performance on twelve benchmark datasets compared to the competitive baselines.(4)We propose the uncertainty and representativeness criteria for active deep image classification in a convolutional neural network.Most of the current deep active learning methods tend to select only uncertain images to train the classifiers,and the performance of active learning methods are easily affected by the distribution mismatch between labeled and unlabeled data.To overcome the above limitations,we propose to explore uncertainty and representativeness of the unlabeled examples in deep active learning,aiming to train a reliable neural network for deep image classification.Uncertainty is learned by maximizing the prediction discrepancies of two additional introduced adversarial classifiers,while the features of labeled and unlabeled data are aligned as much as possible.After that,the(n+1)-tuplet loss,which was developed in deep metric learning,is imposed on the CNN backbone,to force CNN to extract discriminative featrues of unlabeled images.Subsequently,the representativeness is defined by the distance proportion between the given example to its own centroid and the given example to all cluster centroids.The most valuable unlabled images will be identified for active supervision via the combination of the learned uncertainty and representativeness.Extensive promising image classification results demonstrate the superior performance of our proposed deep active framework over other state-of-the-arts active learning approaches on three benchmark datasets.