Label Enhancement In Machine Learning: Theories And Applications

Learning with ambiguity is a hot topic in recent machine learning research.Multi-label learning(MLL)studies the problem where each example is represented by a single instance while associated with a set of logical labels.However,in most practical learning tasks,the relative importance among the relevant labels is more likely to be different rather than exactly equal.On the other hand,the irrelevance of each irrelevant label may be very different.The bipartite partition of the label set into relevant and irrelevant labels in MLL is actually a simplification of the real problem.Therefore,the label distribution is essential for the ambiguity at the label side,since the label distribution represents the degrees to which each label describes the instance.The learning process on the instances labeled by label distributions is called label distribution learning(LDL).Unfortunately,many training sets only contain simple logical labels rather than label distributions due to the difficulty of obtaining the label distributions directly.To solve this problem,we propose a new concept called label enhancement.Label enhancement is to recover the label distributions from the logical labels in the training set via leveraging the topological information of the feature space and the correlation among the labels.This dissertation studies several important issues on label enhancement,and the main results are summarized as follows.A theoretical framework of label enhancement is constructed.The theoretical framework answers three essential questions in label enhancement.Firstly,“ where does the labelingimportance come from”,i.e.,the generative mechanism of label distribution is proposed.Secondly,“how to evaluate the results of label enhancement”,i.e.an evaluation method is proposed.Thirdly,“why label enhancement works ”,i.e.,the effectiveness of label enhancement for subsequent classification is proved theoretically.The three answers are validated theoretically and empirically.A label enhancement algorithm for LDL is proposed.It is important to propose specially label enhancement algorithm for LDL,and the key issue is designing the target function to mine the implicit labeling information.We propose a novel label enhancement algorithm called Graph Laplacian Label Enhancement(GLLE).GLLE recovers the label distributions from the logical labels in the training set via leveraging the topological information of the feature space and the correlation among the labels.Experimental results validate the effectiveness of LDL based on GLLE.Label enhancement is applied to other machine learning paradigms.Label Enhanced Multi-Label Learning(LEMLL)is proposed to solve multi-label learning problem.LEMLL integrates label enhancement and classification into one learning target,which can help training the classifier under the reinforced supervision information.Partial label learning via label enhancement(PLLE)is proposed to solve partial-label learning problem.PLLE recovers the description degrees of candidate labels and non-candidate labels by label enhancement,and transfers the partial-label leaning problem into multi-regression problem.The experimental studies validate the advantage of LEMLL and PLLE.