Research On Theory And Algorithm Of Propagation Information Bottleneck

Information bottleneck(IB)method aims to compress the input variable to a "bottleneck" variable to the greatest extent,while preserving the external relevant information as much as possible,so that latent patterns in the data can be well explored.The traditional IB method on data pattern analysis has two main characteristics:1)data usually comes from a single source with the same data distribution;2)enough data samples are needed to ensure better data patterns,because it only uses the inherent information of the data.However,with the advent of the era of Big Data,data often presents the characteristics of multi-source,high-dimensional,heterogeneous,etc.,which bring great challenges to traditional IB methods,i.e.,how to effectively mine the correlations among multi-source heterogeneous data?How to exploit the relationship between multiple sources of data to learn better data patterns?To tackle the challenges faced by traditional IB methods,this thesis proposes a theoretical framework for propagation information bottleneck(PIB)and specific properties and problem-oriented algorithms.Specifically,the theoretical framework of PIB and the corresponding analysis of its properties are first given,then we propose a specific correlation propagation based information bottleneck theory,theoretically analyze the condition and properties of correlation propagation and verify the effectiveness of the proposed theory on multi-domain image dataset.Finally,the research on PIB algorithms is carried out from three aspects:specificity and correlated information fusion,auto-weight learning,and dual-correlated information mining and propagation.The main research results are as follows:(1)To deal with the clustering analysis problem of multi-source heterogeneous data,we first define the propagation information bottleneck theory and investigate the properties.Then,under the constraint of multi-task learning on multi-source heterogeneous data,we propose a specific correlation propagation based information bottleneck theory,which aims to perform multiple image clustering tasks simultaneously and improve the clustering performance of each task by propagating the correlations between tasks.Specifically,we first use the IB method to cluster each task individually.Then,we find the relevant and irrelevant images between any pair of clusters between different tasks by designing an effective correlation propagation mechanism.so that the two corresponding relationships are propagated between tasks.However,only the positive relationship is used to improve the clustering performance,while the negative relationship is dropped to avoid the occurrence of "negative transfer".In the meanwhile,we theoretically analyze the condition and properties of correlation propagation.Finally,a new sequential and collaborative method is introduced to solve the problem and then verify the effectiveness of the proposed theory on several multi-domain image datasets.(2)To solve the problem of underutilization of the unique information and a wealth of shared information between views in multi-view human action video data,we propose a specific and common information bottleneck(SCIB)algorithm to simultaneously consider the comprehensive contribution of both kinds of information.First,we design a shared bag-of-words model to discover the view-shared visual words,which contains the consistency information between views.Furthermore,we obtain a discriminative shared feature representation,in which the related information between various views can be propagated.Then,we use both the view-specific information and the common information between the views to improve action recognition performance.The proposed algorithm formalizes the problem as an information loss minimization function,which preserves the inherent information of each view and the common information among multiple views when compressing the data of each view.The experimental results show that 1)the proposed algorithm can simultaneously mine and utilize the information of the individuality and commonality of multi-view action data,thereby improving the performance of pattern analysis.2)The proposed algorithm is superior to the existing state-of-the-art algorithms,especially on the more complex multi-view interactive action(such as person-object,person-person)video data sets.(3)Most of weighted multi-view clustering methods fail to comprehensively mine the complementary information among views and need to introduce additional parameters when learning weight parameters.To address these issues,a dual-weighted multi-view information bottleneck(DWIB)algorithm is first proposed,which uses mutual information to automatically learn the weight of each view,and then applies it to the content and context representation of the multi-view data,so that the learned complementary information can be fully propagated to clustering modules to improve overall pattern analysis performance.However,the propagation process of the algorithm is static,failing to consider the impact of the clustering results of each iteration on parameter learning.Aiming at this problem,a dynamic and autoweighted information bottleneck(DAIB)algorithm is proposed.First,we define a new discrimination-compression ratio to evaluate the discriminative ability of each view so as to learn the weight value of each view.Different from existing methods,we apply weights to the compact feature representations of each view to reduce the negative effects of noise and redundant information in high-dimensional data.Then,the cluster information obtained from each iteration is propagated to the weight learning process,forming a dynamic bidirectional information propagation flow.Experimental results verify the effectiveness of the proposed algorithm,especially on highdimensional multi-view data sets.(4)Existing methods are difficult to take into account the correlations of both features and clusters,and it is often difficult to obtain high-quality results.Aiming at this problem,a dual-correlated multi-variate information bottleneck(DMIB)algorithm is proposed to fully mine and propagate the two-level correlations in multi-view data,which provides a new method for pattern analysis.Specifically,first,the traditional multi-variate IB method is used to mine the unique feature information of each view,and meanwhile it learns a shared discriminative feature subspace between views to mine and propagate the shared feature information.Then,for each view,the unique feature information of this view and the shared feature information between views are simultaneously integrated to obtain the local clustering results of each view.Finally,we obtain the shared mutual information of multiple local clustering results to propagate the cluster-level relationships between views so as to obtain the final optimal data partition.In order to solve the optimization problem,a two-stage optimization algorithm is designed,and its convergence is further proved theoretically.The experimental results show that the proposed algorithm can effectively mine and exploit the two-level correlated information at the same time,and can handle complex multi-view clustering problems,and has strong pattern recognition capabilities.