The Research And Application Of Cross-media Heterogeneous Transfer Learning

With the development of the Internet technology and the update of terminal equipment,multimedia technology has developed rapidly.There are more and more choices for people to communicate in the Internet with the popularity of 4G networks and the advancement of smartphones.A large number of words,pictures and video data are generated,which makes the study of multimedia data have a great impact on daily life.Although many scholars have studied the multimedia data and proposed many efficient and high-performance methods,but most of the traditional methods only deal with the single-media data.With the improvement of the Internet speed and the increase of computer storage capacity,many network stations encourage the visitors completing information by multiple media data that each multimedia data has its irreplaceable information and advantages.Even in daily communication,chat software also provides many ways for communication,such as text,voice,emoticons,pictures,short video and so on.The data generated by online chat is more personalized with different living habits.It makes mining inter-domain relationships and finding common space between heterogeneous data more and more important that the unordered information hybrid by multimedia data is increase rapidly.The purpose of transfer learning is to learn the relationship between two domains.The performance of the target domain classifier can be improved effectively by the existing complete information in the source domain.Heterogeneous transfer learning reuse data by solving the common space of two heterogeneous feature domains.Comparing with traditional heterogeneous transfer learning,such as cross-language learning and cross-subject learning,learning transfer knowledge between different media is more difficult.In this thesis,we study the existing problems of heterogeneous transfer learning in cross-media data deeply.Based on a large number of heterogeneous transfer learning models,two different cross-media heterogeneous transfer learning algorithms are proposed.First,it is extremely costly to obtain the co-occurrence data that exists in both source domain and target domain and has a strong one-to-one correspondence when source domain and target domain are different media.The non-co-occurrence data(common data)which only exists in the source domain or the target domain is easier to obtain with the development of big data.To solve this problem,this thesis proposes a cross-media heterogeneous transfer learning model for semi-paired,which can solve the transfer learning problem used co-occurrence data and non-co-occurrence data simultaneously.This model not only solves the transfer problem from source domain to target domain,but also transfers non-co-occurrence data knowledge to co-occurrence data.The main work of this model is as follows:firstly,this model proposes a new heterogeneous distance measurement method,which is different from the traditional method.This method not only solves the heterogeneous distance,but also adjusts the weight between the distances in different media domain through interpolation function.Based on the heterogeneous distance and the method of adjusting weight,the model can solve the mapping matrix for both co-occurrence data and non-co-occurrence data to the public space simultaneously.In the case of less co-occurrence data,the model can also make good use of abundant non-co-occurrence data.Secondly,this thesis proposes a cross-media heterogeneous transfer learning model to prevent over-fitting for that the gap between cross-media data is too large and the samples under different media will produce unique characteristics.Even the samples have a strong one-to-one relationship.Their features under different media do not all have a strong one-to-one correspondence.Therefore,the features between two media domains are divided into strong correlation features and weak correlation features.The strong correlation feature can be used to construct the common feature space between two domains while the weak correlation feature retains the peculiarity of every domain.The framework of the model is matrix factorization,which makes the model compatible with other transfer learning parameters when it solves both common and peculiar features.