Image Classification Algorithm Based On Multi-relational Social Network

The rapid development of the Internet provides a lot of convenience for people to communicate,and users can easily share and browse images through the Internet.Images in online social networks often carry a lot of social network metadata,which can reflect the associated information between images.These social network information can used to eliminate the ambiguities in the image,this improve the performace of image classification.Although some methods have been proposed to perform classification with the help of social network information.However,most algorithms are not able to effectively utilize different types of social network metadata,and some algorithms have defects that cannot accommodate large-scale data sets.In order to solve these problems,this paper proposes a new image classification algorithm MSNet based on multi-relational social network.MSNet first uses the social network metadata of the images to construct different relational networks between images,and then uses the network embedding learning algorithm to learn the representation vector of the images in the relational networks.Finally,we use visual features and network representation for image classification.The main contributions of this paper include the following three parts:(1)A semi-supervised discriminant embedding learning algorithm(DNE)is proposed.DNE algorithm is based on the SkipGram model.It can use the labels information of some nodes to learn the discriminant representation vectors.Traditional discriminant network embedding algorithms need to train an additional classifier to adjust the representation vectors of nodes with labels.This kind of methods usually fail to effectively utilize semi-supervised information.They also have some shortcomings,such as long training time,need a large number of tagged nodes and so on.DNE algorithm is different from traditional methods.When learning network representation vectors,DNE regards each kind of label as a node.DNE learns the representation vectors of each label at the same time when it learns the representation vectors of network nodes.The final representation vectors are obtained by optimizing the network topology loss function and the semi-supervised loss function.(2)A multi-relational network embedding learning algorithm(MNE)is proposed.Most of the existing network embedding algorithms are focus on single network.In a multi-relational network,the nodes in the network contain different relationships and it can form different relationship networks.For example,we can construct a cooperative working relationship network between human beings or a friend relationship network between human beings.Traditional network embedding algorithms usually can not directly learn the representation vectors of multi-relational network.The MNE algorithm proposed by us can learn the consistency representation vectors of multiple relational networks.MNE automatically learns the weight of each relational network and integrate multiple relational networks according to the weight of the networks.Finally,MNE algorithm can learn the consistency representation vectors.MNE algorithm solves the problem by an iterative optimization method,which has a very fast running speed.(3)An image classification algorithm MSNet based on multi-relational social network is proposed.In MSNET,the above two representation learning algorithms(DNE and MNE)are used to improve the performance of image classification.Current image classification algorithms usually regard the social network metadata carried by images as a set.This makes the algorithms can not fully mine the association of images in social network and can not effectively utilize a variety of types of information.MSNet uses a different approach to utilize the social information.Firstly,we construct a variety of relationship networks between images,and then utilize the two network embedding algorithms proposed in this paper to learn the representation vectors.Finally,using the visual features and the network representations of the images to train neural network model for image classification.In order to better classify images,we propose five neural network models in MSNET,and compare the classification performance of different models.Three of the models adopt a new type of neural network structure(Capsule Network).We have changed the structure of capsule network to make it suitable for MSNet framework.Experiments show that MSNet can effectively use social network information to improve the performance of image classification algorithm.