Network Representation Learning Algorithm Research Based On Attention Mechanism

Network Representation Learning(NRL)is the basis of complex information net-work analysis,which aims to represent nodes(node level)or the whole network(graph level)as low-dimensional dense real value vectors,so as to be applied to node classi-fication,link prediction,graph classification and other practical tasks.In recent years,the study of network representation learning has attracted more and more researchers'attention,among them,Graph neural networks based on deep learning are playing an increasingly important role in the field of network representation learning.However,the high-order neighborhood information can not be effectively used is the problem of most existing graph neural networks.This paper studies the problem,and proposes cor-responding solutions in the node-level and graph-level network representation learning tasks.The main research contents are summarized as the following two points:(1)In the node representation learning task,a graph convolution network based on attention mechanism was proposed to fuse multi-order neighborhood information.We first use different masking matrices in the attention mechanism to extract node rep-resentations that fuse neighborhood information of different orders;then we propose a fusion mechanism that fuse multiple representations of nodes into a unified repre-sentation.The use of neighborhood information of different orders is performed in-dependently,avoiding the over-smoothing problem caused by stacking multiple graph convolutional layers.Specifically,in this paper,the powers of the adjacency matrix of the input graph are used as masking matrices,and the corresponding position of the at-tention weight matrix is set to 0 to extract the local information of the neighborhood of different orders.In addition,in the fusion process,a dynamic routing algorithm is used to adaptively determine the contribution of different node representations to the final node representation to learn more effective node representations.The node classifica-tion experiments on the citation network data sets show that our proposed method has higher classification accuracy than the more advanced network representation learning models.(2)In the graph representation learning task,a multi-attention pool based on the attention mechanism is proposed,which comprehensively considers the attribute infor-mation of the nodes,the local and global topology information of the graph.Specif-ically,based on attention mechanism,the graph convolution operation is used in this paper to integrate the property information and structure information of neighbor nodes of different orders,so as to calculate multiple attention scores for each node.Then,the weighted sum of these attention scores is used as the basis for sorting the nodes,so as to pool the nodes.Finally,the representations of the nodes remaining after pooling are fused into the final graph representation.The graph classification experiments on the bi-ological graph data sets show that our multi-attention pool achieves better performance than the existing graph pooling methods.