Temporal Network Representation Learning Based On Deep Neural Networks

Networks are often used to define complex data relationships,where nodes represent entities and edges represent interactions between entities.In the real world,the nodes and edges of networks are usually not static but evolve over time,which are called temporal networks.For example,the number of users in social networks is not constant,which may evolve over time.How to represent the nodes of temporal networks plays an important role in the network analysis task.It aims to effectively extract features from the network for downstream network analysis tasks such as link prediction and node classification.With the appearance of large-scale networks,traditional node representation methods such as spectral clustering and matrix decomposition suffer from some challenges.Besides,networks also contain abundant heterogeneous information and outlier nodes,which make new challenges for the traditional node representation methods.Hence,network representation learning methods based on neural networks appear.Network Representation Learning(i.e.,Network Embedding)aims to learn a lower-dimensional representation vector for each node in the network,which preserves networks’ topology structure information,temporal information,and heterogeneous information,etc and can be used for downstream network analysis tasks such as link prediction,node classification,and community detection,etc.Some studies reveal that network representation learning methods based on deep neural networks are more advantageous in extracting high-dimensional nonlinear features than based on shallow neural networks.Hence,our work in this dissertation is based on deep neural networks to study representation learning on temporal networks.The main research contributions of the dissertation can be summarized as follows:First,we study the representation learning method for temporal networks based on secondorder weighted sampling between nodes.In a temporal network,we can divide the network into a sequence of snapshots by timestamp.We study incorporating previous snapshots of networks into current snapshots for effective feature extraction.Existing work shows that extracting the spatial relation of each node can be used as a valid feature representation for each node.However,existing work only considered the weight of direct neighbors of a given node to extract features,so it was insufficient to capture topological and temporal features of temporal networks.In the real-life networks,a node’s neighbors of neighbors also have some useful information about the node,so we should consider these features together.In the dissertation,we propose a temporal network embedding model TT-GWNN from the perspective of node feature sampling.In the model,we proposed a second-order weighted random walk sampling algorithm(SWRW)to extract topological and temporal features from a given node from its neighbors and neighbors of neighbors in the previous network snapshots.More specifically,our SWRW algorithms combine previous snapshots of first-order and second-order weight into a weighted graph,i.e.the weighted graph combines the topological and temporal features of the temporal network with weights.It also incorporates a damping factor to assign greater weights to more recent snapshots,which can better preserve the evolving weights of temporal networks.Particles then walk according to the weights.In this way,SWRW can better preserve both the topological structure and temporal evolution features of the networks.We then adopted graph wavelet neural networks(GWNNs)to embed the topological and temporal features into vectors.Finally,the effectiveness of the proposed method is proved by link prediction experiments.Second,we study the representation learning method for temporal networks based on hierarchical random walk sampling between snapshots.We study effectively combining previous snapshots of temporal networks to extract network features.In a temporal network,the current snapshot topological structure of temporal networks is derived from the previous snapshot topology,it is necessary to combine the previous snapshot to extract the spatial-temporal features for the current snapshot.In the dissertation,we propose a temporal network embedding model ST-HN from the perspective of node feature sampling.In particularly,we develop a truncated hierarchical random walk sampling algorithm(THRW)to extract both spatial and temporal features of the network,which randomly samples the nodes from the current snapshot to the previous one and it can well extract networks’ spatial-temporal features.Because snapshots closer to the current snapshot contributes more to the current snapshot for temporal features.THRW also incorporates a decaying exponential to assign longer walk length to more recent snapshots,which can better preserve the evolving behavior of temporal networks.Then,we improve upon the state-of-the-art approach,higher-order graph convolutional architectures,to embed the nodes,which can aggregate spatial-temporal features hierarchically and further reinforces the time-dependence for each snapshot.Besides,it can learn mixed spatial-temporal feature representations of neighbors at various hops and snapshots.Finally,we test the embedded vector’s performance on link prediction and node classification task to verify its performance.Third,we study temporal attributed network embedding with outliers.We study addressing the outlier nodes in temporal attributed networks and learning more robust embeddings in a noisy environment.In the real world,many networks contain rich attributes and temporal information,which are called temporal attribute networks.In a real-life network,node structures or their attributes may deviate from the property of the community to which they belong.Such a nodes are an outlier nodes.Relevant studies have shown that outlier nodes can affect the embedding performance of the regular nodes.However,there is no previous work explicitly considering the effect of outlier nodes in the temporal attributed network embedding.In the dissertation,we propose a temporal attributed network embedding framework with outliers based on autoencoder(TAOA)to perform node embedding in temporal attributed networks.In particular,the model utilises an outlier-aware autoencoder to model the node information,which combines the current network snapshot and previous snapshots to jointly learn embedded vectors for nodes in the network.In feature preprocessing,we propose a simplified higher graph convolutional mechanism(SHGC)to preprocess attribute features for each node in each snapshot of temporal attributed networks.The SHGC incorporates attribute information into link structure information,which can leverage attribute information into link structure features.Experimental results on link prediction and node classification show that our proposed method is competitive against various state-of-the-art methods.