Research On Graph Classification Based On Transformer And Capsule Network

Graph data has many key applications in biology,chemical informatics and other fields,such as molecular attribute judgment,new drug discovery and so on.Graph can also be used to represent relational data,such as social networks,knowledge graph,etc.Graph Neural Network(GNN)has received tremendous attention due to their power in learning graph representations by modeling the topological structure and aggregating feature information.However,the scalar node representations learned from GNN may not be sufficient to effectively preserve the attributes of the node/graph features,resulting in sub-optimal graph representation.Repeated averaging gathers too much noise,which makes the features of nodes in different classes over-mixed and leads to the problem of over smoothing.This paper sorts out the development history of deep learning,Transformer and capsule network,and then leads to the relevant graph classification model in recent years.This paper introduces the concepts and principles of graph neural network,transformer and capsule network in detail,and improves the disadvantages of related models.Inspired by the concept of capsule proposed by Hinton,we propose a new framework for graph classification,named Caps Trans,which takes full advantage of Graph Neural Network,Transformer and Capsule Network.Specifically,we firstly use the GCN with residual connection to generate multiple capsules for each node to capture local structure information from multiple angles,in which is added to the convolution kernel of GCN to distinguish the self-connected nodes from other nodes.All capsules of each node are spliced into node features,and the obtained features are normalized by Graph Norm.Then,the Transformer is used to capture the global structure information of the graph and the semantic information between nodes,and the node features obtained by Transformer are split according to the capsule form to obtain the primary capsule.Secondly,we exploit dynamic routing mechanism to capture important information and properties at the graph level by the generated multiple embeddings for each graph,and utilize attention mechanism to focus on important features.Finally,we use dynamic routing for capsule features again to get class capsules and classify them.This paper proposes two improvements.The first is to introduce Transformer to obtain the global structure information of the graph and the semantic information between nodes.The second is to use Graph Norm to normalize the node characteristics of GCN.We evaluate the framework by using six graph datasets on biological information and social networks,and demonstrate that Caps Trans outperforms other SOTA techniques on the task of graph classification.