Research On Key Technologies Of Spectral Graph Neural Networks

Graph is a fundamental data structure,usually used to describe the relationship between different objects.Many real-world data can be modeled by graphs,such as social networks,biological information networks,sensor networks,and so on.These complex graph data often contain rich knowledge,leading to an increasing interest among researchers in mining graph data.Graph neural networks currently serve as the mainstream tool for graph data mining,which update node representations by aggregating neighborhood information and effectively combine graph data with the powerful representation capabilities of deep learning.There are two main routes for the research of graph neural network:spatial methods and spectral methods.Spatial graph neural networks design different message passing mechanisms to perform graph convolution operations in the spatial domain,thereby effectively aggregating information of neighbors.Spectral graph neural networks utilize the spectrum(i.e.,eigenvalues)and basis functions(i.e.,eigenvectors)of graphs to design convolution operators in the spectral domain.By combining graph signal processing theory and neural network approaches,spectral graph neural networks have made a huge impact in the field of graph representation learning.From the perspective of signal processing,spectral graph neural network is a basic tool for processing highdimensional graph signals,which can be regarded as a generalization of signal processing methods for one-dimensional time series data and two-dimensional image data.From the perspective of neural networks,there is a close connection between the spectral methods and spatial methods,and the two research routes can mutually benefit and inspire each other.Although significant progress has been made,it is not trivial to transform graph data into the spectral domain for processing.There are three main challenges as follows:(1)How to learn robust representations from eigenvectors to maintain the stability of spectral graph neural networks?(2)How to adaptively utilize complex frequency information of eigenvalues to learn effective graph representations?(3)How to improve the generalization of spectral graph neural networks and prevent models from overfitting?In response to the above challenges,this thesis studies the key technologies of spectral graph neural networks.First,a eigenvector coding technology is studied to enhance the stability of the spectral graph neural networks;then,two adaptive graph filtering technology are studied to improve the adaptive ability and expression ability of the spectral graph neural networks;finally,a consistent regularization training technology is studied to support the training of graph filters and improve the generalization of spectral graph neural networks.The main work and innovative contributions of this paper are as follows:Firstly,aiming at the problem that spectral domain features are difficult to utilize,an rotation-and reflection-invariant eigen-perceptron is designed.In addtion to retain the high efficiency characteristics of the multi-layer perceptron,this model can also learn stable and invariant spectral representations from the eigenvectors with coordinate-and sign-ambiguilty.Subsequently,a contrastive learning framework is used to combine the eigen-perceptron with a spatial graph neural network to improve the stability of the graph neural network.Secondly,aiming at the problem that the frequencies of spectral domain is difficult to determine,a frequency adaptive graph neural networks is designed,which considers the importance of different frequencies.The model first uses low-pass and high-pass filters to separate low-frequency and high-frequency information from node features,and then uses an attention mechanism to weight and reorganize these two types of frequency information,so that it can adaptively learn the importance of low-frequency and high-frequency information.This design breaks through the low-pass filtering limitation of previous graph neural networks.Finally,the model is extended from transductive learning to inductive learning by approximating the filters.Thirdly,aiming at the problem that the frequencies of spectral domain is difficult to determine,a multivariate filtering graph neural networks is designed,which considers the dependence between frequencies.This model simultaneously feeds the high-dimensional representations of multiple eigenvalues into the self-attention module to learn the dependency relationships between eigenvalues,and adaptively selects the most important dependency information for filtering.Theoretically,this model is a generalization of the previous univariate graph filter and has stronger fitting and expression capabilities.Fourthly,aiming at the difficult problem of training spectral graph neural networks,a consistency regularization framework based on spectral graph theory is designed.This framework first augments the high-frequency information in the graph signals,and then constrains the spectral graph neural network to make consistent predictions on the original data and the augmented data,thereby effectively utilizing a large amount of unlabeled data to assist the traning of spectral graph neural networks.Experimental results on a large amount of semi-supervised datasets show that this framework can improve the generalization of spectral graph neural networks and alleviate the over-fitting phenomenon.