Abnormal Network Operation Data Augmentation Method Based On Generative Adversarial Network

With the continuous expansion of network scale and the continuous complexity of network structure,network operation anomalies have become more frequent.In order to achieve network root cause fault diagnosis and network anomaly detection,it is necessary to provide big data support for alarm association rule analysis algorithms and network abnormality detection models by collecting network alarm log data,network KPI abnormality data,and other abnormal network operation data.However,in the real network environment,there are problems such as insufficient data volume and unbalanced category distribution in abnormal network operation data,resulting in weak generalization of its training model.In response to data expansion requirements,data augmentation techniques based on oversampling techniques,Recurrent neural networks,and Generative adversarial networks can be used to improve the effectiveness of root cause fault diagnosis and network anomaly detection by expanding the volume and diversity of the dataset.However,when existing data augmentation methods process unstructured fault data such as network alarm log data,the generated alarm log data is not sufficiently diverse,and only conforms to context in probability distribution,but cannot model the relationship between alarm cascades in actual scenarios;When generating structured network operation exception data such as abnormal network KPI data,although the distribution of the generated data in a short time window is relatively consistent with the distribution of real data,the generated data in a long time window is significantly different from the real data.At the same time,existing methods are also difficult to extract features of few samples in complex multidimensional time series data.Therefore,in view of the above problems,this paper proposes an abnormal network operation data augmentation method based on generative adversarial network(GAN),which enhances data for unstructured and structured network operation exception data respectively.The research work of this paper is as follows:(1)A method of unstructured alarm log data augmentation based on GAN model and rule mining algorithm is proposed.Aiming at the problem of weak correlation between alarm log data generated by existing methods,a generator model integrating Apriori rule mining algorithm is proposed.The generator generates data according to the context relationship and alarm rule correlation between alarm log data at the same time;A multidimensional discriminant model combining the correlation between data and the authenticity of data is proposed.Combining the structure of rule miner,classifier,correlation matrix,etc.,the correlation between generated data is identified in the adversary learning,and the generator is trained in cooperation with the discriminator in the way of reinforcement learning to further improve the rule correlation between generated data;Finally,aiming at the problem of insufficient diversity of generated data in some deep learning generation models,that is,pattern collapse,a rule dynamic update mechanism based on category distribution difference is proposed,which dynamically adjusts the category distribution of generated data in the generator training process in the form of dynamic update rule set,so as to improve the diversity of generated data.The experimental results show that the method proposed in this paper can generate higher quality alarm log data,and ultimately improve the effect of network root fault diagnosis.(2)A network KPI anomaly data augmentation method based on multi-level discriminant GAN model and autoencoder is proposed.To solve the problem that it is difficult to obtain the characteristics of few samples in complex multi-dimensional time series data,a method of network anomaly KPI data coding dimension reduction based on autoencoder is proposed,which represents the complex multi-dimensional time series data in the form of probability distribution in the lowdimensional space of the data;Aiming at the long distance dependence problem that the generative model of deep learning will lose the previous time series information when generating growth data,a generator model based on the improved Deep Autoregressive Recurrent Networks(DeepAR)is proposed to predict the probability distribution of multiple time series data instead of a single time series cyclic prediction to reduce the impact of long distance dependence on the quality of generated data;Finally,a discriminator model based on multi-level discriminant structure is proposed.Compared with the original GAN model,the discriminator can discriminate the generated data from the time sequence,and improve the temporal correlation between the generated data in the confrontation learning.The experimental results show that the method proposed in this paper can generate higher quality network KPI anomaly data,and ultimately improve the network KPI anomaly detection effect.