Research On Internal Network Security Threat Detection Method For Hybrid Network Data

As a key infrastructure of information,digitization,and intelligence that carries various data,business applications,scientific research and office,the internal network is widely used in government departments,national defense,military industries,enterprises and institutions.However,in recent years,threatening actions about data theft,business disruption,and network outage against internal networks have occurred from time to time,especially for those of high-value target customers(such as military industry,banks,power industry,telecommunications,high-value enterprises,etc.),leading to significant losses to national security,social security,and economic security.Therefore,internal network security has become a key issue that must be paid attention to,while insider threat detection is an important research area to address the security issue.The goal of insider threat detection research is to find potential threat behaviors,abnormal users,and security threats associated with users from intranet log data,known protocol traffic data,and customized or unknown protocol data.The scientific problem contained in it is "computation of insider security threat behavior patterns under the condition of mixed network data".In order to achieve the above research goals,based on the analysis of insider threat detection methods and technologies towards this scientific problem this dissertation devises a multi-layer clustering algorithm for mixed binary protocol data,aiming at addressing the difficulty of data clustering and providing data basis for subsequent research.Furthermore,threat detection method based on crossheterogeneous data domain transfer learning and abnormal user detection method based on business interaction network are respectively proposed to address the difficulties in the detection of insider threat behavior and identification of abnormal users,due to the lack of intranet labeled data.And a threat detection algorithm is designed by processing and analyzing non-numeric fragmented users’ behavior logs.The specific contributions are as follows:Firstly,this dissertation proposes an optimal computing model for multi-layer clustering of binary protocol data to address mixed binary protocol data clustering without prior knowledge.This model uses clustering validity indexes as the objective optimization function to determine the number of clusters.It uses Smith-Waterman algorithm to obtain pairs of similarity values,so as to construct the affinity matrix.Three best-performing clustering validity indexes,i.e.,Calinski-Harabasz,Davies-Bouldinn,and Silhouette,are selected to control clustering.According to the variation of the three indexes in the continuous clustering process,this method can perform multi-level clustering on binary protocol data.The experimental results show that the accuracy and recall rate of most of the data are above 93% in the first clustering procedure.After the second clustering,clusters are subdivided into more high-level protocol types,and no new error clusters are introduced,thus maintaining the protocol layering characteristics of the binary protocol data.At the same time,compared with other methods,the method in this dissertation is better than other methods in terms of the four indicators used in the comprehensive evaluation of clustering effect.Secondly,concerning the lacking of labeled training data and the scarcity of attack samples,it is difficult to train an insider threat detection model.Thus,this dissertation proposes to use the public labeled data set from external networks to assist the construction of an insider threat detection method.Because internal and external networks have differences in terms of attack methods,frequency of occurrence,and data collection methods,there are deviations in the feature space,probability distribution,and other data characteristics between the internal and external network data sets,and it is difficult to directly use external datasets to train models for internal networks.To this end,a transfer learning method based on feature space mapping is proposed to address the feature space heterogeneity and probability distribution mismatch.Specifically,the internal and external network data sets are mapped to a common latent feature space via linear projection.Then,the maximum mean difference(MMD)is used as a measure of the probability distribution difference to construct an optimization problem to minimize the probability distribution difference of the mapped data while keeping the projection loss low.The global optimal solution of the problem is obtained by analyzing the optimality conditions,thereby obtaining the projected samples of the internal network data set in the common feature space.With projected samples as input,traditional classification algorithms for threat detection are trained to identify attack instances in the internal network.Through the transfer learning experiments between NSLKDD and UNSW-NB15 datasets,the accuracy of cross-domain learning tasks is improved by 14.6%,which verifies that the proposed method can significantly improve the accuracy of cross-domain threat detection.Thirdly,it is difficult to identify abnormal users of internal network applications,mainly attributing to no abnormal user labels and the weak representation of the user relationship network within the application scope.In response to this problem,based on research results in social networks,this dissertation proposes to use global features based on behavior and local features based on social principles to improve original feature representation capabilities.This dissertation focuses on business interaction network in internal networks and designs a novel network embedding learning model to infer the status and role of users in the business relationship network.The model takes the user’s local and global features and relationships as input and takes into account the directionality of the business interaction network graph.It uses spatial convolution Graph Neural Network(GCN)to accomplish the tasks for learning the feature representation of the business relationship network.In addition,attention mechanisms and gates are utilized to emphasize the different contributions of neighbors to the central node to achieve better user representation learning capability.With the proposed method,the user embedding representation is obtained for user role inference and abnormal user detection in the business interaction network.In comparative experiments with other methods,the accuracy of node attribute inference accuracy is improved by at least 2% and the average detection accuracy on the real data sets can reach 92%,which proves the effectiveness,superiority,and robustness of this solution.Fourthly,the effectiveness of insider threat detection methods is limited by sparsity of the threat feature and little discrimination of users’ behavior patterns from non-numerically fragmented log data.To address the detection effectiveness problem,an insider threat detection method to distinguish multi-user behaviours is proposed.To more effectively utilize the features extracted from user behavior logs for threat detection,this dissertation proposes an entity embedding method based on TF-IDF,which represents the operated entity according to the frequency of the entity in different sessions.Then,this dissertation constructs an insider threat detection framework based on ensemble learning.Specifically,a detection framework based on ensemble learning is first designed,and a new sampling strategy called Over-Bootstrap is utilized,which can effectively alleviate the over-fitting problem caused by sample imbalance and sparsity of the threat feature.In addition,a selfsupervised auxiliary task that identifies the user corresponding to the behavior sequence,so as to learn the behavior representations of different users.In experiments conducted on CERT4.2 and CERT6.2 datasets,the proposed method can effectively detect insider threats,with AUC of 99.2% and 95.3%,respectively.