Attack Pattern Mining Based On Security Log

Nowadays, computer and Internet technology are developing rapidly.People enjoy the convenience brought by technological progress and have to face more and more serious information security problems at the same time. The application of network security devices have used to defense network attacks. However, due to the increasing number of attacks and the increasing complexity of attack methods, the security logs generated by various devices are becoming more and more complicated. The number of them is getting larger and larger, and the readability is getting worse and worse. A large number of threats is hidden in these massive security logs.Through the data mining of the security logs, the hidden attack mode is unearthed and the attack scene is refactored. So that the true intention of the attack can be found, and the attacking behavior can be more effectively prevented.At present, the analysis and processing of security log is facing many problems. On one hand, the algorithm has a great dependence on experts’prior knowledge, and it is difficult to analyze new attack patterns. Many algorithms can only dig out some fragments of attack patterns. The accuracy of mining results is very low. On the other hand, data source of the security log gradually dispersed, which makes the collection of raw log data becomes more difficult. With the speed of network transmission speed up, the network traffic is also growing, the security log data storage query processing methods are bottlenecks, resulting in overall system performance.In view of these problems, this paper has carried on the thorough research, the main work as follows:(1) The related log collection and storage technology and data mining related theories and algorithms are studied, which accumulates the theoretical foundation for security log mining.(2) This paper proposes an attack pattern mining algorithm based on improved fuzzy clustering analysis and sequence pattern. This algorithm can describe the similarity between the security log and the sequence pattern mining algorithm, and also can describe the logic relation between the attacking steps well, which make it could excavate the hidden attack pattern knowledge in the log more accurately. Firstly, the security log data is transformed into the global attack sequence according to the timestamp.Then, the improved fuzzy clustering algorithm is used to aggregate the sequences that may be in the same scene according to the similarity between the security log attributes. Finally, the attack pattern of the attacker is extracted from the attack sequence by the sequential pattern mining algorithm.(3) Design and build an attack pattern mining system based on ELK.The system is based on ELK technology can be completed from the security log collection to the attack mode mining.(4) Set up the experimental environment to analyze and evaluate the algorithm. Experimental results show that the algorithm can not only effectively exploit the attack patterns hidden in the security log, but also reduce invalid attack patterns in the results and generate more valuable attack pattern knowledge.