Research On Anomaly Detection Technology Of Publish/Subscribe Distributed System Based On Pattern Mining

Publish / subscribe distributed system based on data distribution service(DDS)is a kind of distributed system in which components communicate through DDS and have loose coupling characteristics.It is widely used in the fields of national defense,military industry and so on.With the growing scale of publish / subscribe distributed system and the increasingly complex relationship between components,the security threats faced by the system are also increasing.For example,denial of service attacks against the system will consume a lot of system resources,which makes it difficult for the system to provide services stably.Therefore,it is urgent to carry out effective state monitoring and maintenance management to ensure the security and stability of the system.At present,there are corresponding measures to deal with the security threats faced by the system,but there is a lack of security monitoring of the whole system.In addition,there are many running states in the process of system operation,and there is a lack of anomaly detection scheme based on running pattern,which can not accurately grasp the running state of the system.To solve the above problems,an anomaly detection scheme for publish / subscribe distributed system is designed,which includes two phases.In the offline analysis phase,the data mining technology is used to mine the system running pattern from the historical operation data of the system,and the representation method of the running pattern is proposed to build the running pattern knowledge base.In the online detection phase,AC automata algorithm with attributes and real-time stream processing framework are used to match the running state data of the system,monitor the running state of the system in real time,and detect the abnormal operation of the system.The main work of this thesis is listed as follows.(1)A running pattern mining method based on weighted frequent itemsets mining algorithm is proposed.By studying the operation characteristics and communication mechanism of publish / subscribe distributed system,Apriori algorithm is improved from two aspects of data storage and support calculation.Combined with transaction matrix,the importance of publish /subscribe events is reflected from two aspects of frequency and influence degree,so as to apply to the running pattern mining of publish / subscribe distributed system and improve the mining efficiency.(2)The representation of the running pattern of publish / subscribe distributed system is presented.According to the characteristics of multiple running states in the system,combined with component association,event sequence based on regular expression and event attribute constraint,the running pattern is defined and represented.The knowledge base of running pattern is constructed,and the running pattern is used as the basis of pattern matching in online anomaly detection.(3)An anomaly detection method based on AC automata algorithm with attributes is proposed.In order to detect the abnormal behavior of the system,the running patterns obtained from offline mining are used to construct automata for pattern matching.In order to improve the efficiency of anomaly detection,the index of event attribute constraint is stored in the state node of AC automata,and the correctness of event attribute is detected while the event sequence is matched.According to the characteristics of the sliding window model used in data stream processing,an anomaly correction mechanism is added to the detection results to further improve the accuracy of anomaly detection.(4)Integrating the above theoretical research results,an anomaly detection prototype system of publish / subscribe distributed system is designed and implemented.The anomaly detection prototype system is a monitoring and management platform which integrates offline analysis and online detection.It provides an operation interface for mining and representing running pattern,and uses real-time stream processing framework to monitor the running state of the system in real time to detect abnormal operation.The test results of the prototype system show that the anomaly detection scheme proposed in this thesis is correct and effective.