Design And Implementation Of Operation And Maintenance Knowledge Extraction System Based On Frequent Pattern Mining

With the vigorous development of computer technology,the number of system collection indicators continues to increase,and the industry has produced a large amount of operation and maintenance data.It is difficult for people to extract knowledge from massive data and form expert experience,which can be used as a reference basis for threshold setting of traditional operation and maintenance methods,and can contribute to the analysis of alarm causes and the correlation between query indicators.Therefore,how to extract effective information from operation and maintenance data through data mining related methods is a current research hotspot.Frequent pattern mining is an effective data mining method,but for operation and maintenance data with complex characteristics,the mining results of traditional frequent pattern mining algorithms are not comprehensive enough,and the mining efficiency is low.In view of this,A framework is proposed in this thesis that applies frequent pattern mining to operation and maintenance data mining,and uses different methods to mine operation and maintenance data with different characteristics to improve the effectiveness and efficiency of refining operation and maintenance knowledge.For multi-dimensional intensive monitoring data,first,a discretization method suitable for all indicators is proposed to discretize the monitoring data,and then,a method for mining frequent transition patterns is defined and proposed as a supplement to the traditional mining method.Aiming at rare and aggregated alarm data,an incomplete longest sequence pattern mining method is proposed in this thesis,which is used to efficiently extract frequent alarm sequences and assist in alarm troubleshooting.The main research contents of this thesis are as follows:1.Knowledge extraction of monitoring data.In order to solve the problem that the frequent pattern mining algorithm cannot handle continuous data and the incomplete monitoring data results of traditional frequent pattern and sequential pattern mining algorithms.First,a clustering algorithm combined with box plot method is used in this thesis to eliminate abnormal data,and uses trend prediction algorithm to fill in missing values;Then,based on the idea of peak detection and filtering,a general distribution detection algorithm is proposed to extract the data distribution interval of a large number of indicators to discretize the data;Finally,mining and fusion of traditional frequent patterns and sequential patterns to form a frequent transition patterns,as a supplement to the traditional frequent patterns and sequential patterns,so that users can more intuitively understand the impact of indicator changes on the system.2.Knowledge extraction of alarm data.Aiming at the long time-consuming problem of the traditional longest frequent sequence pattern mining algorithm for mining rare alarm data containing a large number of long frequent items,an incomplete longest sequence pattern mining method is proposed.The method simplifies the process of searching for non-longest sequence patterns in the candidate set of the longest sequence pattern mining algorithm,improving the efficiency of knowledge extraction for alarm data operation and maintenance.3.System implementation and deployment.Designed and implemented an operation and maintenance knowledge extraction system based on a microservice architecture.The system integrates data preprocessing,monitoring data frequent pattern mining and alarm data frequent pattern mining modules.According to actual business requirements,the functions of historical data analysis and online data detection are realized,which effectively improves the intelligent level of operation and maintenance.