Research On AIOps Technology Based On Hadoop

The widespread use of the Internet has led to an unprecedented growth in the number of global Internet users and the overall scale of the Internet.This growth has played a crucial role in driving the transformation of information technology across various industries and domains.Over the past few years,the transformation of enterprise IT architecture and the rapid growth of Internet-based businesses have presented significant challenges to traditional modes of operation and maintenance.This is particularly true in light of the rise of distributed cloud architecture and microservice architecture.In order to deal with the outstanding problems of traditional IT operations,many enterprises have begun to try to apply AIOps technology to improve the efficiency and quality of operations.Alarm convergence is an important issue in IT operation and maintenance.It involves identifying and merging similar or related alarms from a large number of alarms,classifying and handling these alarms,reducing the number and repetition rate of alarms,and improving operation and maintenance efficiency.At present,alarm convergence technology has become one of the hot research directions in the field of IT operation and maintenance.This paper conducts an in-depth study on alarm association mining based on association rules.The objective of this paper is to examine alarm data,including its origin and traits,and investigate the techniques and procedures employed in its preprocessing based on its distinct characteristics.Furthermore,the paper describes and evaluates the alarm data that has been extracted subsequent to the preprocessing stage.This paper delves into the Apriori algorithm,a well-known association rule mining technique,by analyzing the conventional Apriori algorithm that accesses the original transaction database multiple times.To enhance the algorithm’s performance,this paper introduces hash and Boolean array.Moreover,this paper investigates the Apriori algorithm’s implementation using multi-machine parallel computing.In particular,the study provides an in-depth examination of the Apriori algorithm based on MapReduce by outlining its principle and workflow.The shortcoming that MapReduce is only applied to the item set matching operation of the partitioned database is analyzed.This paper suggests the implementation of the two-stage MapReduce technique in the Apriori algorithm.Building on this research,the paper additionally presents the Apriori algorithm based on HBase and Boolean array.By introducing HBase and Boolean array,the Apriori algorithm can access the original transaction database repeatedly.The efficiency of multi-machine parallel algorithm is improved by introducing MapReduce based self-linking process.The enhanced Apriori algorithm is utilized to discover association rules from the preprocessed alarm data,and the paper examines and evaluates the resultant association rules.The focus of this paper is on the investigation of an alarm analysis system,involving the analysis and design of its comprehensive framework,technical structure,and system capabilities.By constructing an alarm analysis system that utilizes association rules,the study aims to provide operation and maintenance staff with more effective IT support for their tasks.