Log-Based Distributed Network Management System Design And Implementation

With the rapid development of internet and the increase of network information, the system log of applications based on very large data scales up increasingly. Consequently, it brings troubles to the debugging of system and knowing the status of system. It is a research work of general and practical significance to solve the problem of using log information generated by application system effectively for analyzing system status and enhancing the abilities of per-alarm to satisfy its urgent demands of performance, robustness and scalability.Log analysis is an important way to keep track of computers and networks. The use of automated analysis always results in false reports; however these can be minimized by proper specification of recognition criteria. Current analysis approaches fail to provide sufficient support for the recognizing the temporal component of log analysis. Temporal recognition of event sequences fall into distinct patterns that can be used to reduce false alerts and improve the efficiency of response to problems.This thesis studies the issue of log management for large scale web applications and network system. Then, it introduces the design of a log-based network management system. The primary contributions of this paper include:First,the article analyzes two normal approaches to realize network monitor, one is based on packets captured and another is based on logs generated by network gateway. Then, it introduces the common kinds of log format, syslog and WELF. It describes the limitations in some products of log management on requirement nowadays.Second,It introduced the reasons for the need of log management, as well as the introduction of the need for log management system, analysis of the log management system functional requirements.Third, Basing on the discussion, it introduces the design of Log-based distributed Network Management System (LDMS) and describes the detail design of each module of LDMS. LDMS is made of log receiver, log processor, report engine, report scheduler, Web Services interface and web application. Using pattern matching of the regular expression to extract effective information from the original log, it innovatively applies the artificial intelligence technology (Rule-based Reasoning, Case-based Reasoning) and fuzzy matching technology to the network management which greatly improves the efficiency of the performance and simplifies the operation.LDMS is architected by J2EE and Struts. It is excellent in configuration, generality and portability. LDMS is integrated with existed network management system through Web Services interface.Finally, the article summarizes the flow of development in software engineer opinion.