Preprocessing Technology And Merge Of Multi-source Heterogeneous Logs

Network security situational awareness system (NSSAS) is a new technology to monitor network security and it is one of the hot research domains in information security. NSSAS observes large-scale network security situation from a holistic perspective, it timely monitors potential or arisen anomalies in network system and it also can forecast network security status in some time later.How to obtain necessary network security situational awareness data is researched comprehensively in this paper. Four modules are designed to complete merging of multi-source heterogeneous Logs.Log acquisition module collects all the logs which reflect the network security situational information from the network, including the critical host logs (Windows, Linux), network data transmission equipment logs (switch, router), network security device logs (hardware firewall, software firewall).Log preprocessing module pretreatments multi-source heterogeneous logs data by using different rule-bases (Windows, Linux, snort logs). Then these logs data have been removed useless and redundant information.Data conversion module designs multi-source heterogeneous logs data's xml schema documents. It deals with xml schema document by using xmlbean technology. It converses logs data by combining with java programming. It gets unified format data after data conversion.Data merge module merges the data according by timestamp. It comprehensively determines the network security incidents at the same time point. Experimental tests show that the method can be used in multi-source heterogeneous logs data merge and preprocessing. Network managers can use these basic data in decision-making network security situational.