Improving Credibility And Predictability Of IDS Alerts Using Correlation Algorithm

Any computer connected with Internet is undoubtedly faced the problem of being threatened by hackers or attackers. In order to protect our computers from being attacked or violated, security researchers have devised many kinds of security tools. Among those tools, Intrusion Detection System (IDS) is an important one. The IDS has the possibility to provide valuable information about the current security status of the network or host having been protected. The IDS is now widely used in all kinds of applications.The goal of IDS is to provide the security administrator with the current security states of the network or host having been protected. In process of implementation, based on the analysis of log files and network packets, the IDS sensor will produce an alert with the purpose of notifying the administrator with the occurrence of malicious activities when certain detection rule is triggered by the data having been analyzed.However, it has become a well-known problem that current intrusion detection systems still have some problems.First, IDS sensors produce large volumes of alerts, including actual alerts, sometimes even false alerts (such as false positives and non-relevant positives). Accompanied by the quick improvements of the network performance, more and more network-based applications are being introduced. Consequently, the IDS sensors are generating increasingly overwhelming alerts. This makes it extremely challenging to understand and manage the intrusion alerts, not to mention timely response to intrusions. The vast information has also submerged communicating channels and influenced normal network services.Secondly, understanding the strategies of attacks is crucial for security applications such as computer and network forensics, intrusion response, and intrusion prevention. Generally speaking, it is easier to predict an attacker's goal and decrease the damage caused by intrusions, if the attack strategy is known during earlier stage of intrusions. In order to understand the attack strategy, however, the users should analyze the sensor alert manually. This process is not only time-consuming, but also error-prone.Third, IDS sensors produce so many false positives and false negatives. The rule sets cannot be updated simultaneously with the emergence of newtypes of attacks, thus the new attacks cannot be detected. The existence of large volume of unrelated attacks impedes the actual testing as well.Finally, the security of IDS itself has been threatened too, and without a nice scalability. Consequently, IDS will fail to meet the needs of the development of large-scale networks.In the hope of reducing the number of alerts that should be manually analyzed by security managers, we propose a method called alert correlation. Alert correlation is an analysis process that takes the alerts produced by intrusion detection systems and produces compact reports on the current security status of the network under surveillance.When it comes to implementation, we first decide which group of attributes should be selected as the candidates for correlation. IP address, ports, timestamp, alert type, sensor ID and so on should be selected. Then, based on the other researchers'work, we propose our own correlation algorithm. The correlation process merges the duplicate alerts produced by both the same or different sensor firstly on the basis of attribute similarities. After the merging, a meta-alert is created to represent the higher level of abstraction. After that, the alert will be given a priority value to represent its severity level. The alert with the highest priority should be firstly managed in the following processes. Next, the merged alerts will be re-organized based on the causal relations between them in order to distill the attack scenarios (also called attack graphs). Finally, the alerts are prioritized according to the site's security policy again and the results are reported (such as the damage of an attack) to security manager. If the corresponding attack graphs are already stored in the database, then the alert can be matched with the node in the graphs, and eventually make a prediction about the upcoming attacks. Then the security administrator can found the attack timely and took corresponding reaction to this kind of attack.After the implementation and a period of observation, the correlation method we have proposed can effectively reduce the number of alerts and most of the attack scenarios can be abstract from the sensor alert. In real network, the result is somewhat different because port scanning is quite common in Internet, thus the correlation process does it well with high effectiveness. However, our algorithm still has some deficiencies. Such as, the priority levels are somewhat difficult to be decided due to the lack of knowledge about the network infrastructure and the services provided. At the same time, the datasets we used cannot represent the real network attacks. Thus the result is heavily dependent on the datasets.