Research On Exception Detection Based On Web Log

With development and popularity of network technologies, Internet is gradually changing the communication and consumption patterns in everyone’s daily life. Web applications, such as search engines, social networking and online shopping, bring convenience to our life. However, personal privacy and data security now are facing with unprecedented challenges, as all kinds of sophisticated threaten emerge from time to time. When a website encounters an attack, generally, the attacker may leave some traces and clues in the Web log, so it is an important solution to security analysis for Web site by analyzing Web log, which can help us to learn hackers’behavior.In this thesis, after analyzing the technology on the anomaly analysis to Web log, we focus on the definition and selection of Web log attributes. Based on the well-defined attributes of Web log, we propose two anomaly detection methods for Web log, which could assistant us to analyze the Web log and discover anomaly access pattern. Our contributions of this thesis are:First, the Web log attributes are defined and analyzed from two inspects which are presentation attributes and hidden attributes. Based on these attributes we set up two kinds of anomaly detection method as following:Second, based on the defined attributes of Web log above, we propose two anomaly detection methods on Web log to analyze the anomaly access pattern:(1) The anomaly detection method based on Web application topology, including selection of attributes, pretreatment of the Web log, Web topology reconstruction. Then we test the experimental data based on selected attributes and Web topology;(2) The anomaly detection method based on the hourly access traffic, including pretreatment of Web logs, statistical analysis of access traffic, and the detection algorithm based on a linear regression model. According to the statistics analysis we know that total number of requests per hour and the total success requests per hour in the Web log are linearly correlative. The detection model is built by using linear regression model, and the regression equation was solved by the Least Squares between time and the value of total number of requests per hour divides the total number of success requests per hour, and then analyzing and detecting Web log need to test by using the detection model mentioned above.