| With the rapid spread and spring up of Internet technology, WWW is becoming the largest public data in the world. The quantity of Web site and content is increasing at an exponential rate, and it is hard for users to choose what they intend to find from the vast information. For the Web site designers, it is vitally significant to decide how to provide personalized service, mining site commercial value r, improve the site structure design according to the user’s browsing behavior.The records of user access to the Web site, the Web server and the proxy server will be recorded in a certain format. Web log mining is that area of Web mining which deals with the extraction of interesting knowledge from logging information provided by Web servers. The process of data preprocessing is the most fundamental, complex and important step in Web log mining process.This paper mainly works on the algorithms and system applications of the data preprocessing in Web log mining. The main works are as follows:Firstly, this paper systematically introduces the concept and classification of data mining, Web data mining, and then the concept and process of Web log mining, with elaborating Web log mining technology and its processes, mainly focusing on the related method and system applications in the process of data preprocessing.Secondly, this paper realizes the common algorithms in the Web log mining preprocessing stage including data cleaning, user identification, session identification, path supplement and so on. Due to the insufficiency for statistical language model used in the session identification, I propose to use the ERR, SER and F-measure as an evaluation method for the basic parameter selection and evaluation of the system, providing the session identification solution. The experimental simulation proves it a higher application value.Thirdly, with the session recognition results of the preprocessing stage as a data source, this paper focuses on the request prediction and cache strategy based on the N-gram model. According to the request preload algorithm based on N-gram model proposed by Zhong Su et al, this paper raises an optimization method which loads multiple prediction request in advance based on the current user navigation paths, and optimization experiments show a better hit rate than the original program. Finally, with the combination of existing cache strategy, propose the site request prediction and caching scheme based on the N-gram model. However, this scheme requires online parameters adaptation with a combination of specific application environment. |
