| As various applications in the Internet become increasingly enriched, the Internet users hope that the Internet can be provided as a network conducive to the physical and psychological development by way of avoiding seeing bad information; and the company's managers hope to avoid disclosure confidential information on the Internet by employees; the government further expects that audit to the network can be carried out at the level of web content as to keep the the reactionary remarks and pornography that will produce bad impact on national and social stability out of the Internet. Hence, the demand for web content audit is badly needed. It is particularly essential to research on the key technologies in regard to the web content audit system and then actualize it.Aiming at research and implementation of key technologies of web content audit system, this paper on the basis of related research at home and abroad, focuses on the following two key technologies and implementations:In the existing web content audit system, the performance of network protocol assembly is relatively low in general, and it cannot meet the need of content audit under the high-speed network traffic. This paper studies the technology of network protocols in parallel to assemble to improve the performance of the network protocol assembly, and then proves of its efficiency and advantage through experiments, and further indicates the parallel assembly technology of network protocols is suitable for web content audit system and can meet the content audit under the high-speed network traffic needs.The web content audit system needs to handle a large number of repeated visits to URL. And the existing web content audit system for repeated URL filtering and determining is limited by the space storage or limited by the filtration rate. This paper makes researches on the URL filtering technology based on the judgment of repeat visits and applies file directory addressing method improved MMAP to it so as to in consequence narrow the URL storage space down on the one hand and increase the rate of duplicated URL removal on the other. That applies MMAP based file directory improved addressing method and hash table based URL filter to web content audit system, can improve the rate of the duplicated URL judgment and filtering.Finally, this paper implements a web content audit system. Both the theory and the practice have proved that the application of the key technologies in this study can improve the performance of web content audit system whereby meet the demands of the web content audit under the high-speed network traffic. |
