An Integrative Content Analyzing System Based On IP Network

With the rapid development of network and Information technology, more and more network application scramble for the limited network bandwidth all the time, which resulted the increasing of the difficulty of network management. At the same time, Internet was more and more popular and developed fast, some kinds of network attacks begin to be launched through the Web site as to spread illegal information, as well as in page mount Trojans, viruses and other malicious code, etc. Therefore, how to ensure network availability and smooth flow of business-critical operations, how to control the spread of illegal Internet content has become a new hotspot. For the Internet Service Providers, they need to make a clear understanding of network bandwidth, which is occupied by a wide range of applications and to analyze user traffic behavior in order to rational planning and allocating the use of the limited network bandwidth, to protect business-critical applications running effectively. At the same time, the general business-to-internal network management also needs to conduct a detailed analysis of export flows to protect the enterprise informationization development and improve the safety and effectiveness of the network. The network bandwidth used for non-productive should be controlled in order to improve the productivity of all staffs. At present, the extensive use of Firewall and Intrusion Detection System for the network traffic analysis capability is still in a low level, these systems are unable to meet higher demands a higher level of security needs. Therefore, in view of this situation, this article will analyze and design an IP-based content security analysis system, starting with the different applications from different scenes, design the appropriate technology to solve different problems. The present system is mainly designed to concern the Internet content from the network in terms of the protocols mainly refers to the transport layer and the content above the transport layer in OSI, this will be the main features of the system.This paper firstly designed and proposed the framework of the system, which is the base for the implementation of the related technologies, and then researched the key technologies used in the framework emphatically. The primary work of this paper includes:1. This paper proposed a new integrated content analyzing system based on IP network. The content analysis system is based on the two aspects as protocol identification and the content identification. Also, a series of network flow control and directional news push technologies are designed and implemented. At the same time, this papers a new scheme of network disposition.2. The protocol identification is aimed to identify and classify the network flow by the different application layer protocol. This paper researched the Deep Packet Inspection and Deep Flow Inspection technologies and proposed a new scheme of protocol identification using DPI and DFI.3. This paper also focused on the analysis of the content transferred above the protocols, which is content identification, which could be understood as a deeper level of protocol analysis which is mainly focused on identifying the real content. This paper designed and implemented a whole URL classification system, including Web spider tool, HTML parser program and text classification components. This paper also researched the malicious code mounted on the Web page and proposed a scheme of malicious code identification for the URL classification system.4. This paper researched the common network flow control technologies and proposed several methods for different flow controlling requirement, also, this paper designed and implemented a new scheme of directional news pushing, which is used to push warning information or inform the objective.