Research On Network Heavy Flow Measurement

Network traffic measurement is an effective way for network behavior research and it isalso the basis for the management and control of the internet.With the development of highspeed network,traffic measurement is facing the storage problem from mass data,whichpresents a great challenge on the storage capacity and speed of the measurement system.Themeasurement based on flow saves a lot of storage space,and it gives a new way for networktraffic measurement.Research shows that the number of flows in the internet is big,but the flows shows avery strong characteristic:heavy tailed distribution,which means that9%of the data flowsoccupies about90%of all the packets.Therefore, to understand the heavy flows can have agood grasp of network communication.In recent years,with the growing of the networkscale and an unprecedented increase of network speed,the study of large flow is becomingmore and more important and it is also a hot point on network traffic measurement.Study onthe efficiency and accuracy of flow measurement algorithm has very important significance inthe present.This paper firstly discusses the relative algorithm of network traffic measurement, thenmakes a new algorithm based on hash and sampling technology.Specifically, the researchwork includes:(1) The algorithm of this paper has separated flow detection and flow storage,and it canfind out the heavy flows in the network effectively.In the flow detection module,thetraditional CBF (counting Bloom filter) is improved.The improved Counting Bloom Filteruses a multilayer structure, which saves a lot of storage space compared with the traditionalone, and it can effectively prevent the overflow problem in CBF;In the flow storage module,LRU structure is used and it can be achieved using a two-way linked list, which can find outthe heavy flows in the network efficiently.By theoretical analysis, LRU-MCBF flowmeasurement algorithm has small space occupation,low_time complexity and it is verified tohave a low missing rate and false rate by the simulation experiment.LRU-MCBF can find outthe heavy flows correctly in the high-speed network.(2)Combine the sampling technology with the standard LRU-MCBF.In the heavy flowdetection of high-speed network, the host computer needs to be more and more quick in theprocessing speed,and it is a constant pursue in the network.Heavy flow measurement basedon the sampling algorithm is a good method in the network measurement.Sampling algorithmis easy to implement, and can improve the speed of host computer in processing a packet onthe premise of accuracy,which is a kind of "low cost, high performance" measurement.Inthis paper, the sampling and non sampling algorithm are compared, which proves thatsampling algorithm has the accurate and efficient characteristics and it play a very importantrole in the heavy flow measurement.