The Research On Low-cost And Accurate Measurement Method Of Elephant Flow In High-speed Network

Network measurement is to collect statistical information on network inbound and outbound traffic,such as the number of flows,flow size,Top-k flows,etc.,by deploying measurement nodes at the egress of network switches,which provides indispensable information for network management,congestion control,quality of service,anomaly detection and other network applications.As a critical task in network measurement,elephant flow measurement is propitious to many management applications.Recently,existing elephant flow measurement usually uses data stream method on Sketch,but there are still some defects and challenges.Thus,this thesis further studies on the accuracy of Top-k flow measurement and active elephant flow measurement.Key research in this thesis can be summarized as follows:(1)For the problem of how to achieve Top-k flow measurement with high precision and low resource overhead as possible,this thesis adopts the design idea of filtering the mice flows first and then measuring the elephant flows,and constructs a low-overhead and high-precision Top-k network flow measurement architecture.The architecture eliminates a large number of mice flows through the pre-built compact mice flow filter,so as to reduce the interference of mice flow packets on the subsequent elephant flow identifier,thereby improving the accuracy of Top-k flow measurement.Further,this thesis proposes a compact mice flow filter that can be adaptively updated to ensure its continuity and effectiveness when processing large traffic,while lowering the resource overhead to deal with mice flows.Then,this thesis designs a high-precision elephant flow identifier based on segmented hashing and voting strategy to reduce the hash collision between elephant flows,thereby improving the accuracy of Top-k flow measurement.Experimental results show that the architecture has higher accuracy and less error under the same memory size than the existing Top-k flow measurement architecture.(2)For the problem of how to accurately identify the active elephant flows,while solving the hash collision caused by numerous mice flows.This thesis utilizes the idea of pre-filtering mice flows,and proposes an accurate measurement method for network active elephant flows with low overhead.The architecture is divided into two parts:the mice flow filter and the active elephant flow identifier.The mice flow filter uses a compact data structure called Sketch to quickly filter the mice flows,and the active elephant flow identifier tracks and reports the active period of the elephant flows.To reduce the interference of mice flows packets on the active elephant flow identifier,the architecture filters a large number of mice flows in the network traffic to initially separate mice flows from elephant flows,thereby improving the accuracy of active elephant flow measurement.The active elephant flow identifier uses the Hopscotch hash algorithm to efficiently store the flows passing through the filter,and further evict low-speed flows to accurately record and track the active elephant flows.Experimental results show that the proposed method has a significant improvement in the recall rate and accuracy of active elephant flow measurement.