Research And Implementation Of Key Technologies Of Software-Hardware Co-Designed Regular Expression Matching System

Deep packet inspection technology is used by network intrusion detection systems to perform a deep analysis of network traffic and identify potential security threats.Deep packet inspection generally consists of three processing steps: Packets Classification,String Matching and Regular Expression Matching,while Regular Expression Matching is the most complex and time-consuming part of the process.With the rapid growth of network traffic and the increasing frequency of security rule base updates,regular expression matching engines are required to: 1)Have a defined high performance;2)Support large-scale rule sets;and 3)Support fast rule updates without service interruption.At the moment,achieving all three goals concurrently using software-only or hardwareonly regular expression matching engines is difficult.To design and implement an efficient and flexible regular expression matching system,we use a Hardware Software Co-design approach in this thesis.The basic principle is that the hardware matching engine matches a small number of complex rules,while the software matching engine matches the remaining rules.The hardware regular expression matching engine is designed and implemented based on FPGA.The rule update time is 1μs,which supports the real-time offloading of complex rules.The following four aspects are covered in detail in this thesis:(1)The overall structure of the regular expression matching system with hardware software codesign.Experiments use the Snort regular expression rule set,CIC IDS2017,Mix and CN7 network traffic datasets to analyze the advantages and disadvantages of the software matching engine and the FPGA-based hardware matching engine,to illustrate the necessity and feasibility of implementing regular expression matching with hardware software co-design,and to give the overall structure of the soft and hard collaborative matching system.The analysis show that hardware software co-design can balance the performance and cost of system implementation,and is an inevitable choice to realize a high-performance regular matching system supporting large-scale rule sets.It is not necessary to offload all rules to hardware,and only a small number of rules can significantly improve the performance of the software matching engine.There is no need for static offloading,and dynamic offloading can greatly save hardware resources.(2)Design and implementation of a high-performance hardware-software communication interface module.The software and hardware high speed interface supporting multiple queues was designed and implemented using the Xilinx QDMA IP core and the Intel DPDK polling mode driver,which provides a basic channel for the software to reconfigure the rule base of the FPGA regular expression matching engine quickly.Under single core conditions,the interface has a maximum data rate of 94.7Gbps(4096 bytes of message length)and a maximum packet rate of 14.52Mpps(64 bytes of message length).In the multi-core case,the average data rate and packet rate are increased correspondingly.(3)Design and implementation of a rapid reconfigurable regular expression matching engine based on FPGA.The technology for designing and implementing a rapidly reconfigurable regular expression matching engine based on FPGA is investigated.The shortcomings of existing implementation methods are improved,and the FPGA hardware design resource consumption,reconfiguration time,throughput rate,and result reporting efficiency are considered in order to implement a rapidly reconfigurable regular expression matching engine based on Ultra RAM cache,which consumes 5.5 percent of logic resources and 17.1 percent of storage resources.The matching engine can update rules without service interruption.The update time is 1μs and the throughput can reach 2Gbps.Compared with similar research,the resource consumption of topology matrix is reduced by 50%,the reconfiguration time in the worst case is increased by at least 4 ～ 5times,and the throughput of single character processing is increased by 14.3% ～ 64.4%.Experiments using real network traffic show that FPGA regular matching engine improves the throughput of software matching engine Hyperscan by 33 times.(4)Design and implementation of a hardware regular matching engine configuration driver.The conversion of regular expressions to hardware matching engine configuration files is automated using open source tools and the conversion algorithm provided in this work.The matching engine configuration driver is designed based on the basic transceiver driver.An automatic matching engine generation tool is designed.Test results show that the configuration driver can achieve correct configuration of the hardware matching engine,and the average time for compiling a single rule for Snort(V2.9),Snort(ANMLZOO)and Power EN rule sets is 0.259 s,0.173 s and 0.119 s respectively,and the speed of compiling new rules is two orders of magnitude faster than Hyperscan.In summary,this paper investigates and implements three key technologies,namely the hardware-software high-speed interface,the FPGA-based rapidly reconfigurable regular expression matching engine,and the hardware matching engine’s configuration driver,all of which can provide key technical support for the implementation of a flexible and efficient deep message detection system.