Research On Suspicious Program Analysis Technologies Based On Guidance Of Control Flow Information

In recent years, the number of malwares is growing rapidly with the quick development of Internet. The private data and property of many company or individuals have suffered serious threat of security. The traditional method to detect malwares is signature-based. Although it has high efficiency in detecting known malwares, but it has great limitation in detecting unknown malwares. So, researchers have proposed many methods to solve this problem such as behavior-based. However, as many anti-analysis technologies were used in malwares code, it’s very hard to detect malicious behaviors. Therefore, some effective methods are especially expected to restraint anti-analysis technologies and recognize malicious behaviorsThis thesis presents a technology of analyzing suspicious program based on guidance of control flow information, which is introduced from both multi-path analysis technology based on guidance of control flow information and anti-anti-virtual execution technology based on critical points reuse. Analysts can identify the malicious program behavior effectively by using the above technologies. The main contents and contributions of this thesis are as follows.1. This thesis researches program analysis and anti-analysis technologies, and then proposes suspicious program analysis technology based on guidance of control flow information. The technology can partially solve the problems of bad disassemble efficiency in static analysis and low code coverage efficiency in dynamic analysis technology, and can be used for multi-path analysis and anti-anti-virtual execution.2. A multi-path analysis method based on guidance of control flow information is presented. In order to obtain more comprehensive program behaviors, multiple paths should be analyzed while analyzing a program. The existing multi-path analysis methods have low efficiency for they don’t select path purposively. Therefore, a multi-path analysis method based on guidance of control flow information is proposed. Through determining the analysis paths, extracting guidance information from control flow graph and analyzing multiple paths with the guidance information, key behaviors can be obtained. For only analyzing important paths, the method can obtain important behaviors with high efficiency.3. A critical points detection method based on deterministic finite automata model is presented. Detecting critical points is the key of the anti-anti-virtual execution method. Therefore, this thesis presents a critical points detection method based on deterministic finite automata model. After taking the method, the signatures of anti-virtual execution methods, such as instruction, instruction sequence and function sequence are effectively recognized. 4. An anti-anti-virtual execution method based on critical points reuse is presented. Malicious behaviors always can’t be detected in virtual environment for anti-virtual execution technologies are taken by malware. Therefore, this thesis proposes an anti-anti-virtual execution method based on critical points reuse. The method has three steps:extracting the signatures of the anti-virtual execution methods to construct a data-base, detecting critical points by using the detection method based on deterministic finite automata model, implementing anti-anti-virtual execution through critical points reuse. This method could efficiently detect malicious behaviors in virtual environment.The above methods have been applied to the analysis tools developed by our project team, and the validity and generality have been proved.