Research On Binary-Executable-Oriented File Format Vulnerability Detection

In recent years, information security has become increasingly prominent, more and more attention was paid on information security research, software security is one of the important branches. File format vulnerability is one of the important class of software vulnerabilities. It’s a vulnerability type that triggered by file input. It is of great significance in the field of software security.Fuzzing is a common binary program vulnerabilities mining technology. Most of the existing Fuzzing technique are unidimensional fuzzing which mutate only one input data element at one time. In fact, many of the actual program code segments are simultaneously affected by multiple input elements. Some potential vulnerabilities need more than one input elements meet certain conditions before they can be triggered. In this case, unidimensional fuzzing cannot be able to meet the needs of vulnerability mining.This paper studies a multi-dimensional Fuzzing technology. Firstly, a binary code vulnerability assessment method based on a static analysis is proposed to locate fragile points in program. Secondly, a runtime monitoring method is proposed to establish the mapping between fragile points and file data elements. Then, mutate multiple data elements in file at the same time to fuzz the specific fragile points. In this paper, a distributed file format fuzzing framework based on asynchronous programming model is designed and implemented to improve the efficiency of fuzzing framework.On the basis of these studies, a prototype system called DMFFuzzer is developed. It has been used to fuzz some well-known office software and media player software. Several Oday vulnerabilities are found. Among them, there are a0day vulnerability of Apple QuickTime Player and VLC media player. They were submitted to the China National Vulnerability Database of Information Security (CNNVD). Those two vulnerabilities are discussed in detail in this paper.