Investigating Complexity Metrics as Indicators of Software Vulnerability

A single exploited software vulnerability can cause severe damage to an organization legally and financially. Early detection of software vulnerabilities can prevent the damage caused by late detection. Security experts claim that complexity is the enemy of security. A complex software system is difficult to understand, maintain, and test by software engineers resulting in errors in code including vulnerabilities. As a result, finding metrics that can measure software complexity and can point toward the code locations that are likely to have vulnerabilities early in the development life cycle is beneficial.;The goal of this research is to investigate complexity metrics that can indicate vulnerable code locations to improve the efficiency of security inspection and testing. For this purpose, this research conducts empirical evaluation of four types of complexity metrics: code complexity; OO design complexity; dependency network complexity; and execution complexity metrics as indicators of vulnerability. The evaluation is performed on four widely used open source projects by testing whether complexity metrics can discriminate vulnerable and neutral code locations and whether the prediction models built using those complexity metrics can predict vulnerable code locations. While complexity metrics have long been used for fault prediction, faults have different distributions from vulnerabilities. Therefore, this research additionally compares the ability of traditional fault prediction models and vulnerability prediction models to see whether traditional fault prediction models can also effectively predict vulnerabilities. Finally, software metrics that quantify code change history and developer collaboration history have been effective for fault prediction. Therefore, this research compares the ability of complexity metrics and other types of metrics obtained from development history as indicators of vulnerabilities. This research improves our understanding on the relationship between software complexity and vulnerability, contributing to the body of empirical knowledge as follows: •This research provides empirical evidence that complexity metrics can indicate vulnerable code locations. •This research provides empirical evidence that vulnerable code is more complex, has large and frequent changes, and has more past faults than faulty code. •This research provides empirical evidence that fault prediction models that are trained to predict faults can predict vulnerabilities at the similar prediction performance to the vulnerability prediction models that are trained to predict vulnerabilities despite the difference in the distribution of faults and vulnerabilities. •This research provides empirical evidence that code execution frequency and duration based on software usage patterns by a normal user can indicate vulnerable code locations. •This research provides empirical evidence that process metrics are better indicators of vulnerabilities than complexity metrics when process metrics are available. •This research defines and uses simple and useful measures of code inspection cost and code inspection reduction efficiency obtained from a prediction model. •This research demonstrates that automated text classification is feasible and useful to classify bug reports for faults and enhancements. •This research reveals that a careful analysis of the relationship between faults/vulnerabilities and software metrics is required because the analysis results largely depend on the distribution of faults/vulnerabilities and the distribution of faults/vulnerabilities is specific to each project.