Research Of JVM Escape And JRE Vulnerability Discovery

Java is one of the most popular Object-Oriented programming languagesand such a lot computers have installed Java Runtime Environment(JRE).The security of JRE has become vital. Nowadays, amount of JVM escapeattack which taking advantages of JRE vulnerabilities and becoming greatthreat to personal computers is increasing. Therefore, the research of JREsecurity features, JRE vulnerability discovery and JVM escape attacktechniques is becoming one of the top research in software security.According to Java API and native API, architecture of JRE securitycontains JRE sandbox and JVM type security features. Analysis work anddrawbacks of JRE sandbox modules are given in this paper, whileweaknesses of JVM type security features is vulnerabilities in native API.JRE vulnerability discovery is based on this analysis work.Classification of JRE vulnerabilities is done according to their features.Analysis of typical vulnerabilities in Java API design and native API isgiven in this paper, leading to modeling of JRE vulnerabilities. Source code audit based on models in JRE vulnerability analysis hasbeen used in Java API design vulnerability discovery. Several vulnerabilitiesare found in Oracle JRE, OpenJDK and Apple JRE. Furthermore, this paperintroduces means of symbolic register monitoring based on symbolicexecution technology in the field of program analysis. Symbolic executionframework S2E is choosen for support. SymJava and SymRegMonitor hasbeen developed as assistant plugins on S2E. Java testcases for vulnerabilitymining is constructed through source code audit to OpenJDK and reversioncode of Oracle JRE. The test has found6JRE native issues in36testcasesbased on Java Native API, including2vulnerabilities that can be exploited.Following is Analysis and PoC.Considering attack and defense of JVM escape attack,5key factors inJVM escape attack are given in this paper. JVM escape attack techniqueswhich can be exploited in bypass static scanning of anti-virus software areproposed. Finally, defense strategies are raised in the end of this paper.