Detection And Function Analysis Of Undocumented Instructions For X86 Architecture Processors

As the fundamental part of hardware,processors are widely used in various electronic devices.However,they have been believed to be secure for a long time,thus few researchers pay attention on their security.In recent years,it has been discovered that there are undocumented instructions in processors due to design flaws or being left in purpose.These undocumented instructions may be used by attackers to gain privileges or cause processors to work abnormally.At present,there aren’t mature and efficient tools for the research on undocumented instructions of the x86 architecture.This paper designs and implements tools to detect,classify and analysis function of undocumented instructions efficiently.The main contents of this paper include:1.Detect undocumented instructions using the fuzzing method based on single-instruction execution.Optimizing the seed mutation algorithm,reducing time consumption,increasing detection amount,and improving the efficiency of x86 architecture undocumented instruction detection by instruction format analysis.2.Proposed a classification method for undocumented instructions based on instruction information and verify the searched undocumented instructions by high-reliability disassembler.3.Proposed a undocumented instruction function analysis method based on execution environment monitoring and port analysis.Execution environment monitoring includes registers and memory.The execution port used by uops are also collected to analyze the function of undocumented instruction.After comparison and testing,the undocumented instruction detection,classification,verification and function analysis system designed and implemented in this paper can be executed efficiently and automatically.Compared with traditional detection tool,the efficiency is significantly improved.This paper also fills in the vacancy of technology in automatic classification,verification and function analysis.