Rootkit Detection System Based On The Windows Operating System Research

Rootkit is a set of program or codes that can be constantly hidden and can not be detected in user's computers. it is a tool which the nalicious software can used to hide themselves and preserve the highest access-rights. it seriously endangers the highest access-rights of target computer, being able to controlling the target computer freely. What's even worse is that the constantly-developed rootkit technology can break through the rootkit detection software and make it unavailable by modifying its execution logic.In this thesis, The current situation of rootkit detection tools has been summarized. Windows operating system, such as access control mechanisms, paging and addressing modes, processes and threads, loadable kernel module (driver), the system service dispatch table, the interrupt descriptor table, PE file and the functions of system information, have been studied. Based on new technologies of rootkit, Rootkit detection tool defects and windows operating system's vulnerability, a rootkit detection system was designed in which the rootkit detection subsystem and the self-checking subsystem are mutually independent.In the rootkit detection subsystem. documents have been repeated randomly combinated to obtain MD5 digital signature to detect file integrity. The hooks have been detected through the address's acceptable range and the first-64-bytes of function; the hidden registry item have been detected by fetching the HIVE document directly; the hidden files have been detected by reading files from disks directly; the hidden processes have been detected by the kernel object handle and thread scheduling queue; In the field of the self-protection, the rootkit detecting subsystem will conceal with self relevant processes, documents, modules. And a dedicated data analysis module has been used to re-analyze the suspected data reported by the rootkit detection modules.This self-checking subsystem will be designed that rootkit detection subsystem can not be directly called. The self-checking subsystem is combinated with certification module, self-detection module, recovery module, and secret key madding module. In this thesis, certification step have been studied in detail. The algorithms of documents integrity detection have been test through experiment; communications between subsystems have also been test through experiment; the detection ablity of the rootkit detection system has been analyzed theoretically.