Detecting Kernel-Level Rootkits Through Binary Analysis

A rootkit is a collection of tools used by intruders to keep the legitimate users and administrators of a compromised machine unaware of their presence. Originally, rootkits mainly included modified versions of system auditing programs (e.g., ps or netstat on a Unix system). However, for operating systems that support loadable kernel modules (e.g., Linux and Solaris), a new type of rootkit has recently emerged. These rootkits are implemented as kernel modules, and they do not require modification of user-space binaries to conceal malicious activity. Instead, these rootkits operate within the kernel, modifying critical data structures such as the system call table or the list of currently-loaded kernel modules.

This paper presents a technique that exploits binary analysis to ascertain, at load time, if a module’s behavior resembles the behavior of a rootkit. Through this method, it is possible to provide additional protection against this type of malicious modification of the kernel. Our technique relies on an abstract model of module behavior that is not affected by small changes in the binary image of the module. Therefore, the technique is resistant to attempts to conceal the malicious nature of a kernel module.