Countering Kernel Rootkits with
Lightweight Hook Protection
Zhi Wang, Xuxian Jiang, Weidong Cui, Peng Ning
16th ACM Conference on Computer and Communications Security (CCS),
November 2009
Presentation by
Rajiv Marothu
Outline







Introduction
Example scenario
Traditional Defense Mechanisms
Motivation
Design and Implementation
Performance
Closing Remarks
University of Central Florida
Introduction - Kernel Space



The core of Operating System resides
Can be accessed through systems calls
Similar to running in real mode assembly language
University of Central Florida
Introduction - Hooking

Definition - Hook


Function pointers, return addresses, e.g.
ext3_dir_operations->readdir
Definition - Hooking

Techniques used to alter or augment the behavior of an
operating system, of applications, or of other software
components by intercepting function calls or messages or events
passed between software components.
University of Central Florida
Introduction - Rootkits


Rootkit is a software program designed to gain control
over a system or network.
Hide presence and activities


Hijack control by modifying kernel spaces
Rootkits can not only hide their presence but also tamper
with OS functionalities to launch various attacks.




Opening backdoors
Stealing private information
Escalating privileges of malicious processes
Disable defense mechanisms
University of Central Florida
Example (TDL4 Rootkit)






From the Rootkit.Win32.TDSS family
Installs in Master Boot Record
Runs before the Operating System
Blocks programs from running
Delivers advertisements
Google redirects
Source: Google/images
University of Central Florida
Traditional Defense Approaches
Three major research categories:

Analysis of rootkit behavior


Search common symptoms exhibited by rootkit infection


Panorama, HookFinder, K-Tracer, and PoKeR
Copilot, S BCFI, and Vmwatcher
Preservation of kernel code integrity by preventing code
from executing

SecVisor, Patagonix, and NICKLE
University of Central Florida
Motivation

Hijacking attack on return address and function pointers

In addition to the preservation of kernel code integrity, it
is also equally important to safeguard relevant kernel
control data

By preserving the kernel control flow integrity, it enables
the system to block out all rootkit infections in the first
place.
University of Central Florida
Contributions

Design, implementation, and evaluation of HookSafe

Hypervisor
Hooksafe - Hypervisor-based
lightweight system that can
protect thousands of kernel hooks from being hijacked
by kernel rootkits.
“In computing, a hypervisor, also called virtual machine monitor
(VMM), is one of many virtualization techniques which allow multiple operating
systems, termed guests, to run concurrently on a host computer, a feature
called hardware virtualization.” Wikipedia


Efficiency of defense against rootkits using HookSafe
Low overhead introduced using the tool
University of Central Florida
Hook Safe

Challenge: Protection granularity gap


Hook protection requires byte granularity
Hardware only provides page level protection

Kernel hooks (function pointers), after initialized have
frequent read access, less write access

Move hooks to page-aligned memory and protect with
traditional page protection


Any write access can be monitored
Small overhead effect
University of Central Florida
Experiment

They analyzed a typical Ubuntu 8.04 server using a
whole emulator called QEMU.

They used 5881 Linux Kernel Hooks

They found that these Kernel hooks are scattered across
41 Pages and some of them located in dynamic kernel
heap
University of Central Florida
Hooks per Page Histogram
Fig: Distribution of Kernel hooks in running Ubuntu system
University of Central Florida
Pages and Page Processing
Fundamental to use
non-continuous memory blocks

Creates a mapping between
a physical and virtual address,


Provides virtual RAM
Source: http://www.answers.com/topic/page-table
University of Central Florida
Problems Overview

Classification of kernel rootkits




Majority of kernel rootkits are KOH rootkits (96%)
KOH can gain control over kernel execution




Kernel Object Hooking (KOH) - hijack kernel control flow
Dynamic Kernel Object Manipulation (DKOM) - modify dynamic
data objects
Code hooks
Data hooks - most common type
Kernel hooks are scattered across kernel space
Prior techniques are not suitable for protecting significant
amount of hooks
University of Central Florida
HookSafe Architecture
Fig: Hooksafe Architecture


Offline Hook Profiler
Online Hook Protector
University of Central Florida
Offline hook profiler

It is a component that profiles the guest kernel
execution and outputs a hook access profile for each
protected hook.

Hook access profile will be used to enable transparent
hook indirection.

Kernel instructions that read or write to a hook called
Hook Access Points (HAPs).
University of Central Florida
Offline hook profiler
Static analysis



Dynamic analysis
It is Performed on OS kernel
source code,
Utilize known program analysis
technique to automatically
collect hook access profile.
More complete, but less
precise.



Doesn’t need OS kernel
source code
Run the target system on the
top of an emulator and monitor
every memory access to derive
the hook access instruction.
Allow for recording precise
runtime information, but less
coverage
University of Central Florida
Offline hook profiler
Implementation





It is based on an open source whole system emulator
QEMU uses binary translation technique which rewrites
guest’s binary instruction.
Then records executions of instructions that read or write
memories.
If instruction accesses any kernel hook it is recorded as HAP
and the value.
At the end, collected HAP instructions and values will be
compiled as corresponding hook access profile.
University of Central Florida
Offline Hook Profiler Implementation

Run in emulation and hooks are recorded with set of
read/write (HAPs) and values
Fig: Hook access profile
University of Central Florida
Online hook protector




Its input is the Hook Access Profile.
Creates a shadow copy of all protected hooks
Instruments HAP instructions such that their
accesses will be transparently redirected to the
shadow copy.
Shadow copies are moved into a centralized
location to be protected from unauthorized
modifications and kernel rootkits. (i.e. page level
protection).
Protection granularity gap problem resolved
20
University of Central Florida
Online hook protector
Three processes of design:

Initialization:
1. Uses a short-lived kernel module (temporary) to
create shadow copy of kernel hooks and load the
code for indirection layer.
2. Use the online patching that provided by the
hypervisor in order to instrument HAPs in guest
kernel.
21
University of Central Florida
Online hook protector
Run-Time Read/Write Indirection

Read Access: reads from the shadow hook copy
and returns to HAP site.

Write Access: indirection layer issues hyper call
and transfers control to hypervisor for validation
check. Memory protection component validates
write request and update shadow hook.
22
University of Central Florida
Online hook protector
Run-Time Tracking of Dynamically Allocated Hooks

Dynamically Allocated Hooks are embedded in
Dynamic Kernel Object.

If one such kernel object is being allocated, a
hypercall will be issued to HookSafe to create a
shadow copy of the hook

Another hypercall is triggered to remove the shadow
copy when kernel object is released.
23
University of Central Florida
Online hook protector
Implementation
 It is developed based on Xen Hypervisor.
 Hypervisor replaces the HAP instruction at runtime
with ‘jmp’ instruction to allow execution flow to
trampoline code in Hook indirection layer.
 Trampoline code collects runtime info which is used
by hook redirector to determine exact kernel hook
being accessed.
 After hook redirector processes the actual read or
write on shadow hook, trampoline executes HAP
specific overwritten instruction.
24
University of Central Florida
Online hook protector
Fig: Architecture of Online Hook Protection
University of Central Florida
25
Online hook protector
Fig: Implementation of hook indirection
26
University of Central Florida
Evaluation




In order to evaluate HookSafe’s effectiveness in
preventing real-world rootkits, They used
the Xen
Hypervisor (version 3.3.o) to protect more than 5900
kernel hooks in Ubuntu 8.04 Linux system.
There experiments with nine real-world rootkits show
that Hooksafe can effectively defeat these nine
rootkits attempt to hijack kernal hooks that are being
protected.
It prevented all of nine rootkits from modifying
protected hooks and hiding themselves.
This large scale protection is achieved with only 6%
slow down in system performance.
27
University of Central Florida
Evaluation
28
University of Central Florida
Closing Remarks - Strengths




Rootkit protection is performed dynamically i.e., without
need of source code
Low overhead of 6% of runtime
Works with variable instruction length architecture
Perform byte equivalent protection by using page
protection of the hypervisor.
University of Central Florida
Closing Remarks - Weakness

Do not record what caused the rootkit infection. It can
detect, but not defend against future attempts.

When discrepancy is found it automatically assumes the
original hook was compromised.

Memory usage for creating shadow copies
University of Central Florida
Suggestions

HookSafe should be tested on cross platforms

Instead of checking discrepancy between hooks and
their copy, we can try checking against a hash value to
find out which is compromised
University of Central Florida
Thank You
University of Central Florida