1
SECURITY OF THE
SECURITY TOOLS
Fu-Hau Hsu
National Central University
Background
2


Plenty of critical personal information is stored in
computers
Many commercial activities are proceeded through
computers and networks.
Results
3

Computers become a major battlefield between
attackers and computer users.
Solutions
4

Diverse security tools,have been developed to
handle this critical work.
 anti-virus
software
 auto-patch mechanisms
 firewall
 intrusion detection systems
 and so on
New Problem
5



The complexity of these security tools increases.
The security tools become attack targets.
The security of the security tools also becomes a
crucial issue.
Disable Anti-virus Software
6

Research has shown that diverse approaches have
been developed to disable different anti-virus
software.
Problems (1)
7

How to guarantee the normal operation of antivirus software.
Problems (2)
8

If attackers can disable anti-virus software, is it
possible that they can also change the behavior of
the anti-virus software through replacing, injecting,
or hooking code?
Impacts of Anti-virus Parasites
9

Given the fact that most, if not all, computers allow
only one anti-virus application to be installed on
them at one time, an attacker that can control the
anti-virus application of a computer can fully
command the computer without being detected.
Auto-Patch
10

How could a computer guarantee that the autopatch code it executes is not malicious code?
Impacts
11

If the correctness of auto-patch code cannot be
protected, instead of being a tool to seal the
security breaches of a computer, auto-patch may
become an efficient channel for attackers to intrude
a computer or spread malicious code.
12
Antivirus Software Shield against the Antivirus
Terminator
Purpose
13

We proposed ANtivirus Software Shield (ANSS) to
protect antivirus software from being terminated by
malware.
Arms Race between Malware Writers
and Antivirus Programmers
14

The simplest way for a computer user to secure his
computer is to install antivirus software on his computer.


81% of computer users utilize antivirus software to protect
their computers, which means over a hundred million
computer users use antivirus software in the USA alone.
Attackers attempt to break any defense mechanism by
masquerading
 social engineering
 hindering antivirus software detection

Malware Self-Defense Techniques (1)
15

Disorder signature-based virus detection methods by
 Transforming
 Encoding
 Compressing
 Packing

malware code.
Avoid executing themselves when they are monitored by
 debuggers
 hypervisors.
Malware Self-Defense Techniques (2)
16


Unmeaning bytes may also be inserted into
malware to hinder static malware analysis.
More aggressive and dangerous malware may even
directly disrupt the functionality of security solutions,
especially antivirus software
Antivirus Terminator
17

Malware that terminates the execution of antivirus
software without the consciousness of the antivirus
software users is called an antivirus terminator.
Who Protect Antivirus?
18

The most effective solution to antivirus terminators is
antivirus software itself.
The Problem
19

Even though antivirus companies claim that they
have implemented protection mechanism to protect
their products against antivirus terminators, our
experimental results shown that techniques utilized
by antivirus terminators to terminate antivirus
software still could terminate antivirus software.
20
Approaches to Terminate Antivirus Software
Null Debugger Method
21



An antivirus terminator can use API
DebugActiveProcess to attach itself to an
antivirus process as a debugger to control it.
Then right after invoking DebugActiveProcess,
the antivirus terminator exits.
Because the controller of the antivirus process does
not exist, the process will crash immediately.
Dll Unloading Method
22


An antivirus terminator can use API
ZwUnmapViewOfSection to unload some
important dll files, such as ntdll.dll, from an
antivirus process to erase some portions of the
virtual address space of the antivirus process.
ntdll.dll defines many common used windows
Native APIs, hence, once the dll file is unloaded
from the antivirus process and an API in the dll
file is invoked, the process will crash.
Process Termination Method
23


An antivirus terminator gets the handle of an
antivirus process by calling API OpenProcess
with the process ID of the antivirus process.
Then the antivirus terminator can use APIs
NtTerminateProcess or
ZwTerminateProcess to terminate the
antivirus process in the kernel level.
Close Message Method
24


An antivirus terminator can use API FindWindow
to search all windows running on the system to find
the window matching the name of an antivirus
software window (e.g. avguard of antivir or
kavsvc of kaspersky).
Then the antivirus terminator continues sending
messages, such as WM_CLOSE or WM_QUIT, to the
related antivirus process by APIs SendMessage
or PostMessage until the process is terminated.
Mouse Simulator Method
25


An antivirus terminator can use API SendInput to
counterfeit a series of mouse events which lead to the
suspension of an antivirus process.
E.g.:
An antivirus terminator may find the icon of an antivirus
process first.
 Then it forges a series of mouse events to move the cursor to
the right lower corner of the screen, press the right button to
expand a work menu, and then chose to suspend an
antivirus process.


This approach was designed by us. Hence, we have not
found an antivirus terminator in the wild utilizing this
approach to terminate antivirus software.
Registry Modification Method (1)
26


An antivirus terminator can modify the registry so
that a NULL debugger will be attached to an
antivirus process when the antivirus process begins
its execution.
The above steps will stop the execution of the
antivirus process.
Registry Modification Method (2)
27



Besides, an antivirus terminator can also modify the
registry to delete antivirus related processes from the
startup process list; hence, the system will be booted
without the protection of an antivirus process.
API ZwOpenKey can be used to open a registry key.
And API ZwSetValueKey can be used to modify a
registry key value.
Thread Termination Method
28

An antivirus terminator can use API
TerminateThread to terminate the threads of
an antivirus process one by one till the antivirus
process stops.
Table I: Methods Utilized by Various Malware to
Terminate Antivirus Software
29
Combine Two or More Terminating
Approaches
30

As Table I shown, attackers may combine two or
three techniques together to strengthen their
malware.
Market Share Survey
31
Most Popular Antivirus Software
32

5 famous brands of antivirus software were used in
our tests.
 Avira
Antivir Personal
 Kaspersky Anti-virus 2012
 Norton Antivirus 2011
 ESET NOD32 Antivirus 4
 AVAST Free Antivirus 6.
Terminator Program
33


For each of the above methods we wrote a program,
called terminator program, to see whether the approach
alone can terminate antivirus.
None of them deems the above 7 programs as malware.
Impact of Terminator Programs
34


The results show that if a piece of malware bundles
itself with one or more of the first 6 terminator
programs; it becomes an effective antivirus
terminator.
As a result, we believe antivirus terminators will
become a severe threat to many computer systems
which are usually protected by antivirus.
Frangible APIs
35


The Windows native APIs that are executed when
the above 7 techniques are used to terminate
antivirus software are called frangible APIs .
The native APIs may not be called directly by
antivirus terminators, but are called by the APIs
utilized by the antivirus terminators.
List of Frangible APIs
36
Intercept Windows API Calls
37



ANSS monitors invocations to frangible APIs.
Frangible APIs calls with parameters that will stop
or suspend antivirus software are blocked by ANSS
directly.
ANSS uses SSDT (System Service Descriptor Table)
hooking to intercept and examine invocations of
frangible APIs.
System Service Dispatch Table
38


The Windows kernel provides various system
services for application programs to use.
Inside the kernel the addresses of these system
services are stored in the SSDT,
KiServiceTable.
Usage of SSDT
39


When a program issues a system call to execute a
system service, kiSystemService looks up the
SSDT to find the address of the corresponding
system service.
Then kiSystemService uses the address to
invoke the system service.
SSDT Hooking
40


Through SSDT hooking, ANSS modifies some function
addresses stored in the KiServiceTable service
descriptor table and replaces them with ANSS API
handlers.
After an ANSS API completes its task, it invokes the
original API.
Execution Flow of a System Call with and
without SDDT Hooking
41
ANSS Structure
42

ANSS consists of three components: interceptor, filter,
and blocker.
Interceptor
43


When a program is going to invoke a frangible API,
the interceptor intercepts the execution flow of the
program and transfers the execution flow to the
filter before the code of a frangible API is
executed.
ANSS utilizes SSDT hooking to accomplish the
above task.
Frangible APIs and ANSS APIs
44


For each frangible API, there is a corresponding
ANSS API in the ANSS filter.
In other words, an invocation to a frangible API is
transferred to its corresponding ANSS API first
through SSDT hooking.
Filter
45


Based on some build-in rules, the filter examines the
parameters of the API call to filter out the intention
to abuse the API that will terminate or suspend
antivirus software.
For a normal API call, the filter transfers the
execution flow back the original frangible API code.
Blocker
46


An API call with a malicious intention is further
passed to the blocker which returns an error code,
ACCESS_DENY, as the output of the API call.
Besides, the blocker prints out an error message to
notify users that it just blocked an attack on the
antivirus software.
Filter Rules
47

The ANSS filter utilizes various rules to prevent
frangible APIs from being executed with malicious
parameters.
Antivirus Process Handle Rule
48

The ANSS filter restrains an application from
obtaining the process handle of an antivirus
program, because once an attacker obtains the
handles of an antivirus program, the attacker can
terminate the program easily.
 To
get the process handle of an application program,
attackers can use API OpenProcess which utilizes
frangible API NtOpenProcess inside the kernel
land to finish its work.
NtTerminateProcess Rule
49

The filter does not allow an application to use
NtTerminateProcess to terminate antivirus
software.
DebugActiveProcess Rule
50


The filter does not allow an application to utilize
DebugActiveProcess to attach itself as a
debugger of antivirus software.
DebugActiveProcess uses frangible API
NtDebugActiveProcess in the kernel address
space to accomplish its task.
PostMessage Rule
51


The ANSS filter prohibits a program from sending
messages, such as WM_CLOSE or WM_QUIT, to
antivirus software, because an antivirus terminator
can use API PostMessage to send the above
messages to the antivirus software to stop it.
API PostMessage sends messages through the
kernel land frangible API NtUserPostMessage.
Registry Key Rule
52

The filter disallows an application to modify or
delete Windows Registry keys to protect antivirus
software, because attackers can use registry APIs,
like NtDeleteKey, NtSetValueKey, and
NtCreateKey to disable antivirus software.
NtUnmapViewOfSection Rule
53

The ANSS filter forbids an application to use
NtUnmapViewOfSection to unload a .dll
file from antivirus, because an antivirus terminator
can use the frangible API to stop antivirus software.
SendInput Rule
54

When the foreground process is antivirus software,
the ANSS filter inhibits an application to use API
SendInput to deliver mouse events to the process,
because an antivirus terminator can use API
SendInput to imitate a series of action leading
to the termination of antivirus software.
Limitation
55


ANSS uses SSDT hooking to intercept frangible APIs
calls.
If after ANSS is installed in a computer, an antivirus
terminator can bypass the detection of the antivirus
software in the computer and hook the system too,
then ANSS may no longer work as expected.
Solution
56

To provide a thorough protection, ANSS needs to
work with the anti-hooking mechanism which is a
well studied issue in the literature.
Evaluation - Compatibility
57

We made diverse experiments to test whether
ANSS influences the operations of normal
applications.
Compatibility Evaluation Samples
58


We use false positive tests to evaluate the influence
caused by ANSS upon normal applications.
There are two types of applications used in our tests.
 The
first type of application is antivirus software.
 Any non-antivirus application is classified as the second
type.
Antivirus Compatibility Evaluation
59



For antivirus software, we installed ANSS first.
Then we manually disabled the antivirus software,
and manually enabled the antivirus software to
check whether these operations proceeded as usual.
Besides, we also
 updated
the virus definition files
 made regular system scan
 shut down the operating system.

All operations proceeded as usual.
Result
60
Non-Antivirus Compatibility Evaluation
61


We installed ANSS in 8 our daily used computers to
test if ANSS influences their normal operations.
Then we used these computers as usual.
Hardware Specification of Test Hosts
62

These computers had installed
 Microsoft
Windows XP Professional Service Pack 3
with
 Intel 2.66 GHz Core 2 Quad CPU Q9400
 1.96 GB RAM.
Software Used in the Test Hosts
63

Besides, lots of popular applications, such as
 Firefox
 Windows
LiveMessenger
 iTunes
 Oracle
VMVirtual Machine
 Avira AntiVir Personal
 Microsoft Office
 Adobe Reader.
Result
64


After 6 months’ tests, none of the applications or
computers was disabled, crashed, or blocked by
ANSS.
In other words, ANSS did not influence the
operations of applications and computers during
our test periods.
Evaluation - Functionality
65

We made diverse experiments to test whether
ANSS can protect antivirus software.
Functionality Evaluation Samples
66

Then we used the 18 antivirus terminators that we
abstracted from 2063 malware samples to test the
5 famous brands of antivirus software:
 Avira
Antivir Personal
 Kaspersky Anti-virus 2012
 Norton Antivirus 2011
 ESET NOD32 Antivirus 4
 AVAST Free Antivirus 6
Challenge
67


However, the virus definition files of some of the
above antivirus software have already had the
signatures of the antivirus terminators used in our
tests.
Hence, before the antivirus terminators execute, they
will be detected and blocked by antivirus software.
Solution
68


But our experiments only want to test whether the
mechanisms that antivirus terminators use to
terminate antivirus software can really close
antivirus software.
Hence, when making our experiments, we
temporarily did not allow antivirus software to scan
files so that we can accurately test the terminating
mechanisms without interference.
Result
69


Experimental results show that every antivirus
terminator can terminate some of these antivirus
programs.
However, if these antivirus programs were
protected by ANSS, none of the antivirus
terminators can close them.
Survivability of Various Antivirus Programs with
Or without ANSS
70
Performance Evaluation Environment
71

We used Windows benchmark tool called
PassMark Performance Test to test the CPU and
memory performance overhead of ANSS on a
Windows XP system with
 Intel
2.66 GHz Core 2Quad CPU Q9400
 448MB RAM.
Performance Overhead Measurement
72

Performance overhead measurement was
performed under two different system setups.
 In
the first setup, we installed a Windows XP system
which did not execute ANSS.
 In the second setup, we installed a Windows XP system
that executed ANSS.


Under each setup, we executed PassMark a
hundred times.
For each benchmark, we use its average value to
represent its final value.
CPU Performance Overhead
73
Memory Performance Overhead
74
Conclusion
75




Experimental results show that many commercial
antivirus programs will be closed by antivirus
terminators.
However, when being protected by ANSS, antivirus
software can function as usual even under the attacks of
antivirus terminators.
ANSS introduces at most 0.42% CPU performance
overhead and 1.77% memory write performance
overhead.
Therefore, ANSS is an effective and efficient solution
against antivirus terminators.
76
Thanks