Lecture 17:
Defeating Malcode
(Shameless Self-Promotion)
CS551: Security and Privacy
University of Virginia
Computer Science
David Evans
http://www.cs.virginia.edu/~evans
Menu: Malcode Defenses
• Constrain program behavior
– Reference Monitors
• In-line Reference Monitors: Naccio
• Prevent possibly harmful code from
running
– Safe Languages
– Proof-Carrying Code
• INFOSEC Panel Talk
27 July 2016
University of Virginia CS 551
2
Program Execution
Monitor
Program
Speakers
Network
Disk
27 July 2016
Memory
University of Virginia CS 551
SuperSoaker 2000
3
Program Execution
Reference Monitor
Monitor
Program
Speakers
Network
Disk
27 July 2016
Memory
University of Virginia CS 551
SuperSoaker 2000
4
Ideal Reference Monitor
1. Sees everything a program is about to do
before it does it
2. Can instantly and completely stop
program execution (or prevent action)
3. Has no other effect on the program or
system
Can we build this?
Probably not unless we can build a time machine...
27 July 2016
University of Virginia CS 551
5
Real
Ideal Reference Monitor
most things
1. Sees everything a program is about to do
before it does it
2. Can instantly and completely stop
program execution (or prevent action)
limited
3. Has no other effect on the program or
system
27 July 2016
University of Virginia CS 551
6
Operating Systems
• Provide reference monitors for most
security-critical resources
– When a program opens a file in Unix or
Windows, the OS checks that the principal
running the program can open that file
• Doesn’t allow different policies for different
programs
• No flexibility over what is monitored
– OS decides for everyone
– Hence, can’t monitor inexpensive operations
27 July 2016
University of Virginia CS 551
7
Reference Monitor as Finite
State Automaton [Schneider99]
All other
instructions
0
All other
instructions
Aim
Aim
1
Aim
Fire
2
All other
instructions
Fire
STOP
Policy Violation
27 July 2016
University of Virginia CS 551
8
SASI (Security Automaton SFI
Implementation)
• [Erlingsson2000]
• Extend SFI to enforce arbitrary safety
policies
• Compile the policy state machine into
the untrusted program
• Try to optimize out unnecessary code
27 July 2016
University of Virginia CS 551
9
What’s SFI?
• Software Fault Isolation [Wahbe, Lucco,
PLDI ’1993]
– Collusa Software (bought by Microsoft)
• Get memory safety by inserting
checking instructions around load, store
and jump instructions
• Do clever things (use delay slots,
masking references) to make overhead
low (~5%)
27 July 2016
University of Virginia CS 551
10
SASI Example
All other
instructions
x := 3
aim (x);
fire ();
fire ();
fire ();
0
All other
instructions
Aim
1
Aim
Aim
Fire
2
All other
instructions
Fire
STOP
Policy Violation
27 July 2016
University of Virginia CS 551
11
SASI Example
STATE := 0
if (STATE = 1)
else if (STATE
x := 3
if (STATE = 0)
else if (STATE
else if (STATE
aim (x);
if (STATE = 0)
else if (STATE
else if (STATE
fire ();
...
27 July 2016
then STATE := 0
= 2) then STATE := 0
then STATE := 1
= 1) then STATE := 1
= 2) then STATE := 1
then goto VIOLATION
= 1) then STATE := 2
= 2) then STATE := 0
University of Virginia CS 551
12
What policies can be
enforced?
• Assume:
– Security Automaton can see entire state of
world, everything about instruction about to
execute
– Security Automaton has unlimited memory,
can do unlimited computation
• Are there interesting policies that still
can’t be enforced?
27 July 2016
University of Virginia CS 551
13
What’s a Security Policy?
• What’s a program?
– A set of possible executions
• What’s an execution?
– A sequence of states
• What’s a security policy?
– A predicate on a set of executions
27 July 2016
University of Virginia CS 551
14
More Formally...
•  : set of all possible executions (can
be infinite)
• S: set of executions possible by target
program S
• P: security policy
set of executions  Boolean
S is safe iff P (S ) is true.
27 July 2016
University of Virginia CS 551
15
Reference Monitors cannot
enforce all Security Policies
• Some policies depend on:
– Knowing about the future
• If the program charges the credit card, it must
eventually ship the goods
– Knowing about all possible executions
• Information flow – can’t tell if a program reveals
secret information without knowing about other
possible executions
• Reference Monitors can only know
about past of this particular execution
27 July 2016
University of Virginia CS 551
16
Safety Policies
• Reference monitors can only enforce
safety policies
• Safety policy is a predicate on a prefix
of states (see Schneider98 for more
formal definition)
– Cannot depend on future: prefix means
once it is false, it is always false
– Cannot depend on other possible
executions
27 July 2016
University of Virginia CS 551
17
SASI Enforcement
• In theory, SASI can enforce all safety
policies.
• In practice, it is not possible to observe
entire state without modifying it.
• In practice, it is impractical to have a
FSM between every instruction.
27 July 2016
University of Virginia CS 551
18
Naccio
• [Evans99]
• Most reasonable
safety policies don’t
depend on every
instruction
• Most securitycritical tasks involve
calls to system API
27 July 2016
University of Virginia CS 551
Untrusted
Program
Safe
Program
19
Problem
User’s View System View
Program
Policy
Platform Interface
WriteFile (fHandle, …)
tar cf *
System Library
OS Kernel
Resources
Files
27 July 2016
University of Virginia CS 551
Disk
20
Safety Policy Definition
• Resource descriptions: abstract
operational descriptions of resources
(files, network, threads, display, …)
• Platform interface: mapping between
system events (e.g., Java API calls,
Win32 API calls) and abstract resources
• Resource use policy: constraints on
manipulating those resources
27 July 2016
University of Virginia CS 551
21
Resource Description
global resource RFileSystem
openRead (file: RFile)
Called before file is opened for reading
openWrite (file: RFile)
Called before existing file is opened for writing
write (file: RFile, nbytes: int)
Called before nbytes are written to file
… // other operations for observing properties of files, deleting, etc.
resource RFile
RFile (pathname: String)
Constructs object corresponding to pathname
27 July 2016
University of Virginia CS 551
22
Platform Interface (Win32)
wrapper BOOL
WriteFile (HANDLE file, char *buffer, long nBytes,…)
{
// Get corresponding RFile object
RFile rf = lookupRFileByHandle (file);
if (rf != NULL)
RFileSystem.write (rf, nBytes);
%%% // original method call
}
27 July 2016
University of Virginia CS 551
23
Resource Use Policy
policy LimitWrite
LimitBytesWritten (1000000), NoOverwrite
property LimitBytesWritten (n: int)
requires TrackBytesWritten;
check RFileSystem.write (file: RFile, nbytes: int)
if (bytes_written > n) violation (“Writing more than …”);
stateblock TrackBytesWritten
addfield RFileSystem.bytes_written: int = 0;
precode RFileSystem.write (file: RFile, nbytes: int)
bytes_written += nbytes;
27 July 2016
University of Virginia CS 551
24
Naccio Architecture
Per policy
Per application
Safety policy definition
Policy
compiler
Policy-enforcing
system library
Program
Policy description file
Application
transformer
Version of program that:
• Uses policy-enforcing system library
• Satisfies low-level code safety
Current Platforms:
JavaVM – program is collection of Java classes
Win32 – program is Win32 executable and DLLs
27 July 2016
University of Virginia CS 551
25
Policy Compiler
Resource use policy
Resource descriptions
Platform
independent
analyses
Platform interface
Describes Win32 API
System library
System DLLs
(kernel32.dll, user32.dll)
Platform dependent
analyses and code
generation
Policy
description file
Policy-enforcing system library
• Implementations of resource operations
– Perform checking described by resource use policy
• Wrapper DLLs replace and call system DLLs
– Call abstract resource operations as directed by platform interface
27 July 2016
University of Virginia CS 551
26
Program
Win32 Executable and DLLs
Policy description file
Application
Transformer
Version of program that:
1. Uses policy-enforcing library
• Replace DLL names in import table
• Wrapper for LoadLibrary
2. Satisfies low-level code safety
27 July 2016
University of Virginia CS 551
27
Low-Level Code Safety
• Need to make sure programs cannot:
– Circumvent checking code
• Manipulate a protected resource without
using modified library routine
– Jump to address after checking is done
– Load unprotected version of library
– Tamper with checking code or state
– Keep executing after a violation is
detected
27 July 2016
University of Virginia CS 551
28
Low-Level Code Safety
• Naccio/JavaVM
– Bytecode verifier provides type and memory
safety
– Wrappers on class loader, reflection to prevent
circumvention
• Naccio/Win32 [Twyman99]
– Scan for kernel traps
– SFI to prevent jumps around wrappers (hard to
implement)
– Read-only pages and wrappers to protect memory
(single threaded only)
27 July 2016
University of Virginia CS 551
29
Summary
• Most interesting policies are safety
policies  can be enforced by a
reference monitor
• Defusing malcode is an active research
area
– Enforcement is pretty easy
– Policy is the hard part (but not many
people work on it)
• Next time: Proof-Carrying Code, JDK
Security
27 July 2016
University of Virginia CS 551
30
Charge: Why are there no new reading
assignments?
A. I read the CD article that it was cruel for
professors to expect students to read over
“reading holiday” and am mending my evil ways.
B. The office staff revolted after I asked them to copy
both sides of the pages.
C. I want to give you enough time to read the Green
and Libertarian party platforms before Tuesday.
D. I want to give you enough time to solve the graph
isomorphism problems you got trick-or-treating.
E. So you can have time to work on your projects.
27 July 2016
University of Virginia CS 551
31