Lecture 21:
Countering
Malicious Code
CS588: Cryptography
University of Virginia
Computer Science
David Evans
http://www.cs.virginia.edu/evans
Menu
• Reference Monitors
• Java/.NET Security
23 April 2005
University of Virginia CS 588
2
Malcode Defenses
• Constrain program behavior
– Reference Monitors
• In-line Reference Monitors
• Prevent possibly harmful code from
running
– Safe Languages
– Proof-Carrying Code
23 April 2005
University of Virginia CS 588
3
Program Execution
Monitor
Program
Speakers
Network
Disk
23 April 2005
Memory
University of Virginia CS 588
SuperSoaker 2000
4
Program Execution
Reference Monitor
Monitor
Program
Speakers
Network
Disk
23 April 2005
Memory
University of Virginia CS 588
SuperSoaker 2000
5
Ideal Reference Monitor
1. Sees everything a program is about to do
before it does it
2. Can instantly and completely stop
program execution (or prevent action)
3. Has no other effect on the program or
system
Can we build this?
Probably not unless we can build a time machine...
23 April 2005
University of Virginia CS 588
6
Real
Ideal Reference Monitor
most things
1. Sees everything a program is about to do
before it does it
2. Can instantly and completely stop
program execution (or prevent action)
limited
3. Has no other effect on the program or
system
23 April 2005
University of Virginia CS 588
7
Operating Systems
• Provide reference monitors for most
security-critical resources
– When a program opens a file in Unix or
Windows, the OS checks that the principal
running the program can open that file
• Doesn’t allow different policies for different
programs
• No flexibility over what is monitored
– OS decides for everyone
– Hence, can’t monitor inexpensive operations
23 April 2005
University of Virginia CS 588
8
Reference Monitor as Finite
State Automaton [Schneider99]
All other
instructions
0
All other
instructions
Aim
Aim
1
Aim
Fire
2
All other
instructions
Fire
STOP
Policy Violation
23 April 2005
University of Virginia CS 588
9
What policies can be
enforced?
• Assume:
– Security Automaton can see entire state of
world, everything about instruction about to
execute
– Security Automaton has unlimited memory,
can do unlimited computation
• Are there interesting policies that still
can’t be enforced?
23 April 2005
University of Virginia CS 588
10
What’s a Security Policy?
• What’s a program?
– A set of possible executions
• What’s an execution?
– A sequence of states
• What’s a security policy?
– A predicate on a set of executions
23 April 2005
University of Virginia CS 588
11
More Formally...
•  : set of all possible executions (can
be infinite)
• S: set of executions possible by target
program S
• P: security policy
set of executions  Boolean
S is safe iff P (S ) is true.
23 April 2005
University of Virginia CS 588
12
Reference Monitors cannot
enforce all Security Policies
• Some policies depend on:
– Knowing about the future
• If the program charges the credit card, it must
eventually ship the goods
– Knowing about all possible executions
• Information flow – can’t tell if a program reveals
secret information without knowing about other
possible executions
• Reference Monitors can only know
about past of this particular execution
23 April 2005
University of Virginia CS 588
13
Safety Policies
• Reference monitors can only enforce
safety policies
• Safety policy is a predicate on a prefix
of states (see Schneider98 for more
formal definition)
– Cannot depend on future: prefix means
once it is false, it is always false
– Cannot depend on other possible
executions
23 April 2005
University of Virginia CS 588
14
Java Security
Real or Decaf?
23 April 2005
University of Virginia CS 588
15
What is Java?
A.
B.
C.
D.
E.
Island in Indonesia
A Programming Language (Java)
A Portable Low-Level Language (JVML)
A Platform (JavaVM)
A (semi-)successful marketing strategy
– JavaScript is not related to Java or Java
F. Work on your projects
G. All of the above
23 April 2005
University of Virginia CS 588
16
Java : Programming Language
“A simple, object-oriented,
distributed, interpreted, robust,
secure, architecture neutral,
portable, high-performance,
multithreaded, and dynamic
language.”
[Sun95]
23 April 2005
University of Virginia CS 588
17
What is a secure language?
1. Language is designed so it cannot
express certain computations
considered insecure.
A few attempt to do this: PLAN, packet filters
2. Language is designed so that
(accidental) program bugs are likely
to be caught by the compiler or runtime environment instead of leading
to security vulnerabilities.
23 April 2005
University of Virginia CS 588
18
Safe Programming Languages
• Type Safety
– Compiler and run-time environment ensure that
bits are treated as the type they represent
• Memory Safety
– Compiler and run-time environment ensure that
program cannot access memory outside defined
storage
• Control Flow Safety
– Can’t jump to arbitrary addresses
Which of these does C++ have?
Not a new idea: LISP had these in 1960!
23 April 2005
University of Virginia CS 588
19
Java Safety
• Type Safety
– Most types checked statically
– Coercions, array assignments type checked at run
time
• Memory Safety
– No direct memory access (e.g., pointers)
– Primitive array type with mandatory run-time
bounds checking
• Control Flow Safety
– Structured control flow, no arbitrary jumps
23 April 2005
University of Virginia CS 588
20
Malicious Code
Can a safe programming language
protect you from malcode?
1. Code your servers in it to protect from
buffer overflow bugs
2. Only allow programs from untrustworthy
origins to run if the are programmed in
the safe language
23 April 2005
University of Virginia CS 588
21
Safe Languages?
• But how can you tell program was
written in the safe language?
– Get the source code and compile it (most
vendors, and all malicious attackers refuse
to provide source code)
– Special compilation service signs object
files generated from the safe language
(SPIN, [Bershad96])
– Verify object files preserve safety
properties of source language (Java)
23 April 2005
University of Virginia CS 588
22
JVML
malcode.java

Java
Source
Code
javac
Compiler
malcode.class
JVML
Object
Code
JavaVM
Joe User
23 April 2005
Joe wants to know JVML code satisfies
Java’s safety properties.
University of Virginia CS 588
23
Does JVML satisfy Java’s
safety properties?
iconst_2
istore_0
aload_0
arraylength
push integer constant 2 on stack
store top of stack in variable 0 as int
load object reference from variable 0
replace array on top of stack with its
length
No! This code violates Java’s type rules.
23 April 2005
University of Virginia CS 588
24
malcode.class
JVML
Object
Code
Bytecode Verifier
Trusted Computing Base
Java Bytecode Verifier
Invalid
“Okay”
STOP
JavaVM
Joe User
23 April 2005
University of Virginia CS 588
25
Bytecode Verifier
• Checks JVML code satisfies Java’s
safety properties
• Type safe – stack and variable slots
must store and load as same type
• Memory safe (guaranteed by instruction
set)
• Control flow safe: jumps must be within
function, or call/return
23 April 2005
University of Virginia CS 588
26
Are Java Bytecode Verifiers
Complicated?
• ~700 rules to enforce, JVML specification is
(not all clearly specified)
• Emin Gün Sirer found > 100 bugs in
commercial bytecode verifiers (using
automatic test generation)
– At least 15 of them were security vulnerabilities
• JVML includes jsr instruction (jump to
subroutine), can be called with different types
in variables and on stack
23 April 2005
University of Virginia CS 588
27
Java
malcode.java
javac
Compiler
malcode.class
JVML
Trusted Computing Base
Java Bytecode Verifier
Invalid
“Okay”
STOP
JavaVM
Joe User
23 April 2005
University of Virginia CS 588
28
JavaVM
• Virtual machine – interpreter for JVML
programs
• Has complete access to host machine
• Bytecode verifier ensures some safety
properties, JavaVM must ensure rest:
– Type safety of run-time casts, array
assignments
– Memory safety: array bounds checking
– Resource use policy
23 April 2005
University of Virginia CS 588
29
JavaVM Policy Enforcment
[JDK 1.0 – JDK 1.1]
From java.io.File:
public boolean delete() {
SecurityManager security =
System.getSecurityManager();
if (security != null) {
security.checkDelete(path);
}
if (isDirectory()) return rmdir0();
else return delete0();
}
23 April 2005
University of Virginia CS 588
30
java.lang.SecurityManager
/**
Throws a SecurityException if the calling
thread is not allowed to delete the specified
file.
This method is invoked for the current security
manager by the delete method of class File.
*/
(Some other comments deleted.)
public void checkDelete(String file) {
throw new SecurityException();
}
23 April 2005
University of Virginia CS 588
31
Security Manager
• Reference monitor
– How well does it satisfy the requirements?
• Complete mediation
• Can stop execution/prevent action
• Limited effect on execution until policy violation
• User/host application creates a
subclass of SecurityManager to define a
policy
23 April 2005
University of Virginia CS 588
32
HotJava’s Policy (JDK 1.1.7)
public class AppletSecurity
extends SecurityManager {
...
public synchronized void
checkDelete(String file) {
checkWrite(file);
}
...
}
23 April 2005
University of Virginia CS 588
33
AppletSecurity.checkWrite
(some exception handling code removed)
public synchronized void checkWrite(String file) {
if (inApplet()) {
if (!initACL) initializeACLs();
String realPath =
(new File(file)).getCanonicalPath();
for (int i = writeACL.length ; i-- > 0 ;) {
if (realPath.startsWith(writeACL[i])) return;
}
throw new AppletSecurityException
("checkwrite", file, realPath);
}
}
Note: no checking if not inApplet!
Very important this does the right thing.
23 April 2005
University of Virginia CS 588
34
JDK 1.0 Trust Model
• When JavaVM loads a class from the
CLASSPATH, it has no associated
ClassLoader (can do anything)
• When JavaVM loads a class from
elsewhere (e.g., the web), it has an
associated ClassLoader
23 April 2005
University of Virginia CS 588
35
JDK Evolution
• JDK 1.1: Signed classes from
elsewhere and have no associated
ClassLoader
• JDK 1.2:
– Different classes can have different policies
based on ClassLoader
– Explict enable/disable/check privileges
– SecurityManager is now AccessController
23 April 2005
University of Virginia CS 588
36
What can go wrong?
• Java API doesn’t call right SecurityManager
checks (63 calls in java.*)
– Font loading bug, synchronization
• ClassLoader is tricked into loading external
class as internal
• Bug in Bytecode Verifier can be exploited to
circumvent SecurityManager
• Policy is too weak and allows damaging
behavior
23 April 2005
University of Virginia CS 588
37
Java
malcode.java
javac
Compiler
malcode.class
JVML
Trusted Computing Base
Java Bytecode Verifier
Invalid
“Okay”
STOP
JavaVM
Joe User
23 April 2005
University of Virginia CS 588
38
Bytecode Verifier
• Checks JVML code satisfies safety
properties
– Simulates program execution to know types
are correct, but doesn’t need to examine any
instruction more than once
– After code is verified, it is trusted: is not
checked for type safety at run time (except
for casts, array stores)
Key assumption: when a value is written to a memory
location, the value in that memory location is the same
value when it is read.
23 April 2005
University of Virginia CS 588
39
Violating the Assumption
…
// The object on top of the stack is a SimObject
astore_0
// There is a SimObject in location 0
aload_0
// The value on top of the stack is a SimObject
If a cosmic ray hits the right bit of memory, between
the store and load, the assumption might be wrong.
23 April 2005
University of Virginia CS 588
40
Improving the Odds
• Set up memory so that a single bit error
is likely to be exploitable
• Mistreat the hardware memory to
increase the odds that bits will flip
• One bit flip allows you to violate type
safety, compromise all security
Sudhakar Govindavajhala and Andrew W. Appel, Using
Memory Errors to Attack a Virtual Machine, July 2003.
23 April 2005
University of Virginia CS 588
41
Getting a Bit Flip
• Wait for a Cosmic Ray
– You have to be really, really patient… (or move
machine out of Earth’s atmosphere)
• X-Rays
– Expensive, not enough power to generate bit-flip
• High energy protons and neutrons
– Work great - but, you need a particle accelerator
• Hmm….
23 April 2005
University of Virginia CS 588
42
Using Heat




50-watt spotlight
bulb
Between 80° 100°C, memory
starts to have a
few failures
Attack applet is
successful (at
least half the
time)!
Hairdryer works
too, but it frys too
many bits at once
Picture from Sudhakar Govindavajhala
23 April 2005
University of Virginia CS 588
43
Should Anyone be Worried?
Java virtual machine
23 April 2005
University of Virginia CS 588
44
Recap
• Verifier assumes the value you write is the
same value when you read it
• By flipping bits, we can violate this
assumption
• By violating this assumption, we can violate
type safety: get two references to the same
storage that have inconsistent types
• By violating type safety, we can get around
all other security measures
23 April 2005
University of Virginia CS 588
45
.NET
JAR
Assembly
ClassLoader
Class
Verify
Exception
Security
exception
Security
exception
Class Loader
Verifier
Java VM
Verify
Exception
Security
exception
JIT Verifier
CLR
Operating System
Operating System
Protected Resource
Protected Resource
Java
23 April 2005
Policy Manager
University of Virginia CS 588
.NET
46
Major Security Vulnerabilities (Cumulative)
50
45
http://www.cs.virginia.edu/evans/pubs/acsac2004.html
40
35
30
25
Java VM
20
15
10
5
0
1996
.NET VM
1997
1998
1999
2000
2001
2002
2003
2004
Course Evaluations
23 April 2005
University of Virginia CS 588
48
CS588 Pledge
I will provide useful feedback. I realize
this is an evolving course and it is important
that I let the course staff know what they
need to improve the course. I will not wait
until the end of the course to make the
course staff aware of any problems. I will
provide feedback either anonymously … or
by contacting the course staff directly. I will
fill out all course evaluation surveys
honestly and thoroughly.
23 April 2005
University of Virginia CS 588
49
Course Evaluations
• Fill out the SEAS Evaluation on-line
– It may not be anonymous, but I promise not to peek
– Respond based on whether you want me to get fired
or promoted
• (optional) Fill out a RateMyProfessors.com
survey
– Provide useful information to future students
• Fill out my course-specific survey (handed out
last day)
– Help improve future versions of the course for later
students
23 April 2005
University of Virginia CS 588
50
Charge
• Links to papers on the web site:
– Hair dryer attacks
– Comparing Java and .NET
• Thursday:
– Quantum cryptography
– Cryptographic hashing attacks (Boyd and
Isabelle)
• Tuesday:
– Project presentations, final out, Sneakers
23 April 2005
University of Virginia CS 588
51