Lecture 18:
Malcode
Countermeasures
CS588: Security and Privacy
University of Virginia
Computer Science
David Evans
http://www.cs.virginia.edu/~evans
Menu
• Reference Monitors
• Java Security
• Proof-Carrying Code (time permitting)
• Monday: Firewalls, Intrusion Detection
Movie – make sure I stop by 3:00!
19 Nov 2001
University of Virginia CS 588
2
Malcode Defenses
• Constrain program behavior
– Reference Monitors
• In-line Reference Monitors
• Prevent possibly harmful code from
running
– Safe Languages
– Proof-Carrying Code
19 Nov 2001
University of Virginia CS 588
3
Program Execution
Monitor
Program
Speakers
Network
Disk
19 Nov 2001
Memory
University of Virginia CS 588
SuperSoaker 2000
4
Program Execution
Reference Monitor
Monitor
Program
Speakers
Network
Disk
19 Nov 2001
Memory
University of Virginia CS 588
SuperSoaker 2000
5
Ideal Reference Monitor
1. Sees everything a program is about to do
before it does it
2. Can instantly and completely stop
program execution (or prevent action)
3. Has no other effect on the program or
system
Can we build this?
Probably not unless we can build a time machine...
19 Nov 2001
University of Virginia CS 588
6
Real
Ideal Reference Monitor
most things
1. Sees everything a program is about to do
before it does it
2. Can instantly and completely stop
program execution (or prevent action)
limited
3. Has no other effect on the program or
system
19 Nov 2001
University of Virginia CS 588
7
Operating Systems
• Provide reference monitors for most
security-critical resources
– When a program opens a file in Unix or
Windows, the OS checks that the principal
running the program can open that file
• Doesn’t allow different policies for different
programs
• No flexibility over what is monitored
– OS decides for everyone
– Hence, can’t monitor inexpensive operations
19 Nov 2001
University of Virginia CS 588
8
Reference Monitor as Finite
State Automaton [Schneider99]
All other
instructions
0
All other
instructions
Aim
Aim
1
Aim
Fire
2
All other
instructions
Fire
STOP
Policy Violation
19 Nov 2001
University of Virginia CS 588
9
What policies can be
enforced?
• Assume:
– Security Automaton can see entire state of
world, everything about instruction about to
execute
– Security Automaton has unlimited memory,
can do unlimited computation
• Are there interesting policies that still
can’t be enforced?
19 Nov 2001
University of Virginia CS 588
10
What’s a Security Policy?
• What’s a program?
– A set of possible executions
• What’s an execution?
– A sequence of states
• What’s a security policy?
– A predicate on a set of executions
19 Nov 2001
University of Virginia CS 588
11
More Formally...
•  : set of all possible executions (can
be infinite)
• S: set of executions possible by target
program S
• P: security policy
set of executions  Boolean
S is safe iff P (S ) is true.
19 Nov 2001
University of Virginia CS 588
12
Reference Monitors cannot
enforce all Security Policies
• Some policies depend on:
– Knowing about the future
• If the program charges the credit card, it must
eventually ship the goods
– Knowing about all possible executions
• Information flow – can’t tell if a program reveals
secret information without knowing about other
possible executions
• Reference Monitors can only know
about past of this particular execution
19 Nov 2001
University of Virginia CS 588
13
Safety Policies
• Reference monitors can only enforce
safety policies
• Safety policy is a predicate on a prefix
of states (see Schneider98 for more
formal definition)
– Cannot depend on future: prefix means
once it is false, it is always false
– Cannot depend on other possible
executions
19 Nov 2001
University of Virginia CS 588
14
Java Security
Real or Decaf?
19 Nov 2001
University of Virginia CS 588
15
What is Java?
A.
B.
C.
D.
E.
Island in Indonesia
A Programming Language (Java)
A Portable Low-Level Language (JVML)
A Platform (JavaVM)
A (semi-)successful marketing strategy
– JavaScript is not related to Java or Java
F. Work on your projects
G. All of the above
19 Nov 2001
University of Virginia CS 588
16
Java : Programming Language
“A simple, object-oriented,
distributed, interpreted, robust,
secure, architecture neutral,
portable, high-performance,
multithreaded, and dynamic
language.”
[Sun95]
19 Nov 2001
University of Virginia CS 588
17
What is a secure language?
1. Language is designed so it cannot
express certain computations
considered insecure.
A few attempt to do this: PLAN, packet filters
2. Language is designed so that
(accidental) program bugs are likely
to be caught by the compiler or runtime environment instead of leading
to security vulnerabilities.
19 Nov 2001
University of Virginia CS 588
18
Safe Programming Languages
• Type Safety
– Compiler and run-time environment ensure that
bits are treated as the type they represent
• Memory Safety
– Compiler and run-time environment ensure that
program cannot access memory outside defined
storage
• Control Flow Safety
– Can’t jump to arbitrary addresses
Which of these does C++ have?
Not a new idea: LISP had these in 1960!
19 Nov 2001
University of Virginia CS 588
19
Java Safety
• Type Safety
– Most types checked statically
– Coercions, array assignments type checked at run
time
• Memory Safety
– No direct memory access (e.g., pointers)
– Primitive array type with mandatory run-time
bounds checking
• Control Flow Safety
– Structured control flow, no arbitrary jumps
19 Nov 2001
University of Virginia CS 588
20
Malicious Code
Can a safe programming language
protect you from malcode?
1. Code your servers in it to protect from
buffer overflow bugs
2. Only allow programs from untrustworthy
origins to run if the are programmed in
the safe language
19 Nov 2001
University of Virginia CS 588
21
Safe Languages?
• But how can you tell program was
written in the safe language?
– Get the source code and compile it (most
vendors, and all malicious attackers refuse
to provide source code)
– Special compilation service signs object
files generated from the safe language
(SPIN, [Bershad96])
– Verify object files preserve safety
properties of source language (Java)
19 Nov 2001
University of Virginia CS 588
22
JVML
malcode.java

Java
Source
Code
javac
Compiler
malcode.class
JVML
Object
Code
JavaVM
Joe User
19 Nov 2001
Joe wants to know JVML code satisfies
Java’s safety properties.
University of Virginia CS 588
23
Does JVML satisfy Java’s
safety properties?
iconst_2
istore_0
aload_0
arraylength
push integer constant 2 on stack
store top of stack in variable 0 as int
load object reference from variable 0
replace array on top of stack with its
length
No! This code violates Java’s type rules.
19 Nov 2001
University of Virginia CS 588
24
malcode.class
JVML
Object
Code
Bytecode Verifier
Trusted Computing Base
Java Bytecode Verifier
Invalid
“Okay”
STOP
JavaVM
Joe User
19 Nov 2001
University of Virginia CS 588
25
Bytecode Verifier
• Checks JVML code satisfies Java’s
safety properties
• Type safe – stack and variable slots
must store and load as same type
• Memory safe (guaranteed by instruction
set)
• Control flow safe: jumps must be within
function, or call/return
19 Nov 2001
University of Virginia CS 588
26
Are Java Bytecode Verifiers
Complicated?
• ~700 rules to enforce, JVML specification is
(not all clearly specified)
• Emin Gün Sirer found > 100 bugs in
commercial bytecode verifiers (using
automatic test generation)
– At least 15 of them were security vulnerabilities
• JVML includes jsr instruction (jump to
subroutine), can be called with different types
in variables and on stack
19 Nov 2001
University of Virginia CS 588
27
Java
malcode.java
javac
Compiler
malcode.class
JVML
Trusted Computing Base
Java Bytecode Verifier
Invalid
“Okay”
STOP
JavaVM
Joe User
19 Nov 2001
University of Virginia CS 588
28
JavaVM
• Virtual machine – interpreter for JVML
programs
• Has complete access to host machine
• Bytecode verifier ensures some safety
properties, JavaVM must ensure rest:
– Type safety of run-time casts, array
assignments
– Memory safety: array bounds checking
– Resource use policy
19 Nov 2001
University of Virginia CS 588
29
JavaVM Policy Enforcment
[JDK 1.0 – JDK 1.1]
From java.io.File:
public boolean delete() {
SecurityManager security =
System.getSecurityManager();
if (security != null) {
security.checkDelete(path);
}
if (isDirectory()) return rmdir0();
else return delete0();
}
19 Nov 2001
University of Virginia CS 588
30
java.lang.SecurityManager
/**
Throws a SecurityException if the calling
thread is not allowed to delete the specified
file.
This method is invoked for the current security
manager by the delete method of class File.
*/
(Some other comments deleted.)
public void checkDelete(String file) {
throw new SecurityException();
}
19 Nov 2001
University of Virginia CS 588
31
Security Manager
• Reference monitor
– How well does it satisfy the requirements?
• Complete mediation
• Can stop execution/prevent action
• Limited effect on execution until policy violation
• User/host application creates a
subclass of SecurityManager to define a
policy
19 Nov 2001
University of Virginia CS 588
32
HotJava’s Policy (JDK 1.1.7)
public class AppletSecurity
extends SecurityManager {
...
public synchronized void
checkDelete(String file) {
checkWrite(file);
}
...
}
19 Nov 2001
University of Virginia CS 588
33
AppletSecurity.checkWrite
(some exception handling code removed)
public synchronized void checkWrite(String file) {
if (inApplet()) {
if (!initACL) initializeACLs();
String realPath =
(new File(file)).getCanonicalPath();
for (int i = writeACL.length ; i-- > 0 ;) {
if (realPath.startsWith(writeACL[i])) return;
}
throw new AppletSecurityException
("checkwrite", file, realPath);
}
}
Note: no checking if not inApplet!
Very important this does the right thing.
19 Nov 2001
University of Virginia CS 588
34
inApplet
boolean inApplet() {
return inClassLoader();
}
Inherited from java.lang.SecurityManager:
protected boolean inClassLoader() {
return
currentClassLoader() != null;
}
19 Nov 2001
University of Virginia CS 588
35
currentClassLoader
/**
Returns an object describing the most recent class
loader executing on the stack.
Returns the class loader of the most recent occurrence
on the stack of a method from a class defined using a
class loader; returns null if there is no occurrence on the
stack of a method from a class defined using a class
loader.
*/
protected native ClassLoader currentClassLoader();
19 Nov 2001
University of Virginia CS 588
36
Recap
• java.io.File.delete
calls
before deleting
• HotJava overrides SecurityManager with
AppletSecurity to set policy
• AppletSecurity.checkDelete calls
SecurityManager.checkDelete
AppletSecurity.checkWrite
• AppletSecurity.checkWrite checks if
any method on stack has a ClassLoader
• If not no checks; if it does, checks ACL list
19 Nov 2001
University of Virginia CS 588
37
JDK 1.0 Trust Model
• When JavaVM loads a class from the
CLASSPATH, it has no associated
ClassLoader (can do anything)
• When JavaVM loads a class from
elsewhere (e.g., the web), it has an
associated ClassLoader
19 Nov 2001
University of Virginia CS 588
38
JDK Evolution
• JDK 1.1: Signed classes from
elsewhere and have no associated
ClassLoader
• JDK 1.2:
– Different classes can have different policies
based on ClassLoader
– Explict enable/disable/check privileges
– SecurityManager is now AccessController
19 Nov 2001
University of Virginia CS 588
39
What can go wrong?
• Java API doesn’t call right SecurityManager
checks (63 calls in java.*)
– Font loading bug, synchronization
• ClassLoader is tricked into loading external
class as internal
• Bug in Bytecode Verifier can be exploited to
circumvent SecurityManager
• Policy is too weak and allows damaging
behavior
19 Nov 2001
University of Virginia CS 588
40
Hostile Applets
• See http://java.sun.com/sfaq/chronology.html
(about 1 new vulnerability/month)
• Easy to write “annoying” applets (policy
is too imprecise; no way to constrain
many resource operations)
• http://www.cigital.com/hostileapplets/index.html
19 Nov 2001
University of Virginia CS 588
41
Proof-Carrying Code
19 Nov 2001
University of Virginia CS 588
42
Maze
(From http://www.cedillasys.com/pages/tech/technotes/)
19 Nov 2001
University of Virginia CS 588
43
Proof-Carrying Maze
(From http://www.cedillasys.com/pages/tech/technotes/)
19 Nov 2001
University of Virginia CS 588
44
Proof-Carrying Code
• Guarantee properties of untrustworthy
code by checking a proof provided by
code producer
• Creating a proof is hard
– Have to make up invariants, choose cases,
pick strategies, etc.
• Checking a proof is easy
– Simple mechanical application of rules
19 Nov 2001
University of Virginia CS 588
45
Have we seen anything like this?
• Java Bytecode Verifier is a simple
instance of PCC:
– Bytecodes include extra information on
typing, stack use, etc.
– Bytecode verifier checks it to enforce lowlevel code safety properties
19 Nov 2001
University of Virginia CS 588
46
Fortune Cookie
“That which must
can be proved cannot be
worth much.”
Fortune cookie quoted on Peter’s web page
• True for all users
• True for all executions
• Exception: Low-level code safety
19 Nov 2001
University of Virginia CS 588
47
Reasons you might prefer PCC
• Run-time performance?
– Amortizes additional download and verification
time only rarely
– SFI Performance penalty: ~5%
• If you care, pay $20 more for a better processor or wait 5
weeks
• Smaller Trusted Computing Base?
– Not really smaller: twice as big as SFI (Touchstone
VCGen+checker – 8300 lines / MisFiT x86 SFI
implementation – 4500 lines)
• You are a vendor who cares more about
quality than time to market
– Not really PCC (not across a trust boundary)
19 Nov 2001
University of Virginia CS 588
48
Feedback Slips
19 Nov 2001
University of Virginia CS 588
49
You are getting a strip of paper with your
recorded grades so far (check they are
correct), and my recommendation for
PS5/project:
You’re doing fine (but keep working on your
project). Enjoy your Thanksgiving.
You’re doing okay, but go easy on the
Cranberry Sauce. You should either do PS5
or work a lot on your project.
You need to get cracking to avoid a mediocre
grade (or worse) in this course. You should do
PS5 and work on your project.
No one got this. Still plenty of time to do well
in the course.
19 Nov 2001
University of Virginia CS 588
50
Movie
19 Nov 2001
University of Virginia CS 588
51