(Duke suicide picture by Gary McGraw.)
CS201j: Engineering Software
University of Virginia
Computer Science
David Evans http://www.cs.virginia.edu/evans

• Low-level Code Safety
– Java Byte Code Verifier
• Policy Code Safety
– Reference Monitors
– Java’s Security Manager
14 November 2002 CS 201J Fall 2002 2

14 November 2002 CS 201J Fall 2002 3


“A simple, object-oriented, distributed, interpreted, robust, secure , architecture neutral, portable, high-performance, multithreaded, and dynamic language.”
[Sun95]
14 November 2002 CS 201J Fall 2002 4

1. Language is designed so it cannot express certain computations considered insecure.
A few attempt to do this: PLAN, packet filters
2. Language is designed so that
(accidental) program bugs are likely to be caught by the compiler or runtime environment instead of leading to security vulnerabilities.
14 November 2002 CS 201J Fall 2002 5

Safe
• Type Safety
– Compiler and run-time environment ensure that bits are treated as the type they represent
• Memory Safety
– Compiler and run-time environment ensure that program cannot access memory outside defined storage
• Control Flow Safety
– Can’t jump to arbitrary addresses
Which of these does C/C++ have?
Is Java the first language to have them?
No way! LISP had them all in 1960.
14 November 2002 CS 201J Fall 2002 6


• Type Safety
– Most types checked statically
– Coercions, array assignments type checked at run time
• Memory Safety
– No direct memory access (e.g., pointers)
– Primitive array type with mandatory run-time bounds checking
• Control Flow Safety
– Structured control flow, no arbitrary jumps
14 November 2002 CS 201J Fall 2002 7

Can a safe programming language protect you from malcode?
1. Code your servers in it to protect from buffer overflow bugs
2. Only allow programs from untrustworthy origins to run if the are programmed in the safe language
14 November 2002 CS 201J Fall 2002 8

• But how can you tell program was written in the safe language?
– Get the source code and compile it (most vendors, and all malicious attackers refuse to provide source code)
– Special compilation service cryptographically signs object files generated from the safe language (SPIN,
[Bershad96])
– Verify object files preserve safety properties of source language (Java)
14 November 2002 CS 201J Fall 2002 9

malcode.java
Java 
Source
Code javac
Compiler malcode.class
JVML
Object
Code
JavaVM
Joe User
14 November 2002
Joe wants to know JVML code satisfies
Java  ’s safety properties.
CS 201J Fall 2002 10


iconst_2 push integer constant 2 on stack istore_0 store top of stack in variable 0 as int aload_0 load object reference from variable 0 arraylength replace array on top of stack with its length
No! This code violates Java
 ’s type rules.
14 November 2002 CS 201J Fall 2002 11

• Type Safety iconst_3 iconst_3 fadd astore_1 iload_1
• Memory Safety iconst_3 iadd
• Control Flow Safety ifeq 2359
14 November 2002 CS 201J Fall 2002 12

malcode.class
JVML
Object
Code
Trusted Computing Base
Java Bytecode Verifier
“Okay”
Invalid
STOP
JavaVM
Joe User
14 November 2002 CS 201J Fall 2002 13

• Checks class file is formatted correctly
– Magic number: class file starts with
0xCAFEBABE
– String table, code, methods, etc.
• Checks JVML code satisfies safety properties
14 November 2002 CS 201J Fall 2002 14

• Type safe
– Stack and variable slots must store and load as same type
• Memory safe
– Must not attempt to pop more values from stack than are on it
– Array bounds?
Can’t check this in bytecode verifier, VM must
• Control flow safe check array bounds at run time.
– Jumps must be to valid addresses within function, or call/return
14 November 2002 CS 201J Fall 2002 15

• ~700 rules to enforce, JVML specification is (not all clearly specified)
• Emin Gün Sirer found > 100 bugs in commercial bytecode verifiers (using automatic test generation)
– At least 15 of them were security vulnerabilities
• JVML includes jsr instruction (jump to subroutine), can be called with different types in variables and on stack
Added to allow implementations to reuse code in finally blocks to save code size. Would have saved more space to keep original language name (“Oak”).
14 November 2002 CS 201J Fall 2002 16

malcode.java
javac
Compiler
Trusted Computing Base malcode.class
JVML
Java Bytecode Verifier
Invalid
“Okay”
STOP
JavaVM
Joe User
14 November 2002 CS 201J Fall 2002 17

• Virtual machine – interpreter for JVML programs
• Has complete access to host machine
• Bytecode verifier ensures some safety properties, JavaVM must ensure rest:
– Type safety of run-time casts, array assignments
– Memory safety: array bounds checking
– Resource use policy
14 November 2002 CS 201J Fall 2002 18

14 November 2002 CS 201J Fall 2002 19

Monitor
Network
Disk
14 November 2002
Program
Speakers
Memory
CS 201J Fall 2002
SuperSoaker 2000
20

Monitor
Reference Monitor
Program
Speakers
Network
Disk
14 November 2002
Memory
CS 201J Fall 2002
SuperSoaker 2000
21

1. Sees everything a program is about to do before it does it
2. Can instantly and completely stop program execution (or prevent action)
3. Has no other effect on the program or system
Can we build this?
Probably not unless we can build a time machine...
14 November 2002 CS 201J Fall 2002 22

most things
1. Sees everything a program is about to do before it does it
2. Can instantly and completely stop program execution (or prevent action) limited
3. Has no other effect on the program or system
14 November 2002 CS 201J Fall 2002 23

• Provide reference monitors for most security-critical resources
– When a program opens a file in Unix or
Windows, the OS checks that the principal running the program can open that file
• Doesn’t allow different policies for different programs
• No flexibility over what is monitored
– OS decides for everyone
– Hence, can’t monitor inexpensive operations
14 November 2002 CS 201J Fall 2002 24

[Schneider99]
All other instructions
0
Fire
All other instructions
Aim
1
Aim
Aim
Fire
2
All other instructions
STOP
Policy Violation
CS 201J Fall 2002 14 November 2002 25

• Assume:
– Security Automaton can see entire state of world, everything about instruction about to execute
– Security Automaton has unlimited memory, can do unlimited computation
• Are there interesting policies that still can’t be enforced?
14 November 2002 CS 201J Fall 2002 26

• What’s a program?
– A set of possible executions
• What’s an execution?
– A sequence of states
• What’s a security policy?
– A predicate on a set of executions
14 November 2002 CS 201J Fall 2002 27

• Some policies depend on:
– Knowing about the future
• If the program charges the credit card, it must eventually ship the goods
– Knowing about all possible executions
• Information flow – can’t tell if a program reveals secret information without knowing about other possible executions
• Reference Monitors can only know about past of this particular execution
14 November 2002 CS 201J Fall 2002 28

• Reference monitors can only enforce safety policies
• Safety policy is a predicate on a prefix of states (see Schneider98 for more formal definition)
– Cannot depend on future: prefix means once it is false, it is always false
– Cannot depend on other possible executions
14 November 2002 CS 201J Fall 2002 29

[JDK 1.0 – JDK 1.1]
From java.io.File: public boolean delete() {
SecurityManager security =
System.getSecurityManager(); if (security != null) { security.checkDelete(path);
} if (isDirectory()) return rmdir0(); else return delete0();
}
14 November 2002 CS 201J Fall 2002 30

/**
Throws a SecurityException if the calling thread is not allowed to delete the specified file.
This method is invoked for the current security manager by the delete method of class File.
*/ (Some other comments deleted.) public void checkDelete(String file) { throw new SecurityException();
}
14 November 2002 CS 201J Fall 2002 31

• Reference monitor
– How well does it satisfy the requirements?
• Complete mediation
• Can stop execution/prevent action
• Limited effect on execution until policy violation
• User/host application creates a subclass of SecurityManager to define a policy
14 November 2002 CS 201J Fall 2002 32

[JDK 1.0 – JDK 1.1]
From java.io.File: public boolean delete() {
SecurityManager security =
System.getSecurityManager(); if (security != null) { security.checkDelete(path);
} if (isDirectory()) return rmdir0(); else return delete0();
}
What could go seriously wrong with this?!
14 November 2002 CS 201J Fall 2002 33

public class AppletSecurity extends SecurityManager {
...
public synchronized void checkDelete(String file) { checkWrite(file);
}
...
}
14 November 2002 CS 201J Fall 2002 34

(some exception handling code removed) public synchronized void checkWrite(String file) { if (inApplet()) { if (!initACL) initializeACLs();
String realPath =
(new File(file)).getCanonicalPath(); for (int i = writeACL.length ; i-- > 0 ;) { if (realPath.startsWith(writeACL[i])) return;
}
} throw new AppletSecurityException
("checkwrite", file, realPath);
}
Note: no checking if not inApplet!
Very important this does the right thing.
14 November 2002 CS 201J Fall 2002 35

boolean inApplet() { return inClassLoader();
}
Inherited from java.lang.SecurityManager: protected boolean inClassLoader() { return currentClassLoader() != null;
}
14 November 2002 CS 201J Fall 2002 36

/**
Returns an object describing the most recent class loader executing on the stack.
*/
Returns the class loader of the most recent occurrence on the stack of a method from a class defined using a class loader; returns null if there is no occurrence on the stack of a method from a class defined using a class loader.
protected native ClassLoader currentClassLoader();
14 November 2002 CS 201J Fall 2002 37

• java.io.File.delete
calls
SecurityManager.checkDelete
before deleting
• HotJava overrides
SecurityManager with
AppletSecurity to set policy
• AppletSecurity.checkDelete
AppletSecurity.checkWrite
calls
• AppletSecurity.checkWrite
checks if any method on stack has a ClassLoader
• If not no checks; if it does, checks ACL list
14 November 2002 CS 201J Fall 2002 38

• When JavaVM loads a class from the
CLASSPATH, it has no associated
ClassLoader (can do anything)
• When JavaVM loads a class from elsewhere (e.g., the web), it has an associated ClassLoader
14 November 2002 CS 201J Fall 2002 39

• JDK 1.1: Signed classes from elsewhere and have no associated
ClassLoader
• JDK 1.2:
– Different classes can have different policies based on ClassLoader
– Explict enable/disable/check privileges
– SecurityManager is now AccessController
14 November 2002 CS 201J Fall 2002 40

• Java API doesn’t call right SecurityManager checks (63 calls in java.*)
– Font loading bug, synchronization
• ClassLoader is tricked into loading external class as internal
• Bug in Bytecode Verifier can be exploited to circumvent SecurityManager
• Policy is too weak and allows damaging behavior
14 November 2002 CS 201J Fall 2002 41

• See http://java.sun.com/sfaq/chronology.html
(about 1 new vulnerability/month)
• Easy to write “annoying” applets (policy is too imprecise; no way to constrain many resource operations)
• After your PS6 phylogeny programs finish on all the ITC machines, try http://www.cigital.com/hostile-applets/index.html
14 November 2002 CS 201J Fall 2002 42

• No classes next week
• Exams due before 4:30pm Tuesday, drop off to Brenda Perkins
14 November 2002 CS 201J Fall 2002 43