Bridging the gap between
specification and implementation
Insup Lee
Department of Computer and Information Science
University of Pennsylvania
November 9, 2004
11/9/04
1
The gap between specification and
implementation
►
Problem
– gap between an abstract model and the
implementation
– scalability challenge (software size and
complexity)
►
Approaches
–
–
–
–
►
Requirements
Software model checking
Model-based code generation
Test generation from specification
Run-time verification/checking
Model checking
Design
specification
– Formal, Complete
– Does not scale well
– Checks design, not implementation
►
Testing
– Tests an implementation directly
– Informal, Incomplete
11/9/04
Implementation
2
Run-time Verification and its Application to
Security
Run-time verification
► The MaC framework
►
– Java-MaC
– Steering
►
Model-based security checking
– Security automata
– Model carrying code
►
Current work
11/9/04
3
Runtime Verification
Program
Verifier
Execution
Information
Feedback
Check
Sat / Unsat
User
11/9/04
4
Run-time verification
►
►
►
Run-time monitoring and checking w.r.t. formal specification
Ensures the runtime compliance of the current execution of a
system with its formal requirement
Steps
1.
2.
3.
4.
►
Complementary methodology to formal verification and program
testing
–
–
–
►
Specify formal requirements
Extract information from current executing program
Check the execution against formal requirements
Steer the computation to a safe state
Validate implementation
Not complete: guarantee for current execution
Prevention, avoidance, and detection & recovery
Joint work with S. Kannan, M. Kim, U. Sammapun, O. Sokolsky, M.
Viswanathan
11/9/04
5
The MaC Framework
Informal
Requirement
Spec
Program
Input
Human
Monitoring Scripts
Low-level
Specification
Automatic
Instrumentation
Automatic
Translation
Static Phase
Program
low-level
behavior
Filter
Event
Recognizer
high-level
behavior
High-level
Specification
Automatic
Translation
Run-time
Checker
Run-time Phase
11/9/04
[Kim et al, ECRTS 99]
6
Design of the MaC Languages
raiseGate
start(position==100)
end(position==100)
position == 100
1:00:10
►
►
1:00:15
1:00:30
Time
Must be able to reason about both time instants and information that
holds for a duration of time in a program execution.
– Events and conditions are a natural division, which is also found in
other formalisms such as SCR.
– Conditions, which are true or false for a finite duration of time (e.g.,
is variable x >5?)
– Events, which are either present or absent at some instant of time
(e.g., is the control right now at the end of method f?).
Need temporal operators combining events and conditions in order to
reason about traces.
11/9/04
7
Logical Foundation
C :: c |defined (C) | [ E1 , E2 ) | C | C1  C2 | C1  C2
E :: e | start (C ) | end (C ) | E1  E2 | E1  E2 |
E when C
► Conditions interpreted over 3 values: true, false
and undefined.
► [., .) pairs a couple of events to define an interval.
► start and end define the events corresponding
to the instant when conditions change their value.
[e1 ,e2)
e1
[e1 ,e2)
e2
[Lee et al, ICPDP 99]
11/9/04
8
Meta Event Definition Language (MEDL)
►
Expresses requirements using the events
and conditions, sent by event recognizer.
►
Expresses the subset of safety
properties.
►
Describes the safety requirements of a
system, in terms of conditions that must
always be true, and alarms (events) that
must never be raised.
►
–
property safeRRC = IC -> GD;
–
alarm violation = start (!safeRRC);
Auxilliary variables may be used to store
history.
–
endIC-> { num_train_pass’ =
num_train_pass + 1; }
11/9/04
ReqSpec <spec_name>
/* Import section */
import event <e>;
import condition <c>;
/*Auxiliary variable */
var int <aux_v>;
/*Event and condition */
event <e> = ...;
condition <c>= ...;
/*Property and violation */
property <c> = ...;
alarm <e> = ...;
/*Auxiliary variable update*/
<e> -> { <aux_v'> := ... ; }
End
9
The MaC languages
►
►
►
PEDL: abstraction
MEDL: abstract transformation
SADL: feedback
MEDL
Run-time state:
•method call
•object state
•local variables
11/9/04
PEDL
Abstract state:
•events
•conditions
SADL
10
PEDL (Primitive Event Definition Language)
►
Primitive Event Definition Language (PEDL)
– Maps the low-level state information of the system to high-level events.
– Depends on target program implementation
– Design Issues
• What can be observed?
• Passive and active probes
• Software probe vs. hardware probe
• Programming language-based vs. systems API-based
►
Java-PEDL
– Provides primitives to refer to values of variables and to certain points in
the execution of the program.
– PEDL is defined so that events can be recognized in time linear to the size
of the PEDL specification
[Kim et al, FMSD 04]
11/9/04
11
Java-MaC Static Phase
MaC Specifications
Java
Bytecode
PEDL
MEDL
SADL
MaC Compilers
Instrumented
Bytecode
Event
Recognizer
Checker
Steerer
MaC Verifiers
11/9/04
12
Java-MaC Dynamic Phase
Instrumented Java Program
Execution
MaC Verifier
Event
Recognizer
(PEDL)
Information
Feedback
Checker
(MEDL)
Steerer
Sat / Unsat
User
11/9/04
13
MaC Language - PEDL
Railroad Crossing Property: - If train is crossing, then gate must be down
- Train is crossing when position is between 30 and 50
position = 0
Abstraction
- When train position is between 30 and 50
position = 20
- When gate starts/ends being down
position = 40
cross = true
export event startGD, endGD;
export condition cross;
position = 55
// specify what to monitor
monobj Train.position;
monmeth Gate.up();
monmeth Gate.down();
Gate.up()
position = 60
Java Program
11/9/04
// specify abstraction
condition cross = (30 < Train.position)
&& (Train.position < 50);
event startGD = endM(Gate.down());
event endGD = startM(Gate.up());
cross
Gate.down()
startGD
cross = false
endGD
PEDL
14
MaC Language - MEDL
Railroad Crossing Property: - If train is crossing, then gate must be down
- Train is crossing when position is between 30 and 50
Abstraction
- When gate is down
Property
cross = true
- If train is crossing, then gate must be down
endGD
PEDL
11/9/04
// specify abstraction
condition gateDown = [startGD, endGD);
Violation
gateDown
cross = false
import event startGD, endGD;
import conditions cross;
cross
cross
startGD
// specify property
property safeRRC = cross -> gateDown;
MEDL
15
Instrumentation
class Train {
int position;
main() {
position = 0;
position = 20;
position = 40;
position = 55;
} }
+
monobj Train.position;
11/9/04
=
class Train {
int position;
main() {
position = 0;
send(x,0);
position = 20;
send(x,20);
position = 40;
send(x,40);
position = 55;
send(x,55);
} }
Sent to Event Recognizer:
[ (position,0), (position,20),
(position,40), (position,55) ]
16
MaC Language - Components
Railroad Crossing Property: - If train is crossing, then gate must be down
- Train is crossing when position is between 30 and 50
position = 0
Abstraction
Abstraction
position = 20
- When train
position is between
30 and 50
- When gate is
down
position = 40
cross = true
Property
cross = false
- If train is
crossing, then
gate must be
down
endGD
Violation
gateDown
Gate.up()
startGD
cross
position = 55
- When gate
starts/ends being
down
cross
Gate.down()
position = 60
Java Program
11/9/04
PEDL
MEDL
17
Evaluation Overview
►
Static Phase: Each property is represented as a tree
– The most basic events/conditions/aux vars are at the leaf level
• For PEDL, variable updates, start/end method events are leaves
• For MEDL, imported events from PEDL and auxiliary variables are
leaves
– Composition of events and conditions link to events/conditions
that are composed of
– PEDL and MEDL are forests
►
Dynamic Phase: In both Event Recognizer (ER) and Checker,
evaluation starts at the leaves and traverses up to the root
– Evaluation starts only at leaves representing occurred events or
changed conditions
– Otherwise, no evaluation is done
– Efficient
11/9/04
18
PEDL Graph
export event startGD, endGD;
export condition cross;
monobj Train.position;
monmeth Gate.up();
monmeth Gate.down();
condition cross = (30 < Train.position)
&& (Train.position < 50);
event startGD = endM(Gate.down());
event endGD = startM(Gate.up());
11/9/04
cross
startGD
endGD
30 < position < 50
endM(Gate.down())
startM(Gate.up())
Train.position
Gate.down()
Gate.up()
19
MEDL Graph
import event startGD, endGD;
import conditions cross;
condition gateDown = [startGD, endGD);
property safeRRC = cross -> gateDown;
safeRRC
cross -> gateDown
gateDown
[startGD, endGD)
cross
11/9/04
startGD
endGD
20
Algorithm
►
Static Phase:
– Create PEDL and MEDL graphs
– Assign a height to each node in both graphs
►
Dynamic Phase:
– Maintain an evaluation list sorted by height
– Add all occurred primitive events and changed conditions to the evaluation list at
height 0
– For each event/condition in the evaluation list,
• Call evaluate() method
• Add its parent in the evaluation list (if not already in)
• Repeat until the list if empty
– Finishing
• ER sends occurred events/changed condition to checker for each exported
events/condition
• Checker notifies user for each event in the alarm list.
►
Complexity
– The size of the PEDL or MEDL graph is linear in the size of the formula.
– Evaluation of a MEDL or PEDL formula on a single observation is linear in the
size of the graph.
11/9/04
21
Java-MaC Dynamic Phase
Instrumented Java Program
Execution
MaC Verifier
Event
Recognizer
(PEDL)
Information
Feedback
Checker
(MEDL)
Steerer
(SADL)
Sat / Unsat
User
11/9/04
22
Steering process
action
invocation
received
violation
steering
condition
satisfied
action executed
system
event
received
checker
11/9/04
action
invoked
detection
23
Simplex Architecture
Safety
Experimental
us
Decision
Module
u
Physical
System
x
x0
ue
EX
SC
Equilibrium
state
►
Experimental controllers provide improved
performance but uncertain stability properties
– Can be dynamically added or replaced
►
Safety controller has the largest stability region
[L. Sha]
11/9/04
24
Inverted Pendulum (IP) Example
g
l
q
uf
Written in C
m
Experimental
Experimental
Controller
Experimental
Controller
Controller
M
x
Device
Drivers
11/9/04
angle,
track
volts
Safety
Controller
Decision Module
Switching logic
[L. Sha]
25
Steering Action Definition Language
►
SADL script
– identifies object used in steering
// the target of steering is the object dm of type DecisionModule
// located in the class IP
DecisionModule IP:dm;
– defines steering actions
// setSC() method of dm is invoked
steering action change2SC = { call (IP:dm).setSC(); }
– specifies steering location
• locations in the code where the actions can be executed
before read DecisionModule:volts;
11/9/04
26
IP and MaC
Written in
Java
Experimental
Experimental
Controller
Experimental
Controller
Controller
Device
Drivers
11/9/04
angle,
track
volts
J
N
I
Safety
Controller
Decision Module
monitor
steer
MaC:
Switching
logic
[Kim et al, RV 02]
27
Applications
►
Where can we get specifications?
– During the design specification and analysis phase, many properties
may be identified and verified.
• Reuse properties model checked during the design phase
• E.g., more than 1000 properties during designing of flight control systems
– Extract from (informal) requirements and specification documents
– Security Policy
– Extract from the target program
• To ensure that the program has not been tampered
• Model Carrying Code
►
Other application areas
–
–
–
–
►
Network routing simulation
Hardware design
Adaptable sensor network systems
Cheat detection in distributed game
Annual run-time verification workshop (01, 02, 03, 04, …)
11/9/04
28
Monitoring and Checking for
Security Properties
11/9/04
29
Similar techniques, different purposes
►
Check security policy
– Security automata, edit automata
– Model-Carrying Code (MCC)
►
Intrusion detection
– Extract from the target program to ensure that the program has
not been tampered
– Signature-based approach
11/9/04
30
Security Policy in Security/Edit Automata
Example (modified from [BLW02]): Limit the amount of memory that
an application can allocate for itself
Property: application must not allocate memory more than n
a = (malloc(q),
q’=q’+q, q’< n)
a = (malloc(q),
q’=q’+q, q’< n)
a
11/9/04
a
a = (malloc(q),
q’=q’+q, q’ >= n)
halt
31
Must not allocate more than n
PEDL
export event mallocCall;
monmeth int malloc(int);
event mallocCall = startM(malloc(int));
MEDL
import event mallocCall;
import action halt;
var int memory;
alarm violateMemoryPolicy = end(memory < 1000);
mallocCall -> { // value(mallocCall,0) returns arg of malloc()
memory’ = memory + value(mallocCall,0);
}
violateMemoryPolicy -> { invoke(halt); }
SADL
steering action halt = // exit before next malloc() call
{ call System.exit(); } before call malloc(int);
11/9/04
32
Model-Carrying Code (MCC)
►
How can we run untrusted code on our machine?
– Untrusted code comes with a model of its security-relevant
behavior
– Users have their own security policies
– Employ two types of checking
►
Static checking: to ensure that untrusted program’s
model respects user’s security policy
– Use model checking to check that Beh(Model) are in
Beh(Policy)
►
Run-time checking: to ensure that program behaves as
specified by model
– Use runtime checking with
• Model is a specification (Automata)
• Events are system calls
11/9/04
33
MCC Framework
[SVB+03]
11/9/04
34
Current Work on MaC
►
Java-MaC available for download
– www.cis.upenn.edu/~rtg/mac
Minimum trace of run-time verification
► MEDL-RE: MEDL with regular expressions
► MaC with probability
► Hierarchical IDS
► Using MaC to detect failed/malicious sensor
nodes
►
11/9/04
35
Minimal Trace for run-time verification
►
Large amounts of monitored data hampers communication between
the system and the checker
– How to extract minimum information necessary to check a given
property?
►
Redundant information in traces
– alarm dropReq = fault when responding
condition responding = [acceptReq,response)
event acceptReq = request when ready==true
event fault = start(abort == true)
– Trace with redundant data:
request ready(t) abort(t) request abort(f) request abort(t)
– Trace without redundant data:
ready(t)
►
request
abort(t)
Approach: define minimally adequate trace, and MEDL automata
11/9/04
36
MEDL-RE: MaC with regular expression
►
Regular expression over events
– Statement:
RE R {Ē} = < R >,
– Grammar of R: R ::= e | R.R | R+R | R*
– Alphabet of R contains events used in R and events in its relevant set {Ē}
►
Regular expressions are neither events nor conditions and cannot be
used alone
– Events associated with RE R:
startRE(R), success(R), fail(R)
►
Example
– Three components of a media must start in the following order: video,
caption, audio
– RE media {} = < startVideo . startCaption . startAudio >
– alarm notOrdered = fail(media)
►
Challenges
– (Possibly infinite) multiple instances of (possibly overlapping) regular
expressions
11/9/04
37
MaC with probability
►
Definition of Probability
– P(E) = |E| / |S| Ex. Sample Space S = {H,T} and Event E = {H}, P(E) = 0.5
– Long run frequency: P(E) = lim n(E)
n  n
• where n = the number of experiments performed, n(E) = the number of outcomes
belonging to the event E
• Ex. experiment is performing coin flipping, P(H) is the number of times the
outcome is H in relative to the number of times the coin flipping is performed
►
Challenges
– How to detect experiments from execution trace
– Accuracy of probability calculated from execution trace
►
Possible approach
– Detecting experiment using events or regular expressions
– Calculate probability from execution trace with confidence interval
– Possible syntax: E ~ [ p, Exp] and C ~ [ p, Exp]
where  = < | > | <= | >=, p is probability, Exp is an experiment
11/9/04
38
Extend MaC Architecture for IDS
11/9/04
39
Thank You!
11/9/04
40