MaCS: Monitoring, Checking
and Steering
M. Kim
O. Sokolsky, S. Kannan,
I. Lee, U. Sammapun,
J. Shin, M. Viswanathan
SECUi.com, Korea
CIS, Penn
Outline





Motivation
Runtime Formal Analysis
Our MaCS Prototype
Examples
Future Work
Motivation


Two traditional approaches to certify correctness of
systems
 Testing cannot guarantee correctness of application
completely and lacks of formality.
 Formal verification lacks scalability and does not apply to
implementation directly
 gap between models and implementations
We need a new approach - run-time formal analysis
Run-time Formal Analysis



Run-time formal analysis ensures the run-time
compliance of an execution of a system with its
formal requirement.
The analysis validates properties on the current
execution of application.
The analysis can

detect incorrect execution of applications

predict error and steer computation

collect statistics of actual execution
Run-time Formal Analysis using MaCS


To detect incorrect execution, we need
 Monitor: monitors changes in an application by
instrumenting monitor code into an application.
This will create an abstract view of the application.
 Checker: uses monitor information and
requirement spec to detect violation
To steer, we need
 The violation information from Checker and the
Feed-back to the application
MaCS Methodology
Formal verification
System
Spec
Requirement
Spec
Design
System
Implementation
Monitoring
Data
Implementation
System
Run-time Check
Filter
Communication
Monitor +
Checker
Issues in Run-time Formal Analysis

An expressive formal language describing correctness
criteria

A proper granularity of monitoring

Automatic v.s. Manual instrumentation

Synchronous v.s. Asynchronous monitoring

Side effect of instrumentation to a target system
Outline





Motivation
Runtime Formal Analysis
Our MaCS Prototype
Examples
Future Work
MaCS languages
Run-time state:
•control locations
•object state
•local variables
PEDL
SADL
Abstract state:
•events
•conditions
•auxiliary variables
MEDL



PEDL: abstraction
MEDL: abstract transformation
SADL: feedback
Events and Conditions

Information about the system comes in
two different forms:
 Conditions, which are true, false, or
undefined for a finite duration of time

Events, which are either present or absent
at some instant of time
Examples of Events and Conditions
E1. event startProg = startM( IP.init() );
E2. event voltsChange= update( IP.volts);
C3. condition safeVolts =
( -10 < IP.volts ) && ( IP.volts < 10 );
E1
E2
E2
C3
IP.init()
IP.volts = 6
IP.volts = 15
Logic for Events and Conditions



Let c be a primitive condition
Let e be a primitive event
Define Condition (C) and Event (E) as:
 <C> ::= c | [ <E>, <E> ) | ! <C> |
<C> && <C> | <C> || <C> |
<C> => <C>
 <E> ::= e | start( <C> ) | end( <C>) |
<E> && <E> | <E> || <E> |
<E> when <C>
Conditions

<C> ::= c |
Primitive condition
[ <E1>, <E2> ) |
! <C> |
T from E1 until E2
T if C is F
<C1> && <C2> |
T if C1 and C2 are T
<C1> || <C2> |
T if C1 or C2 are T
<C1> => <C2>
T if C2 is T whenever
C1 is T
Events

<E> ::= e |
Primitive event
start( <C> ) |
end( <C> ) |
An instant when C becomes
T
An instant when C becomes
F
<E1> && <E2> |
<E1> || <E2> |
<E> when <C>
An instant when both
events happen
An instant when either
events happen
An instant when E
happens and C holds T
Primitive Event Definition Language
(PEDL)


The language that creates an abstract view of
the application
It maps the low-level state information of
the system to high-level events/conditions
used in describing the requirements.
Primitive Event Definition Language
(PEDL)
Program
Abstract View
Execution
Monitor Sees
x<2
x=0,y=0
x=1,y=0
Information
Extraction
x=2,y=0
x =2
x=2,y=1
x=3,y=1
x=3,y=2
x> 2
Primitive Event Definition Language
(PEDL)


Provides primitives to refer to values of variables
and to certain points in the execution of the
program.
 condition safeVolts
=
( -10 < IP.volts ) && ( IP.volts < 10 );
 event startProg = startM( IP.init() );
 event voltsChange= update( IP.volts);
Provides primitive “time” to refer to time when
events happen
 condition slowDoor =
( time(doorClose) - time(doorOpen) ) > 3000;
Meta Event Definition Language
(MEDL)

Expresses requirements using the events and
conditions, sent by a monitor.

Defines Auxiliary Variables, which may be used
to store history.


RaiseGateEvent -> { num_train_pass++; }
Defines more events and conditions using auxiliary
variables and events/conditions sent by a monitor.

event safeStart = startProg when (safeVolts)
Meta Event Definition Language
(MEDL) (Cont. )


Using auxiliary variables, events and conditions, we
describes the safety requirements of the system,
in terms of conditions that must always be true,
and alarms (events) that must never be raised.
 safeprop safeIP = safeTrack && safeVolts;
 alarm violation = start (!safeIP);
Specifies updates to variables and steering
invocations in response to events

violation -> { controller = 0;
invoke change2SafetyController(); }
Property checking

A MEDL specification can also be seen as an
automaton running on a stream of events provided
by the monitor and the auxiliary variable
aux. variables
Steering Action Definition Language
(SADL)

What: Identifies object used in steering
// the target of steering is the object dm of type
// DecisionModule located in the class IP
DecisionModule IP:dm;

How: Defines steering actions
// setSC() method of dm is invoked
steering action change2SC = { call (IP:dm).setSC(); }

Where: Specifies steering conditions

locations in the code where the actions can be
executed
before read DecisionModule:volts;
Steering process
action
invocation
received
violation
steering
condition
satisfied
action executed
system
event
received
checker
action
invoked
detection
MaCS Toolset
Steering Script
(SADL)
Program (.class)
(Java byte code)
SADL
Compiler
Instrumentation
Information
Injector class
(Java byte code)
Monitoring Script
(PEDL)
Requirements
(MEDL)
PEDL
Compiler
MEDL
Compiler
Instrumentation
Information
Filter Generator
(JTREK)
Instrumented
Code
Compiled
PEDL
Compiled
MEDL
Event Recognizer
Checker
Filter (or an instrumented class)


Initializes the connection between application and
monitor.
For each monitored object:
 for each variable update or start/end method, send
events/conditions to the monitor.
Filter (or an instrumented class)
Instrumented
class
IP.init()
IP.volts = 6
IP.volts = 15
send to monitor
Implementation of monitor

EventReconizer class:
 Listener (i.e. server)
 accepts events and conditions, specified in PEDL,
from filter.
 Notify (i.e. client)
 sends events and conditions to runtime checker
Implementation of Monitor
Instrumented
class
IP.init()
IP.volts = 6
Monitor
startProg
voltsChange
safeVolts = T
IP.volts = 15
voltsChange
safeVolts = F
send to checker
Implementation of checker

RuntimeChecker class:
 Listener (i.e. server)
 accepts events and conditions, specified in MEDL,
from monitor.
 Check for any violation
 If violation occurs, check if any action needs to be
invoked by Steering
 Notify (i.e. client)
 sends action to be invoked to steerer.
Implementation of Checker
Instrumented
class
IP.init()
IP.volts = 6
Monitor
Checker
startProg
startProg
voltsChange
safeVolts = T
IP.volts = 15
voltsChange
safeVolts = F
OK
send to steering
component
ALARM
alarm violation = voltsChange
when !safeVolts
Implementation of steering


Injector class:
 Action bodies as methods
 Listener thread
 accepts action invocations and sets invocation flags
Calls to action bodies at fixed locations
 defined in the steering script, added by instrumentation
Implementation of Checker
Instrumented
class
IP.init()
IP.volts = 6
Monitor
Checker
startProg
startProg
voltsChange
safeVolts = T
IP.volts = 15
voltsChange
safeVolts = F
IP.setSC()
Injector
OK
ALARM
alarm violation = voltsChange
when !safeVolts
Action
change2SC
Implementation Issues

How close should the application and
the monitor be?

Tight coupled


Pro: Decrease Steering Latency
Con: Compete for resources
Implementation Issues

Conservativeness

How to ensure that the application does
not suffer a significant performance
degradation as long as steering is not
invoked?
Implementation Issues

Effectiveness


How to ensure that the steering action
cannot be ignored by the application?
How to ensure that after the action is
invoked, the application will not “undo” the
action?
Example
g
l
q
uf
m
M
x
Inverted Pendulum (courtesy L. Sha)
Simplex Architecture (courtesy L. Sha)
Safety
Experimental
us
Decision
Module
u
Physical
System
x
x0
ue
EX
SC
Equilibrium
state


Experimental controllers provide improved performance
but uncertain stability properties

Can be dynamically added or replaced
Safety controller has the largest stability region
Inverted Pendulum in MaCS
Experimental
Experimental
Experimental
Controller
Controller
Controller
Device
Drivers
angle,
track
volts
J
N
I
Decision Module
Safety
Controller
monitor
Switching
logic
steer
Case study results


It works!
Without disturbance, there is slightly
more jitter


partly due to instrumentation but mostly due to
Java
Switch to the safety controller takes
0.32s (16 control cycles)


bad values compensated by the safety controller
in general may not be good enough
Future Work

Synchronous vs. asynchronous steering

currently: the application and may produce
erroneous outputs when checking is slow


alternative: before producing critical output,
pause and wait for checker feedback



May be unacceptable in critical situations
IP case study suggests it is possible in some cases
Need predictable checker turnaround
Currently work in progress
Future Work

Static analysis to enhance steering



currently, user is responsible for specifying steering
locations – error prone
control flow analysis can identify locations by codeindependent criteria
 e.g. “steering location is visited once in each main
loop iteration”
 or even “if action changes variable x, do not place
before the system writes x”
 What are the right criteria?
Study other uses of steering