Run Time Monitoring of
Reactive System Models
Mikhail Auguston
Naval Postgraduate School
maugusto@nps.edu
Mark Trakhtenbrot
Holon Academic Institute of Technology
marktr@openu.ac.il
Outline
Problems in development of reactive systems
Model-based development and analysis
Our approach:
 TL-based assertion language
 Automatic generation of monitors for run-time verification
Outline of translation scheme
Further steps
Why development of reactive
systems is difficult ?
Main problem: complex behavior
 Intricate event-driven interaction with the environment
 Concurrency and timing factors
Typically:
 Critical applications (embedded RT controllers, …)
 Must fulfill tough reqs (safety, timeliness, … )
Challenges:
 Capture reqs; design; check that system meets its spec
So, what to do?
Apply formal methods!
Precise spec of behavior:
 TL [Pnueli, …] (future/past modalities: LTL; timing: MTL)
 templates for typical properties [Avrunin, …]
(response, precedence, existence, absence,…)
Model-based development:
 formal model of system (behavior, structure, …)
 Harel statecharts to capture behavior; a renown standard
 executable model; a basis for automated tools
 model-level analysis (closer to problem domain)
Example:
Early Warning System
ON
[in(DISCONNECTED)]
WORKING
[in(CONNECTED)]
EXECUTE
C
IDLE
COMPARING
RESET
DO_
SET_UP
OUT_OF_RANGE
S
dly(15)/
st!(PRINT_ALARM)
POWER_ON
DONE_
SET_UP
OFF
/CONNECT_OFF
DISPLAYING_
ALARM
POWER_OFF
SETTING_UP
CONNECTED
CONNECT_ON
DISCONNECTED
dly(5)
SAMPLING
WAIT
CONNECT_OFF
SAMPLING_DONE
Model-based analysis
with Statemate
Simulation: check, record and replay test scenarios
Verification: based on predefined assertion templates
Code synthesis: executable C, for host or target OS
Watchdog charts for execution monitoring:
- co-executed with the system model
- property violation: entering into ERROR state
Sample monitor chart
Assertion:
Processing of a request must be accomplished
- within 5 seconds
- before receiving the next request
Wait
Request
Processing
Done
Process
Request or
Timeout(5)
/Violation
Notification
Error
Our approach:
Statemate-based
run-time verification
Main new features:
 TL-based assertion language to spec behavior
 Automatic creation of monitor charts from assertions
Used Statemate capabilities:
 Co-translation of model and monitors to C
 Monitoring of code execution
Advantages of the approach
Flexible assertion language vs. predefined patterns
Smooth transition: assertions  executable monitors
Overcome limitations of simulation and verification
Overcome limitations of
simulation and verification



Analyze models reflecting design decisions
(tasks/events mapping,…), in realistic target environment
Real time analysis - rather than SIM time
SIM synch time (clock-driven systems):
all reactions assume same duration
SIM asynch time (event-driven systems):
all reactions assume zero duration
No "finite state" limitation
may connect to real input sources
Assertion language

Boolean expressions: system state properties
in(S) and (x > 5) ;

entered(S)  started(A)
;
ch(d)
Regular expressions: describe sequences of states
[SELECT (Open | Read | Write | Close) FROM execution]
SATISFY Open (Read | Write )* Close

Temporal formulas: order properties
ALWAYS (request  EVENTUALLY response)
ALWAYS (execute  SOMETIME_WAS do_set_up)

Timed temporal formulas: real-time restrictions
ALWAYS (request  EVENTUALLY (10) response)

Actions: to trace interesting events (e.g. property violation)
reports, profiling info (based on objects’ attributes)
More on TL formula classification
Basic formula: no temporal operators
Restricted formula:
EVENTUALLY(15) (ALWAYS(30) p)
ALWAYS(30) (p  EVENTUALLY(15) q)
Unrestricted formula:
 nesting depth <= 2 for unrestricted future operators
(following Manna&Pnueli for LTL)
 unrestricted operator not embedded in a restricted one
Semantic issues



Monitoring: deals with finite runs;
semantics of TL – based on infinite runs
[Eisner, et.al.]: reasoning on truncated executions
We follow the neutral view
Assume the run finishes 4 sec after the last p
ALWAYS (p  EVENTUALLY(10) q) ; false if no q after the last p
ALWAYS (p  ALWAYS(10) q) ; true if always q after the last p


In general: user responsible to provide proper actions
Example - assertion for EWS
ALWAYS (OUT_OF_RANGE 
EVENTUALLY (15) (RESET or
started(PRINT_ALARM)))
ON_FAIL [printf(“Violation after occurrence of
OUT_OF_RANGE at time %f “,
OUT_OF_RANGE.occur_time)]
Sample monitor construction
ALWAYS (OUT_OF_RANGE 
EVENTUALLY (15) (RESET or started(PRINT_ALARM)))
/ ERROR_MESSAGE
Translation of restricted formula
ALWAYS (N) P
Case 1:
------P a is basic formula
Case 2:
------P contains only restricted temporal operators
/RES:=false
P
/RES:=false
IDLE
START_P
IDLE
dly(1)
F
not P
D
D
F
P
P
.....
START_P
F
dly(N)
dly(2)
P
dly(N)/
RES:=true
IDLE
IDLE
D
F
F
D
F
D
t(FRM) = N
Computation may finish in less than N time units
Maximum time needed to compute value of formula FRM:
t(FRM) = N + t(P)
D
At least one component
is in D / RES:=false
All components
are in F / RES:=true
Maximum time needed to compute value
of formula FRM:
START_P
START_P
Translation of unrestricted formula
ALWAYS (P)
P is a resticted formula: t(P) = N
===================================
/RES:=false
COMP_0
COMP_1
/T0:=
CURR_TIME
COMP_2
COMP_N
IDLE
IDLE
P
IDLE
dly(2)/
T2:=CURR_TIME
dly(1)/
T1:=CURR_TIME
dly(N)/
TN:=CURR_TIME
START_P
P
D
RESTART_P_0/
RES:=true
F
...
START_P
START_P
F
D
RESTART_P_1/
RES:=true
RESTART_P_i = dly(N - mod(CURR_TIME-Ti, N))
P
P
F
D
RESTART_P_2/
RES:=true
...
START_P
D
F
RESTART_P_N/
RES:=true
At least one
component is in D /
RES:=false
D
Further steps
Implementation:
 done - experiments with manually created monitors
 next step – actual implementation
A more friendly assertion language
 e.g. combine English with formulas
Optimized translation for certain types of assertions
Apply to other design paradigms using statecharts
 e.g. in OO: monitor systems with dynamically created
objects (problematic for model checking)
Run Time Monitoring of
Reactive System Models
Thanks for your attention
More on temporal properties
Manna & Pnueli Classification (P – past formula) :
 Safety: ALWAYS (P) , Guarantee: EVENTUALLY (P)
 Obligation: Boolean combination of safety & guarantee
 Response: ALWAYS (EVENTUALLY(P))
 Persistence: EVENTUALLY (ALWAYS(P))
 Reactivity: Bool. combination of response & persistence
 Any TL formula is equivalent to a Reactivity formula
Limitations:
 nesting depth <= 2 for unrestricted future operators
 unrestricted operator not embedded in a restricted one