Application of Design for Verification
with Concurrency Controllers to
Air traffic Control Software
Aysu Betin-Can, Tevfik Bultan
Mikael Lindvall, Benjamin Lux, Stefan Topp
Department of Computer Science
University of California, Santa Barbara
Fraunhofer Center for Experimental Software
Engineering, Maryland
Concurrent Programming in Java
• Java uses a variant of monitor programming
• Synchronization using locks
– Each object has a lock
synchronized(o) { ... }
• Coordination using condition variables
– Objects can be used as condition variables
synchronized (condVar){
while (!condExp) wait(condVar);
...
notifyAll(condVar);
}
Dangers in Java Concurrency
• Nested locks
synchronized m(other) { other.m(); }
• Missed notification
notify(condVar);
• Forgotten condition check
if(!condExp) wait(condVar);
• Dependency among multiple condition variables can be
very complicated
– Conservative notification and condition check:
Inefficient
– Optimizing the notification and condition checks:
Error prone
What will I talk about today?
• A design for verification approach for eliminating
concurrency errors in Java programs
• Application of this design for verification approach to a
safety critical air traffic control software component
• Experiments demonstrating the effectiveness of the
presented design for verification approach
Model Checking Software
• Scalability of software model checking depends on
– Extracting compact models from programs
• This typically requires a reverse engineering step based on
user guidance and/or static analysis techniques to
– Rediscover some information about the software that
may be known at design time
• Alternative approach: Design for verification
– Structure software in ways that facilitate verification
– Document the design decisions that can be useful for
verification
– Improve the scalability of verification using this
information
A Design for Verification Approach
We have been investigating a design for verification approach
based on the following principles:
1. Using design patterns that facilitate automated verification
2. Use of stateful, behavioral interfaces which isolate the
behavior and enable modular verification
3. An assume-guarantee style modular verification strategy
which separates verification of the behavior from the
verification of the conformance to the interface
specifications
4. A general model checking technique for interface
verification
5. Domain specific and specialized verification techniques for
behavior verification
Concurrency Controller Pattern
• A verifiable behavioral design pattern for concurrent
programs
– Defines customized synchronization policies
– Avoids usage of error-prone Java synchronization
primitives: synchronize, wait, notify, notifyAll
– Separates controller behavior from the threads that use
the controller
• Supports modular verification and model extraction
Concurrency Controller Pattern
ThreadA
Shared
Controller
ThreadB
Shared
SharedStub
+a()
+b()
+a()
+b()
Helper
classes
Action
+blocking()
+nonblocking()
-GuardedExecute
used at runtime
used at interface verification
used both times
Controller
-var1
-var2
+action1()
+action2()
int
ControllerStateMachine
+action1()
+action2()
GuardedCommand
StateMachine
GuardedCommand
+guard()
+update()
Reader-Writer Controller
class RWController implements RWInterface{
int nR; boolean busy;
final Action act_r_enter, act_r_exit;
final Action act_w_enter, act_w_exit;
RWController() {...
gcs = new Vector();
gcs.add(new GuardedCommand() {
public boolean guard(){ return (nR == 0 && !busy );}
public void update(){busy = true;}}
);
act_w_enter = new Action(this,gcs);
}
public void w_enter(){
act_w_enter.blocking();}
public boolean w_exit(){
return act_w_exit.nonblocking();}
public void r_enter(){
act_r_enter.blocking();}
public boolean r_exit(){
return act_r_exit.nonblocking();}
}
Action Class: Used As Is
class Action{
protected final Object owner;
. . .
private boolean GuardedExecute(){
boolean result=false;
for(int i=0; i<gcV.size(); i++)
try{
if(((GuardedCommand)gcV.get(i)
).guard()){
((GuardedCommand)gcV.get(i)).u
pdate();
result=true; break; }
}catch(Exception e){}
return result;
}
public void blocking(){
synchronized(owner) {
while(!GuardedExecute()) {
try{owner.wait();}catch
(Exception e){} }
owner.notifyAll(); }
}
public boolean nonblocking(){
synchronized(owner) {
boolean
result=GuardedExecute();
if (result) owner.notifyAll();
return result; }
}
Controller Interfaces
• A controller interface defines the acceptable call sequences
for the threads that use the controller
• Interfaces are specified using finite state machines
public class RWStateMachine implements
RWInterface{
StateTable stateTable;
r_enter
reading final static int idle=0,reading=1,
writing=2;
public RWStateMachine(){ ...
r_exit
stateTable.insert("w_enter",idle,writing);
idle
}
w_exit
public void w_enter(){
stateTable.transition("w_enter");
writing }
w_enter
...
}
Verification Framework
Behavior
Specification
in Action
Language
Controllers
and
Interfaces
Action Language
Verifier
Behavior Translator
Counting Abstractor
Data Stubs
Behavior Verification
Notification-Optimizer
Error Trace
Verified
Optimized
Java Code
Interface Verification
Java PathFinder
Thread Isolation
Program with
Stubs
Rest of the
Program
Error Trace
Verified
Modular Verification
• Utilizes behavior and interface decoupling in the pattern
• Behavior verification
– Verify the controller properties (e.g. safety, liveness)
– Assume that the user threads adhere to the controller
interface
• Interface verification
– Check that each user thread obeys the interface:
• A thread is correct with respect to an interface if all
the call sequences generated by the thread can also
be generated by the finite state machine defining the
interface.
Behavior Verification
• Analyzing properties (specified in CTL) of the
synchronization policy encapsulated with a concurrency
controller and its interface
– Assume threads obey the controller interfaces
• Behavior verification with Action Language Verifier
– Infinite state symbolic model checker
– Suitable for specifications with unbounded variables and
parameterized constants
– We wrote a translator which translates controller classes
to Action Language
Behavior Verification for Arbitrary Number of Threads
• Counting abstraction
– Create an integer variable for each interface state
– Each variable counts the number of threads in a
particular interface state
– Automatically generate updates and guards for these
variables based on the interface specification
• Counting abstraction is automated
Three Types of Controller Properties
• Properties that only refer to controller variables
AG(busy => nR=0)
AG(busy => AF(!busy))
• Properties that refer to interface states
AG(pc=WRITING => AF(pc=IDLE)
AG(pc=READING => nR > 0)
• Properties for arbitrary number of threads
AG(WRITING > 0 => AF(IDLE > 0)
AG(READING = nR)
Interface Verification
• Checks if all the threads invoke controller methods in the
order specified in the interfaces
• Checks if the threads access shared data only at the
correct interface states
• Interface verification with Java PathFinder
– Verify Java implementations of threads
– Correctness criteria are specified as assertions
• Look for assertion violations
• Assertions are in the StateMachine and
SharedStub
– Performance improvement with thread Isolation
Thread Isolation: Part 1
• Interaction among threads
• Threads can interact with each other in only two ways:
– invoking controller actions
– Invoking shared data methods
• To isolate the threads
– Replace concurrency controllers with controller interface
state machines
– Replace shared data with shared stubs
Thread Isolation: Part 2
• Interaction among a thread and its environment
• Modeling thread’s call to its environment with stubs
– File I/O, updating GUI components, socket operations,
RMI call to another program
• Replace with pre-written or generated stubs
• Modeling the environment’s influence on threads with
drivers
– Thread initialization, RMI events, GUI events
• Enclose with drivers that generate all possible events
that influence controller access
Automated Airspace Concept
• Automated Airspace Concept by NASA researchers
automates the decision making in air traffic control
• The most important challenge is achieving high
dependability
• Automated Airspace Concept includes a failsafe short term
conflict detection component
– Dependability of this component is even more important
than the dependability of the rest of the system
– It should be a smaller, isolated component compared to
the rest of the system so that it can be verified
Tactical Separation Assisted Flight Environment (TSAFE)
•
TSAFE is an implementation of this failsafe short term
conflict detection component
– It is developed at MIT based on the design of the NASA
researchers
•
It is responsible for detecting conflicts in flight plans of the
aircraft within 1 minute from the current time
•
Functionality:
1. Display aircraft position
2. Display aircraft planned route
3. Display aircraft future projected route trajectory
4. Show conformance problems
Tactical Separation Assisted Flight Environment (TSAFE)
User
Radar feed
<<TCP/IP>>
Feed Parser
Server
Client
Flight
Database
EventThread
<<RMI>>
Graphical
Client
Computation
21,057 lines of code with 87 classes
Timer
Reengineering TSAFE
• Found all the synchronization statements in the code
(synchronize, wait, notify, notifyAll)
• Identified 6 shared objects protected by these
synchronization statements
• Used 2 instances of a reader-writer controller and 3
instances of a mutex controller for synchronization
• In the reengineered TSAFE code the synchronization
statements appear only in the Action helper class
provided by the concurrency controller pattern
Behavior Verification Performance
Controller
Time(sec)
Memory (MB) P-Time (sec)
P-Memory (MB)
RW
0.17
1.03
8.10
12.05
Mutex
0.01
0.23
0.98
0.03
Barrier
0.01
0.64
0.01
0.50
BB-RW
0.13
6.76
0.63
10.80
BB-Mutex
0.63
1.99
2.05
6.47
P denotes parameterized verification for arbitrary number of threads
Interface Verification Performance
Thread
Time (sec)
Memory (MB)
TServer-Main
67.72
17.08
TServer-RMI
91.79
20.31
TServer-Event
6.57
10.95
TServer-Feed
123.12
83.49
TClient-Main
2.00
2.32
TClient-RMI
17.06
40.96
TClient-Event
663.21
33.09
Fault Categories
• Concurrency controller faults
– initialization faults (2)
– guard faults (2)
– update faults (6)
– blocking/nonblocking faults (4)
• Interface faults
– modified-call faults (8)
– conditional-call faults
• conditions based on existing program variables (13)
• conditions on new variables declared during fault
seeding (5)
Effectiveness in Finding Faults
• Created 40 faulty versions of TSAFE
• Each version had at most one interface fault and at most
one behavior fault
– 14 behavior and 26 interface faults
• Among 14 behavior faults ALV identified 12 of them
– 2 uncaught faults were spurious
• Among 26 interface faults JPF identified 21 of them
– 2 of the uncaught faults were spurious
– 3 of the uncaught faults were real faults that were not
caught by JPF
Falsification Performance
Thread
Time (sec)
Memory (MB)
TServer-RMI
29.43
24.74
TServer-Event
6.88
9.56
TServer-Feed
18.51
94.72
TClient-RMI
10.12
42.64
TClient-Event
15.63
12.20
Concurrency Controller
RW-8
Time (sec)
0.34
Memory (MB)
3.26
RW-16
1.61
10.04
RW-P
1.51
5.03
Mutex-8
0.02
0.19
Mutex-16
0.04
0.54
Mutex-p
0.12
0.70
Conclusions
• ALV performance
– Cost of parameterized verification was somewhere
between concrete instances with 8 and 16 threads
– Falsification performance was better than verification
• Completeness of the controller properties
– Effectiveness of behavior verification by ALV critically
depends on the completeness of the specified properties
• Concrete vs. parameterized behavior verification
– When no faults are found, the result obtained with
parameterized verification is much stronger
– However for falsification we observed that concrete
instances were as effective as parameterized instances
Conclusions
• JPF performance
– Typically falsification performance is better than
verification performance
– In some cases faults caused execution of new code
causing the falsification performance to be worse than
verification performance
• Thread isolation
– Automatic environment generation for threads result in
too much non-determinism and JPF runs out of memory
– Dependency analysis was crucial for mitigating this
• Deep faults were difficult to catch using JPF
– Three uncaught faults were created to test this
Conclusions
• Unknown shared objects
– The presented approach does not handle this problem
– Using escape analysis may help
• We could not find a scalable and precise escape
analysis tool
• Environment generation
– This is the crucial problem in scalability of the interface
verification
– Using a design for verification approach for environment
generation may help
Related Work
• Design for Verification
– [MP03] Mehlitz et al. promote using design patterns and
exploiting their properties for automated verification
– [SBK01] Sharygina et al. focus on verification of UML
models
– Design for verification has been used in hardware design
and embedded systems [SF03], [GSB02], [SBBCM01]
• Assume-guarantee style modular verification for software
– [PDH99], [CCGJV03]
Related Work
• Design Patterns for multi-threaded systems
– [SSRB00], [Lea99], [Gra02],[SPM96]
• Verification and synthesizing monitors
– [DDHM02], [YKB02], [MK99]
• Environment extraction/generation
– [PDH99],[TD03], [TDP03]
Related Work
• Behavioral and stateful interfaces
– Interface compatibility checking [CAHJM02]
– Extended type systems with stateful interfaces and
interface checking as a part of type checking
[DF01,DF04]
– Interface discovery and synthesis by analyzing existing
code [WML02],[ACMN05]