Deciding Choreography Reliazability
Samik Basu
Iowa State University
Tevfik Bultan
University of California at Santa Barbara
Meriem Ouederni
University of Malaga
Motivation 1: Web Services
• Web services support basic client/server style interactions
WSDL
Request
Service
Requester
Client
SOAP
Response
Service
Provider
Server
• Example: Amazon E-Commerce Web Service (AWS-ECS)
Service Composition via Choreography
• Can we compose a set of services to construct a new service?
• For example:
– If we are building a bookstore service, we may want to use both
Amazon’s service and Barnes & Noble’s service in order to get
better prices
Choreography: A specification of how the individual services that
participate to a composite service should interact with each other
• A choreography is a global specification of interactions among
services
• Web Services Choreography Description Language (WS-CDL)
Motivation 2: Singularity OS
• Experimental OS developed by Microsoft Research to explore new
ideas for operating system design focusing on dependability
• Software Isolated Processes (SIPs)
– Closed code space (no dynamic code loading or code generation)
– Closed object space (no shared memory)
• Inter-process communication occurs via message passing over
channels
• Singularity channels allow 2-Party asynchronous communication via
FIFO message queues
– Sends are non blocking
– Receives block until a message is at the head of a receive queue
Singularity Channel Contracts
• Written in Sing #
• Contracts specify two things:
1. The messages that may be sent
over a channel
• out message are sent from the
Server endpoint to the Client
endpoint (SC)
• in messages are sent from the
Client endpoint to the Server
endpoint (CS)
public contract KeyboardDeviceContract {
out message AckKey( uint key );
out message NakKey();
out message Success();
in message GetKey();
in message PollKey();
state Start {
Success! -> Ready;
}
state Ready {
GetKey? -> Waiting;
PollKey? -> (AckKey! or NakKey!)
-> Ready;
}
2. The set of allowed message
sequences
• out message marked with !
• in messages marked with ?
state Waiting {
AckKey! -> Ready;
NakKey! -> Ready;
}
}
Motivation 3: Erlang
• Erlang is a general purpose programming language developed initially
at Ericsson for improving dependability of telephony applications
• In Erlang distributed processes do not share memory and only interact
with each other via exchanging messages asynchronously
• UBF(B) is a language for specifying communication contracts in
distributed Erlang programs.
• UBF(B) specifications list
transitions between states where
each transition is identified with a
request (the message received)
and response (the message sent)
+NAME(“IRC SERVER”)
...
+STATE start
logon()
=> ok() & active
| error() & stop
+STATE active
ls()
=> files() & active
getFile() => fileSent() & active
| noFileErr() & stop
...
Common: Asynchronous Messaging
• Sender does not have to wait for the receiver
– Message is inserted to a message queue
– Messaging platform guarantees the delivery of the message
• Why support asynchronous messaging?
– Otherwise the sender has to block and wait for the receiver
– Sender may not need any data to be returned
– If the sender needs some data to be returned, it should only wait
when it needs to use that data
– Asynchronous messaging can alleviate the latency of message
transmission through the Internet
– Asynchronous messaging can prevent sender from blocking if the
receiver service is temporarily unavailable
• Rather then creating a thread to handle the send, use
asynchronous messaging
Common: Conversations
• Specifications of message-based asynchronous communication
– Web Service Choreography Specifications: Global specification
of interactions for composition of services
– Singularity Channel Contracts: Coordinating inter-process
communication in Singularity OS
– Erlang Communication Contracts: Coordinating interactions
among distributed processes
• All these specifications can be modeled as state machines and they all
specify sequences of send actions (aka, conversations):
Conversation: A sequence of send actions
Conversation Protocol (aka Choreography): Specifies a set of
conversations
Example Singularity Channel Contract
• A Singularity channel contract corresponds to a finite state machine
• Each message causes a deterministic transition from one state to
another state
public contract KeyboardDeviceContract
out message AckKey( uint key );
out message NakKey();
out message Success();
KeyboardDeviceContract
in message GetKey();
in message PollKey();
Start
SC:AckKey
SC:Success
Waiting
SC:AckKey
Ready
SC:NakKey
state Start {
Implicit
Success! -> Ready;
State
}
CS:PollKey
CS:GetKey
{
Ready$0
SC:NakKey
state Ready {
GetKey? -> Waiting;
PollKey? -> (AckKey! or NakKey!)
-> Ready;
}
state Waiting {
AckKey! -> Ready;
NakKey! -> Ready;
}
}
Example Singularity Channel Contract
• Each contract state machine specifies a set of conversations, i.e., it is
a conversation protocol:
KeyboardDeviceContract
Start
SC:AckKey
SC:Success
CS:PollKey
CS:GetKey
Waiting
SC:AckKey
Ready
SC:NakKey
Ready$0
SC:NakKey
Conversation set:
Success(GetKey(AckKey|NakKey)|PollKey(AckKey|NakKey))*
Going to Lunch at UCSB
• At UCSB Samik, Meriem and I were using the following protocol for
going to lunch:
– Sometime around noon one of us would call another one by phone
and tell him where and when we would meet for lunch.
– The receiver of this first call would call the remaining peer and pass
the information.
• Let’s call this protocol the First Caller Decides (FCD) protocol.
• At the time we did not have answering machines or voicemail due to
budget cuts at UC!
FCD Protocol Scenarios
•
•
•
Possible scenario
1. Tevfik calls Samik with the decision of where and when to eat
2. Samik calls Meriem and passes the information
Another scenario
1. Samik calls Tevfik with the decision of where and when to eat
2. Tevfik calls Meriem and passes the information
Yet another scenario
1. Tevfik calls Meriem with the decision of where and when to eat
• Maybe Samik also calls Meriem at the same time with a
different decision. But the phone is busy.
• Samik keeps calling. But Meriem is not going to answer
because according to the protocol the next thing Meriem has
to do is to call Samik.
2. Meriem calls Samik and passes the information
FCD Protocol: Tevfik’s Behavior
Let’s look at all possible behaviors of Tevfik based on the FCD protocol
Tevfik calls Samik with the
lunch decision
Tevfik is hungry
Tevfik calls Meriem
with the lunch decision
Tevfik receives a call
from Meriem telling him
the lunch decision that
Tevfik has to pass to
Samik
Tevfik receives a call from
Samik passing him the
lunch decision
Tevfik receives a call from
Meriem passing him the
lunch decision
FCD Protocol: Tevfik’s Behavior
T->S:D
Message Labels:
!
?
Tevfik calls Samik with the lunch decision
send
receive
S->M:P
Samik calls Meriem to pass the decision
!T->S:D
?S->T:P
!T->M:D
?M->T:P
?S->T:D
!T->M:P
?M->T:D
!T->S:P
State machines for the FCD Protocol
Tevfik
Meriem
!T->S:D
?S->T:P
!M->S:D
?S->M:P
!T->M:D
?M->T:P
!M->T:D
?T->M:P
Samik
!S->T:D
!S->M:D
?S->T:D
?M->T:D
?S->M:D
?T->M:D
?T->S:D
!T->M:P
!T->S:P
!M->T:P
!M->S:P
!S->M:P
?T->S:P
?M->S:P
?M->S:D
!S->T:P
• Three state machines characterizing the behaviors of Tevfik, Meriem
and Samik according to the FCD protocol
FCD Protocol Has Voicemail Problems
• After the economy started to recover, the university installed a
voicemail system FCD protocol started causing problems
– We were showing up at different restaurants at different times!
• Example scenario:
– Tevfik calls Meriem with the lunch decision
– Samik also calls Meriem with the lunch decision
• The phone is busy (Meriem is talking to Tevfik) so Samik leaves
a message
– Meriem calls Samik passing the lunch decision
• Samik does not answer (he already left for lunch) so Meriem
leaves a message
– Samik shows up at a different restaurant!
• Message sequence is: T->M:D S->M:D M->S:P
– The messages S->M:D and M->S:P are never consumed
• This scenario is not possible without voicemail!
A Different Lunch Protocol
• To fix this problem, I suggested that we change our lunch protocol as
follows:
– As the most senior researcher among us I would make the first call
to either Meriem or Samik and tell when and where we would meet
for lunch.
– Then, the receiver of this call would pass the information to the
other peer.
• Let’s call this protocol the Tevfik Decides (TD) protocol
State machines for the TD Protocol
Samik
?T->S:D
Meriem
Tevfik
?M->S:P
?T->M:D
!T->S:D
?S->M:P
!T->M:D
!S->M:P
• TD protocol works fine with voicemail!
!M->S:P
FCD and TD Conversation Protocols
TD Protocol
FCD Protocol
T->M:D
T->S:D
S->M:P
S->M:D
M->T:D
T->S:P
M->S:D
S->T:P
T->S:D
T->M:D
S->T:D
T->M:P
M->T:P
S->M:P
M->S:P
M->S:P
Conversation set:
{ T->M:D M->S:P,
T->S:D
M->T:D
M->S:D
S->T:D
S->M:P,
T->S:P,
S->T:P,
T->M:P,
S->M:D M->T:P
}
Conversation set:
{ T->S:D S->M:P,
T->M:D M->S:P}
Observation & Question
• The implementation of the FCD protocol does not obey the FCD
protocol if asynchronous communication is used
• Implementation of the TD protocol obeys the TD protocol even if
asynchronous communication used
– Given a conversation protocol can we figure out if there is an
implementation which generates the same conversation set?
Realizability
• Conversation protocols identify the global communication behavior
– How do we implement processes that conform to the conversation
protocol?
• Realizability question:
– Given a conversation protocol, are there processes whose
communication behavior in terms of conversations (i.e., send
sequences) is equal to the set of conversations (i.e., send
sequences) specified by the conversation protocol?
Conversations specified by
the conversation protocol
?

• The FCD protocol is unrealizable
• The TD protocol is realizable
Conversations generated by
some processes
Top-Down Verification
Conversation
Protocol
(Choreography
Specification)
Peer T
?M->S:P
T->S:D
T->M:D
?
S->M:P
LTL property
F(S->M:P  M->S:P)
M->S:P
Peer X
Peer J
!T->S:D
?S->M:P
?T->M:D
?T->S:D
!S->M:P
Conversation
!T->M:D
!M->S:P
... ?
F(S->M:P  M->S:P)
LTL property
Input
Queue
Unrealizable Conversation Protocols
• There are unrealizable conversation protocols:
AB: m1
BA: m2
AB: m1
AB: m1
BA: m2
CD: m2
CA: m2
AB: m1
AC: m3
Unrealizable Examples
• Some conversation protocols are unrealizable!
AB: m1
!m1
!m2
?m1
?m2
CD: m2
Peer A
Conversation
protocol
Peer B
Peer C
Peer D
Projections of the protocol to the processes
Conversation “m2 m1” will be generated by all
implementations which follow the protocol
Unrealizable Examples
• Some conversation protocols are unrealizable!
AB: m1
!m1
!m2
?m1
CA: m2
?m2
Peer B
Conversation
protocol
Peer C
Peer A
Projections of the protocol to the processes
Conversation “m2 m1” will be generated by all
implementations which follow the protocol
Unrealizable Examples
Conversation:
B
BA: m2
A, C
m2
A
m1
B
m3
C
AB: m1
BA: m2
AB: m1
m2
m1
m3
AC: m3
Generated conversation:
m2 m1 m3
Challenge & Contribution
• Finite state processes that communicate with FIFO message queues
can simulate Turing Machines
– Checking conformance to a conversation protocol is undecidable
• We show that conversation protocol realizability problem is decidable
• We implemented the realizability check and applied it to many
specifications
– Demonstrated that realizability can be checked efficiently in
practice
Refining Realizability
• Just looking at equivalence of the conversation sets is not enough
Conversations specified by
the conversation protocol
?

Conversations generated by
some processes
Another Conversation Protocol
aP1->P2
bP2->P1
cP2->P1
aP1->P2
bP2->P1
A conversation protocol for 2 processes: P1 and P2
Projections on P1 and P2
Process P1
!a
?b
!a
?c
?b
Process P2
?a
!b
!c
?a
!b
Synchronous Communication
Process P1
!a
!a
aP1->P2
?b
?c
?b
bP2->P1
Process P2
?a
!b
!c
?a
!b
cP2->P1
Synchronous Communication
Process P1
!a
!a
aP1->P2
?b
?c
?b
bP2->P1
Process P2
?a
!b
!c
aP1->P2
?a
!b
cP2->P1
bP2->P1
Synchronous Communication
Process P1
!a
!a
aP1->P2
?b
?c
?b
bP2->P1
Process P2
?a
!b
!c
aP1->P2
?a
!b
cP2->P1
bP2->P1
Synchronous Communication
Process P1
!a
!a
aP1->P2
?b
?c
?b
bP2->P1
Process P2
?a
!b
!c
aP1->P2
?a
!b
cP2->P1
bP2->P1
Synchronous Communication
Process P1
!a
!a
aP1->P2
?b
?c
?b
bP2->P1
Process P2
?a
!b
!c
aP1->P2
?a
!b
BLOCKED
cP2->P1
bP2->P1
Synchronous Communication
Process P1
!a
!a
aP1->P2
?b
?c
aP1->P2
?b
bP2->P1
cP2->P1
bP2->P1
Process P2
?a
?a
Conversations sets are equal but
processes may get stuck
!b
!c
!b
Asynchronous Communication
Process P1
!a
?b
!a
?c
Process P2
?a
!b
Queue:
!c
?b
Queue:
?a
!b
Asynchronous Communication
Process P1
!a
Queue:
!a
aP1->P2
?b
?c
Process P2
?a
!b
!c
?b
Queue: a
?a
!b
Asynchronous Communication
Process P1
!a
Queue:
!a
aP1->P2
?b
?c
Process P2
?a
!b
!c
?b
Queue:
?a
!b
Asynchronous Communication
Process P1
!a
Queue:
!a
aP1->P2
?b
?c
?b
bP2->P1
Process P2
?a
!b
!c
Queue:
?a
!b
Asynchronous Communication
Process P1
!a
Queue:
!a
aP1->P2
?b
?c
aP1->P2
?b
bP2->P1
Process P2
?a
!b
!c
Queue: a
?a
!b
Asynchronous Communication
Process P1
!a
Queue:
!a
aP1->P2
?b
?c
aP1->P2
?b
bP2->P1
Process P2
?a
!b
!c
Queue:
?a
!b
Asynchronous Communication
Process P1
!a
Queue: c
!a
aP1->P2
?b
?c
?b
cP2->P1
Process P2
?a
!b
!c
aP1->P2
Queue:
?a
!b
bP2->P1
Asynchronous Communication
Process P1
!a
Queue: c
!a
aP1->P2
?b
?c
?b
Cannot consume c
cP2->P1
Process P2
?a
!b
!c
Queue:
?a
!b
aP1->P2
bP2->P1
Asynchronous Communication
Process P1
!a
Queue: c
!a
aP1->P2
?b
?c
?b
Cannot consume c
cP2->P1
Process P2
?a
!b
!c
Queue:
?a
!b
aP1->P2
bP2->P1
Realizability Requirements
We refine our realizability condition to eliminate such cases.
We have two requirements for realizability:
1. Conversations specified by the conversation protocol =
Conversations generated by the asynchronous system
2. Asynchronous system is well-formed:
All sent messages can be eventually consumed
Conversation protocol is realizable if and only if there exists such an
asynchronous system
Summary of our contribution
Conversation protocol: C
Asynchronous System
• with unbounded buffer: I
• with k size communication buffer: Ik
Synchronous System: I0
C is realizable if and only C is equivalent to determinized I1 obtained from
projections of C
Determinizing Projections
Process P1
!a
?b
!a
?c
?b
Process P2
?a
!b
!c
?a
!b
Determinizing Projections
Peer P1
!a
?b
!a
?c
?b
!a
?c
?b
Peer P2
?a
!b
!c
?a
!b
?a
!c
!b
Observation 1: Behavioral Order
• Behavior exhibited by projections when communicating synchronously
is larger than the conversation
• Behavior exhibited by projections when communicating
asynchronously is larger than that exhibited by projections when
communicating synchronously
C ≤ I0 ≤ I1 ≤ I2 ≤ … ≤ I
Observation 1: Behavioral Ordering
aP1->P2
bP3->P4
cP2->P1
!a
?c
P1
?a
!c
P2
!b
P3
?b
P4
Observation 1: Behavioral Ordering
aP1->P2
bP3->P4
cP2->P1
!a
?c
P1
?a
!c
!b
P2
P3
?b
P4
bP3->P4
aP1->P2
bP3->P4
cP2->P1
cP2->P1
aP1->P2
bP3->P4
Synchronous System
Observation 1: Behavioral Ordering
aP1->P2
bP3->P4
!a
?c
P1
cP2->P1
aP1->P2
aP1->P2
cP2->P1
bP3->P4
Asynchronous System
?a
!c
!b
P2
P3
?b
P4
bP3->P4
cP2->P1
cP2->P1
aP1->P2
bP3->P4
Synchronous System
Observation 2: Synchronizability
A system is synchronizable if and only if its behaviors are identical for
asynchronous and synchronous communication
For synchronizable systems:
Forall k ≥ 0: Ik is equivalent to I
I is synchronizable iff I0 is equivalent to I1
[WWW’11: Choreography Conformance via Synchronizability]
Observation 2: Synchronizability
Peer P1
!a
?b
!a
?c
?b
Peer P2
?a
!b
!c
?a
!b
This system is synchronizable
since the asynchronous and
synchronous versions are
equivalent in term of
sequences of send actions
but it is not well-formed
Observation 3:
Synchronizability & Determinism
A synchronizable system that consists of deterministic processes is wellformed (all sent messages are eventually consumed)
Observation 3
Peer P1
!a
?b
!a
?c
!a
?c
?b
?b
Synchronizable
but not well-formed
Peer P2
?a
!b
Synchronizable
and well-formed
!c
?a
!b
?a
!c
!b
Outline of the Realizability Check
• Project conversations to processes
• Determinize peers
• Check equivalence between conversation C and I1
– C = I1 if and only if I is synchronizable [Obs 1, 2] and C = I
– C = I1 implies I is well-formed [Obs 3]
• C = I1 if and only if C is realizable
Implementation
• Implemented using CADP toolbox
– Automatically generate a LOTOS specification for the conversation
protocol
– Generate determinized projections (in LOTOS)
– Check equivalence of the 1-bounded asynchronous system and the
conversation protocol
• Checked realizability of
– 9 web service choreography specifications
• 8 are realizable
– 9 collaboration diagrams
• 8 are realizable
– 86 Singularity channel contracts
• 84 are realizable
• Realizability check takes about 14 seconds on average
Related Work
• Sufficient conditions for realizability:
– [Fu et al. TCS’04] Conversation Protocols
• [Honda et al. POPL’08] has similar conditions for session types
– Arbitrary Initiators are not allowed: Conversation protocol cannot
have two different peers initiating send actions from the same state
– [Stengel and Bultan ISSTA’09]: Application of sufficient realizability
conditions to checking Singularity channel contracts
– [Halle and Bultan FSE’10]: more relaxed sufficient condition that
allows arbitrary initiators
• [Kazhamiakin, Pistore FORTE’06]: Realizability for restricted
communication models
• [Lohmann, Wolf ICSOC’11]: Shows decidability of realizability with
unbounded asynchronous communication when messages are not
ordered (i.e., FIFO requirement is dropped)!
Related Work
• Message Sequence Charts (MSC)
– [Alur, Etassami, Yannakakis ICSE’00, ICALP’01] Realizability of
MSCs and MSC Graphs
• Defines similar notion of realizability
– [Uchitel, Kramer, Magee ACM TOSEM 04] Implied Scenarios in
MSCs
– Different conversation model
Related Work
Results on synchronizability:
• [Fu et al. TSE’05]: Sufficient conditions for synchronizability
• [Basu and Bultan WWW’11]: Necessary and sufficient condition for
synchronizability
• [Basu, Bultan, Ouderni VMCAI’12]: Synchronizability considering send
sequences + reachability of synchronized states
• [Manohar, Martin MPC 98] Slack elasticity
– Presents conditions under which changing the size of
communication queues does not effect the behavior of the system
– Behavior definition also takes the decision points into account in
addition to message sequences
– It gives sufficient conditions for slack elasticity and discusses how
to construct systems to ensure slack elasticity
Related Work
• Singularity:
– [Hunt, Larus SIGOPS ‘07] Singularity: rethinking the software stack
– [Fähndrich, Aiken, Hawblitzel, et. al SIGOPS/Eurosys ‘07]
Language support for fast and reliable message-based
communication in singularity os.
– Influenced by work on Session Types
• [Honda, Vasconcelos, Kubo ESOP ’98] Language primitives
and type discipline for structured communication-based
programming
– Source code and RDK: http://codeplex.com/singularity
Future Directions
• Choreography realizability for other communication models
• Analyzing failure of realizability
– Correcting unrealizable choreographies with minimal changes to
the choreography
THE END