Transactional Workflow
9:00
11:00
13:30
15:30
18:00
Aug. 2
Intro &
terminology
Reliability
Fault
tolerance
Transaction
models
Reception
Aug. 3
Aug. 4
Aug. 5
Aug. 6
TP mons
Logging &
Files &
Structured
& ORBs
res. Mgr.
Buffer Mgr.
files
Locking Res. Mgr. &
COM+
Access paths
theory
Trans. Mgr.
Locking
CICS & TP
CORBA/
Groupware
techniques & Internet
EJB + TP
Queueing
Advanced
Replication Performance
Trans. Mgr.
& TPC
Workflow Cyberbricks
Party
FREE
Chapter 9

What Is the Problem ?
“A workflow management system is an
active system that manages the flow of
business processes performed by multiple
persons in multiple steps. It gets the right
data to the right people with the right tools
at the right time.”

(This definition omits a umber of aspects:
roles, events, errors, cooperation, ...)
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
2
In More Technical Terms: What Is
Workflow ?




WF is a long-lived execution involving a potentially
large number of autonomous agents such as
programs, databases, sensors, actors, humans.
Control flow and data flow are (partially) pre-defined
and may evolve over time.
There are numerous interdependent consistency
criteria.
A WF must be kept alive by “the system” under all
circumstances.
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999

3
Components of a Workflow
compensation
step
fork
Source

case
case
loop
Step
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
4
Steps ...






execute application logic of any kind,
can interact with human operators,

access shared data in databases,
depend on events and can create their own
events,
have a short duration and (should) behave like
classical transactions,
are invoked depending on the execution history.
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
5
The Script ...






specifies the control flow and the workflow,
defines the event and data conditions under

which a step is to be executed,
defines the synchronization criteria for accessing
shared data,
maintains the local execution context of a
workflow instance,
handles resource conflicts, in particular on
shared data,
represents a persistent execution.
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
6
What Is Transactional Workflow ?



There are different interpretations:
Extended transaction models adapted to the
needs of workflow: Sagas, Flex transactions,
etc.
Application of some transactional properties
such as isolation and durability to workflows.
Using classical distributed transactions to
implement the control flow machinery of a
workflow system.
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999

7
What Happens to the Transactional
Properties ?
A: Atomicity does not apply to an entire workflow
C: Consistency must be redefined including the
temporal dimension.
I: Isolation must be limited in time; cooperation
must be allowed.
D: Rather than the effects of transaction, the
execution itself must be durable.
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999

8
Correctness


Transactional correctness guarantees a
consistent overall state if each individual step is
executed correctly (or not at all) and there was a 
consistent initial state.
In long-lived executions, this definition cannnot
be used, because

strict isolation is not feasible and

rollback is not option at the workflow level.
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
9
Rollback vs. Compensation


Rollback is based on the assumption that an
erroneous state can be reverted to the previous
(correct) state without affecting anybody =>
Isolation.
Compensation tries to modify an erroneous state
such that all the consistency constraints work as
though the faulty operation was never executed
=> Formal definition of consistency.
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999

10
Correctness For Long-Lived Activities


Transactions can be executed iff the possibility
of rollback is maintained.
A step in a workflow can be executed iff



the individual step can be rolled back and
its commitment does not block any of the previously
executed steps from being compensated if needed.
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
11
What Are Invariants For?

Compensation must be guaranteed for
completed steps =>



certain predicates on the shared and local state must
be maintained.

The requirements for a state to be executable
are formalized as combined event / state
predicates called “invariants”.
Invariants are alos useful to describe
correctness criteria for forward execution.
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
12
Types of Invariants


Entry invariants guard the execution of a step.
If an entry invariant is violated, there are different
options:




give up (compensate),
negotiate,
resolve conflict.

Exit invariants formalize the new consistent state. Its
protection can be



strict (must),
moderate (want),
lose (hope).
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
13
Invariants: Virtual Objects




Invariants may contain expressions like:
obj_1 + obj_2 rel_op value.
The objects are not necessarily managed by the
same RM.
To support such invariants, they are established
as virtual objects, which:

have a special name,
have a “value” method,
are stored at each participating RM,

are evaluated locally whenever possible.


© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999

14
Dynamic Aspects of Invariants
pa2
pa3
pa4

p1
pb2
pb3
pb4
Individual invariants established by each step
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
15
Dynamic Aspects of Invariants
p1&pa2
p1&pa2&pa3 p1&pa2&pa3&pa4

p1
p1&pb2
p1&pb2&pb3
p1&pb2&pb3&pb4
Accumulated invariants for a “case of” control flow
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
16
Dynamic Aspects of Invariants

Invariants can be deleted if the step with the
corresponding entry invariant will never be
executed. This implies:




All invariants become obsolete at the end of a
workflow.
Dead code must be detected dynamically.
We need special loop invariants.

If a step´s compensation step is dynamically
modified, this may cause problems.
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
17
Supporting Invariants

In order to support invariants, database systems
must give up some their autonomy. In particular,
they have to:





provide notification about lock conflicts,
implement recoverable locks,
implement semantic locks (e.g. escrow),
implement existence locks at the tuple and at the
schema level.
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
18
Using Transactions in a WorkflowSystem
Transaction 2

Transaction 1
Application-level transactions for grouping multiple
steps
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
19
Using Transactions in a Workflow-System
B
C

System-level transactions for transferring control
from one step to the next
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
20
Using Transactions in a Workflow-System
B
C

System-level transactions for transferring control
from one step to the next
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
21
Using Transactions in a Workflow-System
System transaction A
transfer
control
receive
request

C
B
notify
CM
notify
CM
CM
System-level transactions for transferring control
from one step to the next
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
22
Workflow and Transactional Queues
local
processing

Input queue
commit
B
post
C
Version 1: Synchronous transfer
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
23
Workflow and Transactional Queues
CommitPost-Transaction
local
processing
Input queue
commit
B

Transfer Transaction
C
output
queue
Version 2: Asynchronous transfer
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
24
Queue-Driven Step Processing
Input queue
Output queue
3. post
1.
dequeue
C
Local code
2. local execution
© Jim Gray, Andreas Reuter

Complete
local transaction
incl. transfer
of control
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
25
Queues And Rollback
Normal processing
dequeue
local
processing

C
post
local
rollback
Transaction abort
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
26
Problems Related to Rollback






Why did the local transaction fail (system abort or
application-initiated rollback?)
In which cases should the TA be re-posted (and how
often)?
Who gets notified about an abort (source or CM)?
Who gets notified about the eventual failure to restart a
transaction?
Should application-level TAs be treated as distributed or
nested transactions at the system level?
Which programming level should handle these issues
(step or script)?
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999

27
Summing It Up - 1


Transactional concepts can be carried over to
workflow management in multiple ways and at
differet levels.

The most obvious application of transaction
technology is at the level of the workflow engine,
where transactions provide persistent execution
of a script, local recovery in case of partial
failures, reliable state transitions, recoverable
events, and consistent context.
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
28
Summing It Up - 2



Transactions at the system level need a number
of extensions: nesting, chaining, leave-resume,
transfer.

Participating resource managers need to be able
to support an open distributed two-phasecommit protocol.
Persistent storage managers need a number of
functional extensions such as recoverable locks.
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999
29
Summing It Up - 3



At the step level, transactions provide atomicity
for short-term related computations.
The TM must be able to support dependen-cies
among transcations.
At the script level, the concepts of atomicity and
consistency have to be translated into more
abstract notions (compensation, invariants).
© Jim Gray, Andreas Reuter
Transaction Processing - Concepts and Techniques
WICS August 2 - 6, 1999

30