Flexible
Service Composition
Adam Barker
School of Informatics
University of Edinburgh
Robert G. Mann
Institute for Astronomy
University of Edinburgh
Introduction
• Service Oriented Architectures
– Distributed computing platform targeted at the web
– Define a standard way to perform program to program
interaction
– Can tie together any OS, application, data store,
programming language, device etc.
– Defined using: XML, SOAP, WSDL etc.
• A Service Oriented World
– Wide spread adoption and interest in simple, vanilla web
service standards
– This standardised architecture is unlikely to change in the
forcible future
• Technologies need to fit around and consume these existing
standards
Scientific Workflow
• Most workflow engines focus on Business Process
Modelling
• A Scientific Workflow captures a series of analytical
steps which describe a computational experiment
– In order to verify a hypothesis etc.
• Scientific Workflow has an extra set of requirements:
–
–
–
–
–
–
–
Rapid prototyping of experiments
User Interaction with the scientist
Reliability and Fault Tolerant execution
Transparent access to resources
Repeatability, Smart re-runs and parameterisation
Provenance information, presentation of the results
Control Flow vs. Data Flow
Motivation
• Few languages which deal with the flexible,
knowledge acquisition and discovery
processes found in the sciences
– myGrid, Kepler, ICENI frameworks allow statically
defined, pre-designed/pre-planned workflows to
be executed by a centralised workflow engine
• The following science use-case serves as a
counter example of coordination which is
difficult to achieve by existing systems
– Large Synoptic Survey Telescope (LSST)
– Time-Domain astronomy
Science Use Case Background
• Current observatories are able to look very
deeply at very small sections of sky, not likely
to catch transient events:
– Observatories always looking somewhere else
– Small field of vision means that an impractically
large number of separate observations are
required to map the entire night sky
– Observing facilities are scarce
• Observations of change in the universe are
difficult to obtain
Science Use Case Background
• Automated Classification
– For known classes of object
– LSST is a first attempt at time-domain astronomy
• Likely to discover new (previously undetected) classes of object
– Always data which the automated stage cannot classify
• Typically most of these will be junk, however this may only be revealed
by comparisons with other detections made on the same night
Science Use Case
Web Service
Reasoning
Web Service
Web Service
Web Service
WorkFlow Requirements
• Complex coordination behaviour
– Data is inherently distributed
– Resources are scarce
• Requires selection and negotiation
– Large quantities of data
• Requires Autonomous, Intelligent software
• Human in the loop only when needed, e.g. specialist
– Workflow specification cannot be fully predicted
at design-time
• Difficult to classify data type or quantity in advance
• Component choice, e.g. web service, database etc.
– (semi) Flexible, Dynamic, runtime composition
• In a decentralised, peer-to-peer environment
Flexible
Reliable
Autonomous
Secure
Cooperation,
Negotiation
Scaleable
Run-Time
Decisions
Middleware
Support
Uncertain
Environments
Working Systems
SOA
Agents
A convergence of Interests
Workflow based on Interaction
Protocols
• Interaction Protocols
– Collection of conventions that allow cooperation between
agents in an open MultiAgent System
– A recipe for communication
– Defines ‘if and when’ agents communicate
– Order and kind of messages relating to a certain domain
• Multi Agent Service Composition (MASC)
– Agent-Based Workflow language and framework based on
Interaction Protocols
– Decentralised, peer-to-peer architecture
– Aimed at Scientific Workflow composition
• In order to solve the motivating science use-case
– Discuss the language and framework in the following slides
MASC: Scenes
• Scene
– Bounded space in which a group of
agents interact on a single shared task
– Divide a large, complex protocol into
manageable chunks
– Scenes cannot begin execution until all
agents have reference to the protocol
S
:= scene(ids, {ip}, {op}, {R}, {A})
ip
:= inport(ids:idpin, T)
op
:= outport(ids:idpout,T )
MASC: Roles
• Role
– A role type allows an engineer to specify a
pattern of behaviour which an agent can adopt
• Many agents can adopt the same role type
• Roles are defined as a set of methods {M}
– Methods are constructed from operations and
actions
A
:= agent(ida, idr, Φ(k) )
R
:= {idr, config(k) , {M}}
M
:= method idm(Φ(k)) = op
Φ
T
:= v:T, _ , c:T
:= XML Data Types
MASC: Operation Set
op
:=
|
|
|
|
|
action
op1 then op2
op1 or op2
op1 par op2
waitfor op1 timeout op2
invoke idm(Φ(k))
THEN
OR
PAR
RECURSION
MASC: Action Set
action :=
empty
(empty)
| p(Φ(k)) → agent( ida, idr)
(send)
| p(Φ(k)) → multicast(idr)
(multicast)
| p(Φ(k)) → user()
(user send)
| p(Φ(k)) ← agent(ida, idr)
(receive)
| p(Φ(k)) ← user()
(user receive)
| Φ(k) = portread(idpin)
(port read)
| portwrite(idpout, Φ(k))
(port write)
| Φ(k) = p(Φ(l)) fault Φ(m)
(decision)
| Φ(k) = service(ws, Φ(l)) fault Φ(m)
(service)
MASC: Decision Procedures
• Decision Procedures
– Connect the protocol code (describing the interaction
model) to an agent’s internal reasoning model
– Each agent interacting within a scene references a set of
decision procedures
• Implemented as a set of methods inside a reasoning web
service
• Throughout protocol execution agents can invoke methods
on its reasoning web service
• Does not sacrifice the autonomy of the agent
• Each agent can subscribe to their own reasoning model (BDI
etc.)
• Up to the agent engineer to specify these methods
• Allowing a personalised strategy within the interaction
protocol
MASC: Service Enactment
• External Web Services
– Agents consume external web services, acting as
a proxy to their execution
• Extra level of abstraction
– Allows agents to consume the passive Service
Oriented Architecture found in Internet and Grid
Systems
– Can be hard coded in the interaction protocol
or determined at runtime
• e.g. from a broker or message exchange
MASC: Execution
Web Service
R
W
R
P
R
P
W
P
W
P
R
W
MASC: Dataflow
P := protocol(idp,{S}, link(L)*)
L := source → sink+
• source/sink:
–
–
–
–
Web Service
Application
User interaction
File reading/writing
A
A
A
A
A
A
A
A
A
A
A
A
A
• High level experiment
composition
A
A
A
A
A
A
A
MASC: Layers of Abstraction
Scientist:
Has a Scientific Hypothesis
DataFlow
Engineer
Dataflow MASC
Port Definitions
Interaction
Engineer
Agent
Engineer
Scene Definition
Agents
Existing
SOA
Reasoning Model
Engineer:
Has an Engineering Aim
The MASC Framework
• Implementation:
– Full Java implementation
• Built using the Sun Java Web Services Development
Pack (JWSDP) 2.0
– Agents can be executed locally: as a closely
coupled system or as distributed processes
– SOAP messaging
– XML Language specification
– https://sourceforge.net/projects/multiagent
• Application
– Requirements based on Real Science use cases
– Applied to the UK e-Science project, AstroGrid
Advantages
• Standard MAS Arguments
– Inherently a Distributed Peer-to-Peer System
• No centralised server
• No central point of failure
• Allows engineering in an open environment
• Interactions that are too complex, or simply
cannot be specified at design-time
– Agents act as a proxy to the services which are
being coordinated, extra level of abstraction
– Reactive agents, runtime decision and
coordination
Advantages
• Inter-operability
– Infrastructure independent: Interaction model
always remains a layer above any specific
middleware or OS
– The web services being invoked require no
modification before taking part in the interaction
– Low engineering requirement
• Layer of s/w to translate protocol steps
• Reasoning Models
– Protocols do not sacrifice the autonomy of the
agents
– Each agent can subscribe to their own reasoning
model
Conclusions
• Internet and Grid Systems are filled with passive
objects (services)
• Agency paradigm offers a way of programming
autonomous, social and active components which
consume this SOA
• Language provides a way of applying the principles
and well understood concepts of agency to the
web service composition problem
• Framework fits in with existing standards
• AstroGrid provides a live science test bed
Questions
Thanks for listening