Architecture Task Force2
Third Meeting
CeSC
Cambridge
Centre for Mathematical Sciences
18th December 2003
1
Administration
Present
Malcolm Atkinson (chair & note taker)
Howard Chivers
John Darlington
Andrew Herbert
Dave De Roure
Tom Rodden
Joe Sventek
Notes of Last Meeting
Agreed as useful
Actions
Some people still haven’t supplied info for the Web Page!!!!
2
Agenda
Notes of Last Meeting
Actions
Presentations
Chivers
Sventek
Presentations
Visions & Showstoppers
Integration & Classification
The Next Steps
The Next Meeting
3
Note
The slides beyond this one were typed by me
(Malcolm Atkinson) as we went through the
meeting – they were slightly modified afterwards
by John Darlington and myself reading them
through quickly
They are an impression of the meeting in
chronological order, with some repetition
They do not necessarily represent the combined
views of the meeting or the eventual position of
the ATF2
4
Presentation 1: Howard Chivers
Architecture in Industry
Or what is an architecture?
What kind of things do we want to constrain?
Large scale data warehousing systems
Data cleaning _. Analysis
Large systems + high Tx rates + lots of users
Each component large
User requirement sets dictate division
Different developers
Enterprise view
Want to share infrastructure
Life cycle costs, etc.
Continuous business processes
5
HC: cont
Total system-wide properties (e.g. security)
Security, legal, etc.
Architecture
Good enough set of block diagrams and infrastructure
constraints
X
Can give guarantees at total business requirements
To constrain individual developments just enough
X
Leave the component development teams enough freedom
What is the enterprise
Building for the future ⇒ don’t know use cases
6
HC: cont
Design approaches
Top down – scales well
X
High initial cost, inflexible & bespoke subsystems results
Cyclic replacement
X
Flexible, maximises immediate,, COTS infrastructure
– Unspecific contracts, assumes infrastructure
– ignores whole life cost, difficult to scale
Alternatively, set of infrastructures around
X
X
X
Define a virtualisation layer
Does it exist already?
E.g. Java was a VM for HC’s example
Deployment issues
X
X
What is an application
What is the deployment life cycle
7
HC: cont
Process objectives
Early choice of the infrastructure
X
X
X
X
X
What set of services
Virtualisation layer?
What physical services buy & deploy
First set
Cycled development of infrastructure
Void early specification
X
X
X
Collect generic requirements initially
Not specific requirements ∴reqs change
Avoid collecting detail!
8
HC: cont
Three views
Application view
X
X
Business process view – data flows and workflows
Broad but shallow use cases
Functional model at a high-level early
X
X
Major interfaces aservices
Deployment into application subsystems
Quantitative model
X
X
Data requirements get missed unless you do this at a high-level
Pairwise agreements ignore
9
HC: cont
Infrastructure Model
Foundation layers: Network & Computing
Application Environments (EJB, web, DB, …)
Infrastructure services (security, directories/registries,
event logging, data storage, messaging, …)
Quantitative model
Application deployment
System management features
X
Event monitoring, system diagnosis & control)
10
HC:cont
Interfaces
Layered from infrastructure to application
X
X
Comms style, e.g. SOAP, opaque msg delivery
Generic standards, XSDs, protocols,
Generic interoperability << valuable place to look at this
X
X
X
Format and content or message
Environments, e.g. namespaces
Services required to interpret content
Application integration
X
X
Behaviour of application (high-level protocol)
May have deployment information
11
HC: cont
Frameworks, e.g.
Data access
X
X
X
Defines ‘n-tier’ access architecture between user & data
Ensure common approach between several systems
Prevent each application inventing their own
Security << care about this in larger system
X
X
X
Policy framework for whole system
– E.g. localise data requiring a particular sensitivity applies
– People’s names, etc. in only one place
– Then protect with a firewall
Required services
Required constraints on parts of the system
– Infrastructure & application model
12
HC: discussion
Never time to just do the circle/triangle
Do we have a clear idea of what the apps will be
Major changes to business practices
High rate of perturbation now
What was Grid specific?
Cross organisational aspects
Discussion about relationships between organisations
No one without core business survives
Projects & IPR
Business strategy
Microsoft is the “Big shark” needs fish in the ecosystem
13
Presentation 2: JS
Systems & Infrastructure Vision
Previous experience at DOE lab.
Most successful infrastructure mechanisms
IP → IP6
What should we do
IP at the Grid level
Build on distributed O/S
14
JS: cont
O-O architecture
Statically defined and assembled into apps
X
Inflexible and brittle
Didn’t scale at network level
ANSA/CORBA transparencies
Aspect-Oriented Computing
Statically defined separate views
Tools weave views together to produce applications
X
Again static result
Policy-Oriented Computing
Where we are going
15
JS: cont
Policy-oriented
Constraints/preferences that must be met
Dynamically use the policies to integrate components
Continue to adjust composition
Beware large monolithic applications
Two forms of dynamism
X
X
Every person / group uses it differently
– Business processes evolve
Changing technology
Control-plane is where we most need commonality
“Lesson” from telecoms partners of ANSA
X
But business model adapted to infrastructure – we carry flaky phones
16
JS:cont
Why did IP work
Sit at centre of hour glass
⇒ must adapt to changes above & below
Agreed that wouldn’t always be optimal
What does the hour glass mean for us at this
level
Big change – we have many kinds of resources
Job dispatching
What are the crucial activities above us?
What do we have beneath?
Dynamically changing sets of machines, …
17
JS: cont
Can view this as a distributed virtual operating system
Deal with scheduling and sharing of all kinds
of dynamically varying resources
Back to policy
Some work on policy based management
Does this this lead to (semi) autonomic computing
Much work needs doing here
Basis for understanding the hour glass
We then understand what we are trying to define
Key features: Virtual Organisations
Resources
Plug & Play only recent
Think about what they do as an input
18
JS:Cont
Where does policy take us?
How do we represent policy
What can we say / not say
What do policies apply to
What resolution systems can we build
How do we engage the human
How do policies compose
Internet routing policies are a microcosm
19
JS:cont
History of networks
We just add the number of features we can assume
Seamless access to all resources available in
the virtual organisation, at any scale.
This implies
Distributed O/S, with pre-emptive scheduling
X
Aggressive split between policy and mechanism split
21st Century security
20
JS:cont
Big question
What are we trying to architect
The policy space at the centre of the hour glass
Abstract patterns needed?
Expect that these will evolve
What tests our progress
What information do we want to get into policies
Follow the abstract patterns
Avoid the concrete framework, etc. traps
What are the dominant patterns we should capture
What vocab do we need?
21
JS: Discussion
Policy
Chance to link between formal models & systems
Opportunity for good tools
X
X
Can map to what you have today
With tool input for what we want tomorrow
Declarative things should play well here
Separate meaning from behaviour
X
Separation of concerns
How do we define our boundaries
How far above / below the waist of the hour glass
Need a temporal model
22
Presentation 3: Howard Chivers
Architecture – some requirements
Centre projects by type
Data centric (information grids) and
users/visualisation applications
39% of the projects developing infrastructure
Because it wasn’t there (MPA’s theory)
Indefinite behaviour of composing things
May always have the wrong emphasis in common
infrastructure
“The Grid” is too simple a deployment model
23
HC: cont
The Grid has missing structure
Support for groups of resources
Support for groups of people
Separate the VO of users
From the VO of resource suppliers
Get linked by policy and brokering
Virtualisation
Applies to provide & publish resources
Does the separation of supply and consumption win
us anything?
24
JS: cont
Resource provider states what policy operates
for use of the resources
25
Presentation 4: John Darlington
Deploying applications over Grids
Information Capture & Use
Keywords from e-Science
X
X
Easy, transparent, …
Use, development & execution
Keywords from Grid
X
Complex applications on complex machinery
Capture knowledge about build, composition, use
ICENI
Component architecture
X
Place to hang knowledge
Resources that will be used
X
Place knowledge about these with their proxy
Discover, interact, broker
X
Dynamically revise binding between Component & Resource
26
JD: cont
Component
Unit of composition in which all context dependencies are
explicit
Service
Component performing a task
Abstraction levels for components
Roles
X
Sources of information (in CXML)
Developer
X
Produces implementations
Scientist
X
Designs useful components
User
X
Composes and executes application
27
JD: cont
Separation of concerns
Meaning, behaviour & implementation
Composition must satisfy constraints
Tool help possible
Slides on steps from composition to execution
Component definition to Execution
On slides
28
JD: continued
What description
Not yet related to standards
Describes both resources and services
Structured language
X
X
High-level language
Set of information that needs to be collected
Tools to extract workload and performance
X
Magpie [HOTOS 2002]
Feedback loop into development and maintenance
X
Failure analysis
29
Oral Presentation: Andrew Herbert
Conversation last week
Used to be difficult to persuade company
interest
Hard to find believable business case
Even harder on standards
Trying to get focus
Steve Tuecke + Indigo project team
Debate went surprisingly well
Distinguish between services and resources
Recognition of the possibilities
30
AH: Cont
Where it ended up
WS-Context story carrying things around
Many major problems evaporated
Now seems to be dialogue
Person assigned person
X
Marvyn Theimer
Encouraging conversation
Frustrations between Globus
Harmony about where want to go with WS
31
AH: cont
We should look at composing WS & resources
Where do we see opportunities
Collaboration
Living rooms, consumers & wireless
32
Presentation 5: Dave De Roure
Semantic Grid workshop @ GGF8
Grid problem: interoperability
Scale driving the requirement
E-Research
Computation, data
People & VOs
Knowledge – digital artifacts
Extended enterprise
Need large scale heterogeneous, interoperability
Components, … bring them all together
33
DDR: cont
Architecture that describes frameworks for components
and their operation
Principles
Means of describing collections of infrastructure cmpnts
Meta-information architecture
Quote from Tom (coherence crucial)
Not just doing it
Make it easy to do
Shift of focus
X
X
X
From developer
Through experience builder
Towards participant
34
DDR: cont
Focus on user
Ease of assembly
Ease of design-build-test
Self configuration
Ease of ownership
Semantic Web – brings technology
Network effect
URI server/relationships
RDF bus – rules
Examples: comb-e-chem goes to lab and
publication – early provenance, etc. – smart tea
35
DDR: cont
Bootstrap problem
Incentives to create metadata not apparent to early
users
Standards for standards versus standards
Community-based standards
X
X
Gene-ontology vs Globus ontology
Can we every reach such a complete / extensive / coherent
knowledge integration
GGF provides some of the framework for this
Both sharing and interoperating have potential
problems
X
Lots of small things that interoperate
36
DDR: cont
Semantic web might not work
Distributed semantic web unproven
Still being researched
Live & real work underway
37
Digesting Meeting
Consistent points
Descriptions central
Not a common infrastructure
Minimal knowledge and tools to make architecture work
Agreed Document target for completion by May
Identification of scope, requirements and structure of
architecture
Constraints
Can’t assume static system
Avoid detail
But danger of nothing to get teeth into
Understand variants from practical systems
Pilots and IRCs at NeSC
38
Digestion 2
February’s meeting
How do we extract the high-level patterns
Are these classes of business practices
Or common patterns within applications
What is our idea about composition
Examples
Information used in build and deploy
Lists of terms / words
OMII harvest – look at this – find persistent principles
Describing architectures and tests
Agent-oriented architecture relevant?
39
Digestion 3
Very high cost of looking after components and
deployed code
Several lines of discussion here
Got involved and stopped taking notes
Timed out
40
DONMs
February UCL
Identification of Common Patterns to Support
General business
X
X
Outline of Report (Scope, Limits, Goals & Structure)
Commitment to write Parts for Review at next meeting
Invite 2 or 3 e-Science visionary scientists for 2 hours
X
Peter Coveney was agreed to be a good choice
Late March & early April
Imperial
May/June
Goal: A report / published matrix / draft road map
By end of June 04
41