ORION CI
ORION CI
Conceptual Architecture Team
Progress Report
Agenda
Context
Presentation (Options)
Discussion
28 Mar 2006
ORION CI
Context
28 Mar 2006
ORION Context
ORION CI
28 Mar 2006
ORION CI
28 Mar 2006
Expanding Role of Environmental Observing
ORION CI
• From individual expedition to collaborative observation.
–
–
–
–
–
Long-term persistent presence, continuous 24/7 time series
Increasing numbers and diversity of sensors
Multiple, complex data streams
Many more types of users and applications
Ability to control instruments, observing systems, and large-scale
experiments.
• Ubiquitous data requiring greater differentiated expertise.
– From kilobits to terabytes in a matter of hours.
– Collaborative discovery by groups of researchers with different
vocabularies and expertise looking at complex data streams.
• A need for cooperating networks of data/processing/analysis
systems that are different than in the past.
28 Mar 2006
Team Members
ORION CI
Matthew Arrott (UCSD, Chair)
Alan Chave (WHOI)
John Graybeal (MBARI)
Eric Guillemot (NEPTUNE Canada)
Ingolf Krüger (UCSD)
Benoît Pirenne (NEPTUNE Canada)
28 Mar 2006
Team Mandate
ORION CI
• Develop a set of information system capabilities.
• Develop conceptual level design options to
support the Ocean Observatories Initiative (OOI)
systems operations and RFP development.
• Develop a CI Work Breakdown Structure (WBS).
• Create resource estimates for each line in the
WBS to prepare a Rough Order of Magnitude
(ROM) cost and timeline for the project.
28 Mar 2006
ORION CI
What do you want us to talk
about?
28 Mar 2006
Presentation Options
ORION CI
• How will using ORION be different from the way
science is done today?
• What will CI do for us as individuals and as a
community?
• How has CI been implemented in other
environments?
• What is the current architectural concept?
• How can we be sure this is a viable,
implementable infrastructure?
• How did you go about putting this together?
28 Mar 2006
ORION CI
How will using ORION be
different from the way
science is done today?
28 Mar 2006
Getting A Book: The Way It Used To Be
ORION CI
Quic kT i me™ and a
T IFF (LZW) dec om pres s or
are needed t o s ee thi s pi c ture.
Quick Time™a nd a
TIFF ( Unco mpre ssed ) dec ompr esso r
ar e nee ded to see this pictur e.
I need to special order a book.
What book do you want?
It’s by Rachel Carson, Silent Spring.
Yes, we can order that. It’s 23.95.
When will you have it?
Maybe Tuesday, but if it’s not in stock, not for a month.
Can you send it to me?
No, you’ll need to pick it up.
OK, can I pay by phone?
Yes, we’ll need your credit card and address.
OK, here’s my card info and address…
28 Mar 2006
OK, we’ll call you when it’s in.
Getting A Book: The Way It Is Now
ORION CI
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
28 Mar 2006
Doing Science: The Way It Has Been
ORION CI
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Hey Francisco, you still running that mooring in Monterey?
Yeah, John, what do you need?
Is it collecting current data?
Yeah, you looking for profiles?
Yup, are you doing that with an ADCP?
Yes, we’ll probably start profiling soon, too.
No, don’t need that. How do I get the data?
You want the raw data or QCd? Real-time?.
Oh, the QC’d, delayed mode data, definitely, last 3 years worth.
That’d be off our dods site, here’s the address..
Documentation come with that?
We got some, I’ll have to get one of my techs to send it to you…
28 Mar 2006
Doing Science: The Way It Is and Will Be
ORION CI
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
QuickT ime ™an d a
TIFF ( Uncomp res sed) deco mpre ssor
ar e need ed to see this pictur e.
28 Mar 2006
Doing Science: Another Option
ORION CI
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
28 Mar 2006
Or…
ORION CI
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
28 Mar 2006
Interfacing with ORION
ORION CI
Working with the system will be at least as easy
as it is now.
• Building an instrument?
– Instrument interfaces are defined
– Libraries and adapters are provided to help you.
– Core instruments will be taken care of.
• Discovering data?
– Lots of options, most of which actually work.
– Just “get the data” (as you want it) once you find it.
– Not just ORION data, data from lots of places.
• Running an observatory?
– Access to system status.
– Ability to coordinate the entire system as needed.
28 Mar 2006
ORION CI
Forward to
science
scenarios
Return to menu
28 Mar 2006
A Science Scenario
ORION CI
• What we have here
– An example for illustration
– Only one of many applicable science scenarios
• The example
– “Developing a complex instrument for ORION”
– Actually a platform, but we’ll call it an instrument here
– Covers most of the lifecycle of the instrument
• And furthermore…
– Accessing and using data from the instrument
28 Mar 2006
Developing a Complex Instrument for ORION
ORION CI
•
•
•
•
•
Platform carrying multiple sensors
Commandable
Not always connected
Multiple non-trivial data sets
Data transport varies
– may be stored and copied out of the instrument later
– may be streamed in (near) real time
28 Mar 2006
Steps Along The Way
ORION CI
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
28 Mar 2006
Develop: Create Instrument
ORION CI
What
What
does
does
my
Can
ORION
I ORION
really justify
IO may
In any
give
case, IFinally,
have ORION
I’m
just sent
Darn,me
need to
I guess I’d
instrument
say my have
instrument
buildingme
this?
funds
(May
(e.g.,
to getfor
moneybuilding
(may instructions
the
on testing
fix that./
better test it.
to do?
must call
do? for research.)
core systems)!
involve research).
instrument.
my instrument.
Yay, it’s ready!
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
28 Mar 2006
Commission: Test and Validate Instrument
ORION CI
Does system
Does my Put instrument
Make
in sure theMake sure my
know
my
instrument
talk to the water and
instrument is instrument is
Tell about
system
about
Record the
instrument?
system
correctly? hook it up. “behaving.”working correctly.
my instrument.
transaction.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
28 Mar 2006
Publish: Announce Availability of Instrument
Declare what
Describe
is
my Make sure this
Enter myGroup this with the
Tell everyone
ORION CI
the finalinstrument
version (using
description
instrument
is valid into
other
the instruments
this instrument
of my instrument
ORION forms).and complete.
approved list.
of similar products.
is available.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
28 Mar 2006
Discover: Find the Instrument
What
What’s
kind of
the
OK,
anbest
I’ll browse/
Son of a gun,
Let’s Ilook over Describe
Start the
Darn,
theneed to
ORION
CI
instrument
way toam
find
search
I it? using this didn’t knowhere,
they this list search
detailstry
I’m
again./
looking
(How &for?
where?)
interface.
have a forbitz!
looks relevant. already.
looking
Yay,
for.found it!
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
28 Mar 2006
Acquire: Get OK to Command Instrument
Please let “Can
me
“Sure,
Do user
youifagree
user
Nevertomind,
call I didn’t
“Authorizing
Your“Instrument
instrument Hi, Pat?
Of
ORION
control the frobnitz
control
the
calls
provider
this
me and
want
now to
and
use that access,
access
documentation
list includes:is
JanCI
course.
(private) instrument.
thing?”
credits
give credit
me.”
instrument
later? anyway.
pleasefrobnitz
wait.”
in the
ID#2365
mail.”
here.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
28 Mar 2006
Use: Command Instrument
This“set
is an
sample rate
Changing
I’m
on getting the dataI’d better cite
Thank you
OK,
fordon’t need
Do I need
ORION CI it
instrument
‘instrument
I can 23493’
sample
back,
=10
rateIto
can start mythat scientist
giving me credit
to command
to do more?
command.
Hz for 3600
10 Hz
secs”
for 1analysis
hour. now. like I agreedonto.your paper.
any more.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
28 Mar 2006
Govern: Restrict Instrument Control
ORION CI
Define accessIs it OK for this
“Is it, or is it not, OK
Is it still OK to use
rights and policies
user to use theto use instrument the instrument
for instrument instrument?
right now?
(so much)?
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
28 Mar 2006
What About the Data?
ORION CI
Forward to data
access scenarios
Return to menu
28 Mar 2006
Publish: Announce Data Available
Decide on the
Describe data
Make sure
Enter
this
my stream
Group this data
Tell everyone
this
ORION CI
final formatstream
for
(using
description into
is valid
the list ofwith the other data product is
data stream(s).
ORION forms).
and complete.
available data.
similar products. available.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
28 Mar 2006
Discover: Find the Data
What
What’s
kind of
the
OK,
anbest
I’ll browse/
OK, it knows
Sonme
of a gun,
Let’s Ilook over
Describe
Start the
the
Darn, need to
ORION CI
data am
wayI looking
to find
search
it? using
and this
is letting
didn’t
meknow
here,
theythis data
data sets
search
I want
try
again./
(How
for?& where?)
interface.
see the data
haveset.
carbonset
data!
relevant.inalready.
detail. Yay, found it!
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
28 Mar 2006
Acquire: Obtain the Data
Please giveCan
me
Sure,
Do
user
you
if user
accept
Oh, I this
didn’t want“Authorizing
OhClick
look,onthe
link,
CI
access to thehave
data
agrees
is not
tothe
quality
data if it isn’t
access,
Yes. download
please
watch
button’s
dataORION
(done)
frobnitz data stream.
data?
disclaimer
controlled?
Quality Controlled. wait.” enabled.
stream in.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
28 Mar 2006
Use: Modify Data, Present Results
I’m getting data, my software’s
I’d better cite
Thank
Canyou
IYup,
turn
fordon’t need
This is just data.
ORION it
CIany
running, I detected upwelling!
that scientist
giving
thisme
stream
to
credit
access
On we go.
I write my paper. like I agreedonto.youroff?
paper.more.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
28 Mar 2006
ORION CI
Return to menu
28 Mar 2006
ORION CI
What will CI do for us as
individuals and as a community?
28 Mar 2006
For the individual
ORION CI
•
•
•
28 Mar 2006
Provide workflow and resource management
tools to automate experimental design and
execution
Provide a secure work environment that
automatically mediates conflicts
Generalize and automate query and
publish/subscribe processes for retrospective
and real-time data
ORION CI
28 Mar 2006
ORION CI
•
•
•
28 Mar 2006
Dynamic Data Driven Assimilation System
(DDDAS) use case
Workflow binds diverse fixed and mobile
instruments to assimilation model and data
repository on shore
Resource conflict mediation is automated
ORION CI
28 Mar 2006
ORION CI
•
•
•
28 Mar 2006
Distributed remote multi-mission laboratories
distributed on an RCO
Resource intensive, shared use operations
require automated brokering
Quality of service policy constraints are
important
For the community
ORION CI
•
•
•
28 Mar 2006
Facilitates collaborative experimentation and
communication
Automated integration of ORION policies and
external obligations
Ability to archive data with reliable discovery
services for future use
ORION CI
28 Mar 2006
ORION CI
•
•
•
28 Mar 2006
Automated tracking and coordination of the
state of observatory resources
External resources can be integrated with
observatory assets and operations
Science use case becomes the observatory
use case
ORION CI
Return to menu
28 Mar 2006
ORION CI
How has CI been
implemented in other
environments?
28 Mar 2006
CI: Evolved from Grid Computing
ORION CI
• Grid Computing is a term for loosely coupled distributed
computing across a diverse community of resources, owners
• Many Grid Computing efforts and products
– Computational Grid (TeraGrid, Open Science Grid)
• Decomposition of large problems into many small atomic tasks
– Data Grid (Storage Resource Broker, OGSA-DAI)
• Federation & cataloguing of distributed data repositories
– Service Grid (On-Demand and Location-based service models—IBM,
Microsoft)
• Generalization of computational and data grids as “Service” patterns
• Emergence of Service Oriented Architectures
– Autonomic Grid (DMTF, WSDM from HP, IBM, CA, Oracle, Opsware)
• Autonomous resource management, load balancing and fault
detection isolation and recovery applied to networks of coupled
resources
• Grid computing contributes to overall cyberinfrastructure
28 Mar 2006
ORION CI Building on Other CI Projects
ORION CI
• GriPhyN, Atlas, Ligo, CMS
– Data distribution and shared computational grid
• NVO (National Virtual Observatory)
– Community data model, and shared data repositories and
applications
• BIRN (Biomedical Information Resource Network)
– Federated data repositories of disparate data models into a
common meta-catalog
– Resolution of disparate data models through mediation
• GEON (GEOsciences Network)
– Extends data mediation model with ontologies
• TeleScience and NEES
– Developed real-time control of remote instrumentation and the
coupling of remote physical and simulated systems
28 Mar 2006
Environmental Cyberinfrastructures like ORION
ORION CI
• What other projects are most like this?
– LEAD (Linked Environments for Atmospheric Discovery)
– NEON (National Ecological Observatory Network)
• What makes them similar?
– Significant increase in the number and diversity of
instruments and data products
– Real-time stream processing and inquiry
– Interaction with the sensing environment
– Semantic heterogeneity (mixed vocabularies)
28 Mar 2006
ORION CI
Return to menu
28 Mar 2006
ORION CI
What is the current
architectural concept?
28 Mar 2006
External Architecture
ORION CI
28 Mar 2006
Lifecycle of ORION Activities (v.B)
ORION CI
28 Mar 2006
Internal Architecture
ORION CI
28 Mar 2006
System Components
ORION CI
28 Mar 2006
Deployment Architecture
28 Mar 2006
ORION CI
Enterprise Service Bus
ORION CI
28 Mar 2006
ORION CI
Go to
Scenarios
Return to menu
28 Mar 2006
Viable Architecture
ORION CI
How can we be sure this is a
viable, implementable
architecture?
28 Mar 2006
Reasons for Confidence
ORION CI
• Open and transparent design process
• Ongoing review by external and internal experts and
stakeholders
• Iterative functional deployments reflecting use priorities
and system risks
• A rapidly maturing field
– Major subsystems have multiple implementations in existence
– Science efforts on which to build (BIRN, GEON)
– Many sophisticated commercial collaboration architectures,
particularly in web domain (Flikr, eBay, MySpace)
– Pioneering efforts in ocean cyberinfrastructure (LOOKING on
research side, NEPTUNE Canada in operation)
• Innovation will be focused on increased usability,
effectiveness and reliability
28 Mar 2006
Concerns
ORION CI
• Some technical areas deserve extra attention
– Incorporation of streaming data, a new data type, into
scientific process
– Specification and adoption of instrument services
– Effective operation of systems of systems that will
scale with the anticipated growth of ORION
– Interfacing with external organizations and resources
• Implementing Organizations take this on
– Evaluation and response to the most challenging tasks
– Collaborative development encouraged to maximize
success
28 Mar 2006
ORION CI
Return to menu
28 Mar 2006
ORION CI
How did you go about putting
this together?
28 Mar 2006
Methodology & Timeline
ORION CI
• DoDAF model
• Development Process
– Requirements
– Operational activities
– Organizational and operational entities
• Dictionary
– Domain model
– System components
– WBS
– Use domain experts for final costing
• Timeline
– May 15, 2006
28 Mar 2006
Characterize the System
ORION CI
• Identify the main areas of Research Activity,
Internal/External lines of Operations, and/or Support
Functions.
• Describe the operations that are normally conducted in
the course of achieving an experiment or research goal.
– Description includes: Capabilities, operational activities,
input/output flows between activities, input/output flows from/to
activities outside the scope of the architecture
• Characterize the command, governance, and business
relationships among the human, organizations, and
organizational type roles that are the key
participants/stakeholders in the architecture.
• Depict the operational nodes, including organizations, with
need lines between those nodes that indicate a need to
exchange information.
28 Mar 2006
Entity Activities
ORION CI
28 Mar 2006
Lifecycle of ORION Activities
ORION CI
28 Mar 2006
Example of Activity
ORION CI
28 Mar 2006
Internal Interaction Model
ORION CI
28 Mar 2006
Reference: NOSA Logical Data Model
ORION CI
28 Mar 2006
ORION CI
Return to menu
28 Mar 2006
ORION CI
Discussion: Questions,
Comments, Suggestions
28 Mar 2006