Integration of Simulation Results
into Information Systems
Gio Wiederhold
April 2002, updated Nov 2002
Gio Wiederhold SimQL 1
Information Integration
Information Integration provides new Information for
improved Decision Making when it
 presents more data
(mediation 1991)
 Risk: much may be irrelevant
 captures new relationships
(knowledge bases 1977)
 Often requires expert inter-domain knowledge
 includes current sensor data
(fusion, streams 1999)
 Data reflect the past only
 includes predictions about future courses
******* A new, potentially major topic *******
Gio Wiederhold SimQL 2
Decision-making (DM)
Analyze Alternatives
• Current Capabilities
• Future Expectations
Process tasks:
• List resources
• Enumerate alternatives
• Prune alternative
• Compare alternatives
now
future
Gio Wiederhold SimQL 3
Prediction Requires Tools
E-mail this book,
Alfred Knopf, 1997
Gio Wiederhold SimQL 4
Future information systems
Combine data from the past, with current data,
knowledge, and predictions into the future
o o
oo
o o
Assessment of the
values of alternative
possible outcomes
Gio Wiederhold SimQL 5
DM support is disjoint
does not interoperate
Planning Science
Distribution
extensions to move
to networked support
are also disjoint
Gio Wiederhold SimQL 6
Current state of DM Support
past
now
organized support
Data integration
Databases
distributed, heterogeneous
future
time
disjointed support
x17 @qbfera
ffga 67 .78 jjkl,a
nsnd nn 23.5a
Intuition +
• Spreadsheets
• Planning of allocations
• Other simulations
various point assessments
Gio Wiederhold SimQL 7
Information Systems should also
Project into the Futures
past
now
future
time
Support of decision-making requires dealing with the futures,
as well the past
• Databases deal well with the past
• Sensors can provide current status
• Spreadsheets, simulations deal with the likely futures
Information systems should be able to combine all three
Gio Wiederhold SimQL 8
New Resources for Information
Application
Layer
Mediation
Layer
Foundation
Layer
decision-makers at workstations
value-added services
data and simulation resources
Gio Wiederhold SimQL 9
Interfaces enable integration:
SimQL to access Simulations
past
now
future
time
Databases,
accessed via SQL or
XML, CORBA compliant
wrappers
Simulations,
accessed via SimQL and
compliant wrappers
Msg
systems,
sensors
Gio Wiederhold SimQL 10
Prototype Implementation
Developer
Customer
Query
Development
Interaction
Help
Schema
Manager
Parser
Schema
Commands
Metadata
Manager
Filing of
Access
Specs Metadata
Use of
Access
Specs
Help
Production
Interaction
Schema
Commands
Query
manager
Initiation and
Results of
Simulations
Error
reports
Wrapped ..
Simulations
Gio Wiederhold SimQL 11
Stanford experiment, supported by DARPA & NIST
Phase 1 Architectures
Logistics
Application
Manufacturing
Application
SimQL access
SimQL access
SimQL access
SQL access
wrapper
Spreadsheets
wrapper
wrapper
wrapper
Weather
(short-, long-term)
Test
Data
Engineering
Gio Wiederhold SimQL 12
Language - simple for demo
SQL
SELECT Temperature, Cloudcover, Windspeed, Winddirection
FROM WeatherDB
WHERE Date = `yesterday' AND Location = `ORD'.
----> {75, .30, 5, NW}
SimQL
PREDICT Temperature, Cloudcover, Windspeed, Winddirection
FROM WeatherSimulation < name of wrapper accessing web pages >
WHERE Date = `tomorrow' AND Location = `ORD'.
-----> { (75, .8), (.30,.8), (5, .8), (NW, .8) }
Would prefer XML representation
Gio Wiederhold SimQL 13
Place of SimQL in Objective-based Planning
Higher Level Objectives, Intel, OB,
ROE, Commanders Guidance & Intent, Etc.
Campaign
1
Status
Execution Feedback Determine
Status
2 Develop *
Objectives
Phased Sequenced
*: w/Measures
Objectives
3 Phase & *
Sequence
Objectives
Prioritized Sequenced
Tasks
4 Assign
Task / Activity
Assessment
Simulation
Plan
results
5 Develop
Assessment Plan
SimQL
Access to Simulations
Required Resources
Simulation
parameters
Determine
Req’mts
Resource Constraints
from JFACC PIP
7 Assess and/or
Rehearse Plan
Plan Assessment Feedback
Gio Wiederhold SimQL 14
Types of simulation services
1. Continously executing: weather prediction
– SimQL result reports best match samples
2. Execution specific to query: what-if assessment
– may require HPC power for adequate response
3. Past simulations collect results in a base: materials
– performs inter- or extra-polations to match query parameters
4. Combinations, i.e., 2. + 3.: top layer simulation using stored
partial lower level results: device performance in new setting
5. Human-in-the-loop (mediated by an agent program): SAFs
Note
• A simulation service program can be written in any language
• A simulation service must be compliant to the interface spec.
Gio Wiederhold SimQL 15
Enabling Interoperation
Databases
• serve clients via SQL by
Sharing a Model (The Schema)
A query language over the model
the SQL interface enables
• independence of
application development
DBMS technology development
reuse of infrastructure
Today
• most new systems use a
DBMS for data storage
even with less performance,
inability to handle all problems,
but enough of them well enough.
Simulations should
• serve clients via SimQL by
Sharing a Model (research q.)
A query language over the model
a SimQL interface enables
• independence of
application development
simulation technology develop’t
reuse of infrastructure
Objective
• build information systems
combining DBMS, Simulations
even with less performance,
inability to handle all problems,
but enough of them . . .
Gio Wiederhold SimQL 16
Internet requirements
• Ubiquitous access to simulations
of a wide variety of types
• Rapid response to parameter changes
– often High-Performance computation is
needed
– distributed simulations with synchronization
• Rapid Service Composition
– High bandwidth among simulations
– Access to multiple services in parallel
Gio Wiederhold SimQL 17
Use of Simulation Results
0.6
0.3
0.5
0.2
0.1
0.5
0.07
0.03
0.5
0.2
0.1
time
0.4
0.2
0.1
0.3
0.1
prob
Simulation results can be composed for
alternative Courses-of-actions
Composition should include computation
and recomputation of likelihoods
Likelihoods change as now moves forwards
and eliminates earlier alternatives.
Gio Wiederhold SimQL 18
The branches can be labeled with probabilities,
then assessed using the outcome with values
prob
value
Next period alternatives
1200
& subsequent periods
0.6
0.1 100
1266
-1086
0.4
-1220
600
0.1
500
0.1 0.3 1100 0.2
66
134
0.4
0.5
1000
0.3
200
0.2
0.3
0.2
2000
5000
1000
200
0.1
0
0.07-420
-6000
-820 0.13
-400
-3000
0
Values
past
now
future
time
Gio Wiederhold SimQL 19
Integrating data & planning support will make
our data reusable and much more valuable
A Pruned Bush
Re-assess as time
marches forward !
100
1200
1266 ?
66
?
?
600
1100
200
500
200
0
past
now
future
1000
2000
5000
1000
0
time
Spreadsheets,
other simulations,
Databases, . . .
Msgs
sensors
Gio Wiederhold SimQL 20
Even the present needs SimQL
last recorded observations
point-in-time for
situational
assessment
simple simulations
to extrapolate data
past
Not all data are current:
now
time
future
Is the delivery truck in X?
• Is the right stuff on the truck?
• Will the crew be at X?
• Will the forces be ready to accept delivery?
Gio Wiederhold SimQL 21
Recent State of SimQL Research
GUI
collect language
requirements
Test Application
wrapper
wrapper
Spreadsheets
Weather
wrapper
Civil
Gio Wiederhold SimQL 22
Engineering
SimQL research questions
• How little of the model needs to be exposed?
• How can defaults be set rationally?
• How should expected execution cost be reported?
• How should uncertainty be reported?
• Are there differences among application areas that
require different language structures?
• Are there differences among application areas that
require different language features?
• How will the language interface support effective
partitioning and distribution?
Gio Wiederhold SimQL 23
Why not use DB-like storage?
• Volume is large -– multiple future alternatives
• Value is transient
– tomorrow all values must be updated
– expect high write/read ratio
• jut the opposite of typical DB operations
Gio Wiederhold SimQL 24
Research questions for Decision Support
• How to move seamlessly from the past to the future?
• How can multiple futures be managed (indexed)?
• How can multiple futures be compared, selected?
• How should joint uncertainty be computed?
• How can the NOW point be moved automatically?
Gio Wiederhold SimQL 25
Interfaces enables
research integration
Research on
•
Multiple futures
– Uncertainty in databases
•
Probability estimation
– Expert Systems uncertainty
– Fuzzy algebra
– Prade algebra
•
Planning Models
– allocation and distribution
– reduction to current values
•
Risk management
– beta estimation
Planning Science
Distribution
Gio Wiederhold SimQL 26
Prediction as a Service
• Server is an independent contractor, defines service
• Client selects service, and specifies parameters
• Server’s success depends on value provided
•
Some form of payment received for services
x,y
Databases are a current example.
Simulations have the same potential.
Gio Wiederhold SimQL 27
Summary of SimQL
A new service for Decision Making:
• follows database paradigm
– ( by about 25 years )
• coherence in prediction
– displacement of ad-hoc practices
• seamless information integration
– single paradigm for decision makers
• simulation industry infrastructure
– investment has a potential market
– should follows database industry model:
Interfaces promote new industries
Gio Wiederhold SimQL 28
Definitions of Integration
• Data Integration: presenting data from multiple sources so that a
suite of applications* can deal with it a single (perhaps virtual)
database
• Information integration: presenting information, obtained by
processing data and metadata from multiple sources so that an
suite of application can deal with it as a coherent information
resource.
* application suite:: a set of applications
(collect, maintain, query, analyze)
that have a consistent domain model
• Application integration: Interoperating processes.
The ideal of open systems
Standards are more difficult, because of hidden semantics
Gio Wiederhold SimQL 29
Defining Application Integration ?
• Application Integration I [Hergula]: include data extracted (via
wrappers or transformers) by functions from multiple applications.
•
Application Integration II: Use a workflow model to integrate
processes into a higher level representation.
• Application Integration III: Creating new, higher-order (query, ...)
functions by combining existing application functions and made-toorder functions (in SQL, ...)
Gio Wiederhold SimQL 30