handling complexity (mess?)
integration or federation
Stephen Todd
IBM WebSphere MQ
e-Science Institute: Edinburgh
14 October 2003
The opinions expressed here are those of the author
and do not necessarily represent those of IBM.
outline
ƒwhat are the difficulties facing
• our customers?
• the industry?
ƒhow should we address these difficulties
• integration?
• federation?
?
customer difficulties
ƒlots of departments
• every customer address stored 5 times
•in 5 different technologies
• don't even know if they are the same customer
ƒmergers and acquisitions
• complexity - scale - heterogeneity
i.e. .....
complexity
ƒclean complexity
bomb
• quantum theory
• non first normal form
ƒdirty complexity
• islands of automation
• heritage applications and systems
ƒ(smart complexity?)
• (autonomics?)
the industry has a solution
ƒlet us sell you our
• magic middleware
–database system
–application server
–messaging system
• application solution
even for legacy
ƒwe can even wrap your old one
• eg relational front end to an IMS database
"It's easy to put a
relational front end on a
pure IMS database
~~~~
at least, it would be if
there were any."
dirty
complexity
we can all grow with your needs
1
3
2
4
DB2
Oracle
S
y
b
a
s
e
IMS
and the result is
p
p
a
e
r
MQ
e
h
r
p
e
SR rv
b
e
e
e
nd
s
W
ezv
oSu
C s
I
C
MS
c
MQ
i
g
o
L
b
e
W
different dirty complexity
luckily, we have a solution
ƒlet us sell you our systems management
system
messaging
system
database
syste
ms
m a na
ge m e
nt
syste
m
application
server
so ....
ƒcan't you give us a more integrated solution?
ƒbut ...
but ... middleware religion
ƒcorporate directive
•
•
•
•
databases are ...
application servers are ...
messaging system is ...
(no MS software, but 1000 VB programmers)
"We can't install your messaging system if it requires DB2 -even if it is hidden.
Corporate directive is Oracle."
complexity and contradiction
so, what are our problems
when providing middleware to
help?
messaging
system
database
syste
ms
m a na
ge m e
nt
syste
m
application
server
our own dirty complexity
ƒmany overlapping solutions
• integrated islands
• heritage products
ƒhow many transaction coordinators?
ƒhow many databases?
• and even more persistent stores...
database
messaging
system
syste
ms
mana
geme
nt
syste
m
application
server
product growth example: MQ
ƒ'simple' point-to-point messaging/queuing
• reliable, heterogeneous
ƒresource manager not database because ...
ƒtransaction coordinator not external because ...
ƒpublish/subscribe
ƒbroker
• message semantics and dictionary not schema because ...
• transformations not SQL because ...
• database interaction
-with many databases so no integration ...
• almost an application server but not because ...
so, potential for integration
ƒcommon tooling
ƒcommon systems administration
ƒcommon data and programming model
ƒetc etc
messaging
system
database
application
server
database
application server
integration potential
ƒleast affinity ~~ impedance mismatch
ƒsubsumption, not integration
• even back to CICS, IMS
ƒDB subsumes application server
• stored procedures & UDFs make DB an app server
ƒapplications subsume database
• programming persistence or object DB
–removes need for (explicit) DB
–but loses much DB modelling and query power?
integration potential
application servers
messaging
ƒincreased 'active' component in messaging
ƒneed for wider reach in app server
• more heterogeneity
• wider geographies
–implies distributed, async
–linked transaction model
integration potential
database / messaging
ƒlow level
• persistence, resource management, transactions
ƒhigh level
• transformations, data models, streams
ƒdata placement and replication
relation
input stream
result stream
integration potential
same messages, same pictures
ƒthe data you want
• where you want it
• when you want it
• in the form you want it
but should we integrate, or federate, or ...?
ƒintegration
• cleaner models
• easier administration
ƒfederation
• heterogeneity
• choice
• handle dirty complexity
Can componentization give us the best of both?
How big must the components be?
How interdependent?
What does the future hold?
Will it change anything fundamentally?
ƒWebServices
• same technology, another name
• very strong federation credentials
•(how widely will it really work)
ƒGrid
• ??? ### ???
ƒAspect programming
ƒPickled chocolates
so, to summarize
ƒbig, horrid monsters
• dirty complexity
ƒwhat's the solution?
• face our customers
• face the industry
ƒ (We know how to draw the picture)
• integration
• federation
ƒor ....
brand solution
ƒcustomers want integration
ƒbut it's impossible in the real world
ƒso rebrand federation as integration
• and give them what they want
• AND what they need