handling complexity (mess?)
integration or federation
Stephen Todd
IBM WebSphere MQ
e-Science Institute: Edinburgh
14 October 2003
The opinions expressed here are those of the author
and do not necessarily represent those of IBM.
outline
what are the difficulties facing
• our customers?
• the industry?
how should we address these difficulties
• integration?
• federation?
?
customer difficulties
lots of departments
• every customer address stored 5 times
•in 5 different technologies
• don't even know if they are the same customer
mergers and acquisitions
• complexity - scale - heterogeneity
i.e. .....
complexity
clean complexity
bomb
• quantum theory
• non first normal form
dirty complexity
• islands of automation
• heritage applications and systems
(smart complexity?)
• (autonomics?)
the industry has a solution
let us sell you our
• magic middleware
–database system
–application server
–messaging system
• application solution
even for legacy
we can even wrap your old one
• eg relational front end to an IMS database
"It's easy to put a
relational front end on a
pure IMS database
~~~~
at least, it would be if
there were any."
dirty
complexity
we can all grow with your needs
1
3
2
4
and the result is
different dirty complexity
luckily, we have a solution
let us sell you our systems management
system
messaging
system
database
application
server
so ....
can't you give us a more integrated solution?
but ...
but ... middleware religion
corporate directive
•
•
•
•
databases are ...
application servers are ...
messaging system is ...
(no MS software, but 1000 VB programmers)
"We can't install your messaging system if it requires DB2 -even if it is hidden.
Corporate directive is Oracle."
complexity and contradiction
so, what are our problems
when providing middleware to
help?
messaging
system
database
application
server
our own dirty complexity
many overlapping solutions
• integrated islands
• heritage products
how many transaction coordinators?
how many databases?
• and even more persistent stores...
messaging
system
database
application
server
product growth example: MQ
'simple' point-to-point messaging/queuing
• reliable, heterogeneous
resource manager not database because ...
transaction coordinator not external because ...
publish/subscribe
broker
• message semantics and dictionary not schema because ...
• transformations not SQL because ...
• database interaction
-with many databases so no integration ...
• almost an application server but not because ...
so, potential for integration
common tooling
common systems administration
common data and programming model
etc etc
messaging
system
database
application
server
database
application server
integration potential
least affinity ~~ impedance mismatch
subsumption, not integration
• even back to CICS, IMS
DB subsumes application server
• stored procedures & UDFs make DB an app server
applications subsume database
• programming persistence or object DB
–removes need for (explicit) DB
–but loses much DB modelling and query power?
integration potential
application servers
messaging
increased 'active' component in messaging
need for wider reach in app server
• more heterogeneity
• wider geographies
–implies distributed, async
–linked transaction model
integration potential
database / messaging
low level
• persistence, resource management, transactions
high level
• transformations, data models, streams
data placement and replication
relation
input stream
result stream
integration potential
same messages, same pictures
the data you want
• where you want it
• when you want it
• in the form you want it
but should we integrate, or federate, or ...?
integration
• cleaner models
• easier administration
federation
• heterogeneity
• choice
• handle dirty complexity
Can componentization give us the best of both?
How big must the components be?
How interdependent?
What does the future hold?
Will it change anything fundamentally?
WebServices
• same technology, another name
• very strong federation credentials
•(how widely will it really work)
Grid
• ??? ### ???
Aspect programming
Pickled chocolates
so, to summarize
big, horrid monsters
• dirty complexity
what's the solution?
• face our customers
• face the industry
 (We know how to draw the picture)
• integration
• federation
or ....
brand solution
customers want integration
but it's impossible in the real world
so rebrand federation as integration
• and give them what they want
• AND what they need