ICS 123
Software Engineering Challenges
Web Services and SOAP
Richard Taylor & Eric Dashofy
UC Irvine
http://www.isr.uci.edu/classes/ics123s02/
Topic 16
Web Services and SOAP
Traditional Software Systems
• are developed by a single organization
• undergo a phased development process
– with multiple phases of review, inspection, testing
• have a synchronized release schedule
• have a proprietary design and proprietary
component interfaces
• have a monolithic code base
• go through a painful evolution
2
ICS 123
Topic 16
Web Services and SOAP
Our Vision
The future of
software development
requires integrating
network services
that are very far away
and owned by strangers
Rohit Khare
16 May 2002
O’Reilly Emerging Technology Conf
UC Irvine & KnowNow, Inc.
3
ICS 123
Topic 16
Web Services and SOAP
Three Elements of Our Vision
ICS 123
• The abstract unit of software is a network service
– Prior levels of abstractions were subroutines, libraries, processes, objects, …
– In this view, the only way to understand software is to watch its message
traffic
• Latency is an absolute limit of system architecture
– Just as “real” architects grapple with gravity, so with the speed of light
– London will always be 30ms from New York, regardless of Moore’s Law
• Decentralization means crossing agency boundaries
– The essential complication above and beyond distributed (parallel) computing
is extending a system to achieve consensus between several separate
agencies
– Every computing and communication device is owned by someone.
4
How Our Vision is Emerging
Today
Topic 16
Web Services and SOAP
ICS 123
• The technological shift from a component model to
a service model of software is already underway
– SOAP and Web Services technologies make it easy to write new
protocols
• The Internet is inexorably shifting to higher-latency,
decentralized connectivity
– Ad-hoc, peer-to-peer networking, mainly wireless, is being deployed at
every scale from one meter (handhelds) to billions of meters (space
probes)
• Enterprise Application Integration (EAI) is only now
reaching ‘outside the firewall’ to business partners
– The next revolution in productivity requires reengineering processes
end-to-end
5
Topic 16
Web Services and SOAP
So What’s This Talk About?
ICS 123
• There is a ‘missing link’ between the near-future
•
predicted by today’s Web Services hype and
realizing the true potential of our vision
It is a new architectural style for software
integration we call Application-Layer
Internetworking (ALIN)
– ALIN applies networking concepts to software engineering
• The primary way that developers will use ALIN is an
unheralded implication of SOAP 1.2: Routing
• … so this talk is about a new style and a new device
6
What’s EAI?
Topic 16
Web Services and SOAP
ICS 123
1. Enterprise Application Integration is the unrestricted
sharing of data and business processes throughout
the networked applications or data sources in an
organization
2. The Linking of Data, Business Processes,
Applications that result in a common virtual system
7
What’s EAI?
Topic 16
Web Services and SOAP
ICS 123
• Enterprise Application Integration is real-time integration of the information
systems within an organization that support the different steps of a business
process
• Consider the following typical business process
• An order might be captured by:
•A CSR using an order entry system
•A consumer using a web application
•A business partner system using a web service interface
• Regardless of the channel, the order must be financially and operational
processed:
•The financial system processes the billing
•The operational system processes the manufacturing, scheduling,
provisioning, clearance, etc. depending upon industry.
8
Topic 16
Web Services and SOAP
Without EAI
ICS 123
AR
Internal
Credit
AP
External
Logistics
External
ECommerce
Inventory
9
•
Numerous Point-to-Point Interfaces
•
Manual Hand-Offs
•
Web Applications Increase the Number of
Enterprise Integration Points
•
Inconsistent Business Processes
•
Inconsistent Qualities of Service
Without EAI
• Numerous Point-to-Point Interfaces
• Manual Hand-Offs
• Web Applications Increase the Number of
Enterprise Integration Points
• Inconsistent Business Processes
• Inconsistent Qualities of Service
10
Topic 16
Web Services and SOAP
ICS 123
Topic 16
Web Services and SOAP
With EAI
ICS 123
EAI
Infrastructure
AR
Business
Process
Automation
Internal
Portal
Credit
AP
Data
Transformation
Logistics
External
Message
Routing
Web
Service
External
Adapter
11
ECommerce
Inventory
Message
Storage
With EAI
• Reduced Number of Common, Consistent,
Known Interfaces
Topic 16
Web Services and SOAP
ICS 123
• Consistently Automated Business Processes
• Increased Quality of Service (e.g. Order can Still be
Captured even when Financial and Operational
systems are Unavailable)
• Framework for Integration of Web Services
12
Components in an EAI Solution
Topic 16
Web Services and SOAP
ICS 123
EAI Infrastructure
Business Process
Automation
Data Transformation
Message Routing
Message Storage
Adapter
13
Adapter
•
Adapters
• Also known as “Connectors”, “Translators”,
“Wrappers”
• Provide seamless connectivity to the undrelying
application or data
• Convert information and events into data that can
be utilized by the infrastructure
• Two way translation between the proprietary lingo
and the common lingo
• Many adapters available off the shelf from EAI
Software vendors such as Tibco, or Packaged
Software vendors such as SAP.
Components in an EAI Solution
Topic 16
Web Services and SOAP
ICS 123
EAI Infrastructure
Business Process
Automation
Data Transformation
•
• An intelligent broker that directs the flow of messages between
sources and consumers of information
•
Message-Oriented Middleware (MOM)
• A set of products that connect applications running on different
systems by sending and receiving application data as messages
•
Message Routing
Message Storage
Message Broker
Message Routing
• Set of tools that route messages between sources and
consumers based on pre-defined business rules
•
Message Storage
• Also identified as “Message Warehousing”
• Central repository for temporary storage of transactions
•
Adapter
• asynchronous communications style and provides a loosely
coupled exchange across multiple operating systems
Adapter
•
14
Message Queuing
Publish Subscribe
Components in an EAI Solution
Topic 16
Web Services and SOAP
ICS 123
EAI Infrastructure
Business Process
Automation
•
Data Transformers
• Mapping of data elements to achieve common
denominator
Data Transformation
Message Routing
• Transformation to and from Formats Used by
Different Systems
Message Storage
• GUI Configuration
• Handles simple or complex data structures
Adapter
15
Adapter
Components in an EAI Solution
Topic 16
Web Services and SOAP
ICS 123
EAI Infrastructure
•
• Often GUI based
• Describing the flow of information in the context of
business processes
• Using the input/output of processes as the Integration
points between diverse business processes
Business Process
Automation
Data Transformation
Message Routing
•
•
16
Adapter
Process Brokering
• Execution of discrete steps within a business process
• Ability to recover from failed steps
• Workflow engine
Message Storage
Adapter
Process Modeling
Process Management
• Monitor business processes
• Correlate metrics to specific business process steps
EAI Power Tools
Topic 16
Web Services and SOAP
ICS 123
• Many vendors have off the shelf connector technology that
allows connectivity and integration of existing software
packages and data stores.
• GUI Transformation tools allow for visual creation and
maintenance and versioning of data transformations.
• Many MOMs (Message Oriented Middleware) already
support automatic SOAP transformations
• Many software packages are committed to SOAP as a
standard which would make the EAI piece transparent
17
Topic 16
Web Services and SOAP
ICS 123
EAI: The Next Level
Business Webs
While EAI focuses on connecting data and applications
internally, Business Webs focus on connecting applications
and business flows across organizational boundries
18
Current Condition
Distribution/Warehousing
Raw Materials
Transportation
Manufacturing
• Paperwork intensive
• Human Intervention
• Inherent inefficiencies
The Challenge
The Future
Distribution
Channels
1000’s
100’s
10’s
1
Your
Close
Affiliates
Your
Company
1
10’s
100’s
1000’s
10,000’s
Relationships Are Dynamic.
Topic 16
Web Services and SOAP
ICS 123
Time & Cost
Each eBusiness relationship is unique.
Every eBusiness initiative integrates disparate systems.
Every change requires a skilled programmer.
1
21
5
20
100
# of unique eBusiness relationships
1,000
What are Web Services?
Topic 16
Web Services and SOAP
ICS 123
• A Web service is a collection of functions that are packaged
as a single entity and published to the network for use by
other programs.
• Web services are building blocks for creating open
distributed systems, and allow companies and individuals to
quickly and cheaply make their digital assets available
worldwide.
• A Web service can aggregate other Web services to provide
a higher-level set of features
22
What are Web Services?
Topic 16
Web Services and SOAP
ICS 123
• Based on UDDI (Universal Description, Discovery and
Integration) and XML
• Platform-independent, open framework for describing
services, discovering businesses, and integrating business
services.
23
Web Services Strengths
• Interoperability
o
Topic 16
Web Services and SOAP
ICS 123
Based on open standards, utilizes existing infrastructure
• Ubiquity
o
Communicates through XML/HTTP – Any system that supports
these standards, supports Web Services
• Low barrier to entry
o
Concepts easy to understand, easy to implement
o
Toolkits allow COM, J2EE, and CORBA components to be
exposed as Web Services
• Industry Support
24
Web Services Layers
Topic 16
Web Services and SOAP
ICS 123
Publish, Find, Use Services: UDDI
25
Service Interactions:
SOAP
Universal Data Format:
XML
Communications Layer:
HTTP
SOAP- Definition
• Simple Object Access Protocol
Topic 16
Web Services and SOAP
ICS 123
• W3C standard: Not proprietary to any one software vendor
• Like CORBA and COM, designed as an interoperability
protocol that allows objects to “talk” to each other
• SOAP defines a simple mechanism for expressing
application semantics by providing a modular packaging
model and encoding mechanisms for encoding data within
modules
26
SOAP- Structure
Topic 16
Web Services and SOAP
• Objects can be functions, programs, or systems
ICS 123
• Three parts to SOAP:
• Envelope: Defines a framework for describing what is in a
message and how to process it,
• Encoding rules: Expressing instances of application-defined
datatypes,
• RPC Representation: Convention for representing remote
procedure calls and responses.
27
Topic 16
Web Services and SOAP
SOAP Example
• Example 1: Sample SOAP Message embedded in an HTTP Request
POST /StockQuote HTTP/1.1
Host: www.stockquoteserver.com
Content-Type: text/xml; charset="utf-8"
Content-Length: nnnn
SOAPAction: "http://example.org/2001/06/quotes"
<env:Envelope xmlns:env="http://www.w3.org/2001/06/soap-envelope" >
<env:Body>
<m:GetLastTradePrice
env:encodingStyle="http://www.w3.org/2001/06/soap-encoding"
xmlns:m="http://example.org/2001/06/quotes">
<symbol>DIS</symbol>
</m:GetLastTradePrice>
</env:Body>
</env:Envelope>
28
ICS 123
Topic 16
Web Services and SOAP
Example, continued
ICS 123
• Example 2 shows the SOAP message sent by the StockQuote service in the
corresponding HTTP response to the request from Example 1.
HTTP/1.1 200 OK
Content-Type: text/xml; charset="utf-8"
Content-Length: nnnn
<env:Envelope xmlns:env="http://www.w3.org/2001/06/soap-envelope" >
<env:Body>
<m:GetLastTradePriceResponse
env:encodingStyle="http://www.w3.org/2001/06/soap-encoding"
xmlns:m="http://example.org/2001/06/quotes">
<Price>34.5</Price>
</m:GetLastTradePriceResponse>
</env:Body>
</env:Envelope>
29
SOAP- Advantage
Topic 16
Web Services and SOAP
ICS 123
• Once applications are wrapped with SOAP – they are in
effect a web service
• A web service can be invoked directly through an HTTP call
• A web service can be “discovered” utilizing a directory
service
30
What’s underneath SOAP?
Topic 16
Web Services and SOAP
ICS 123
• Soap is the abstraction layer – not the integration tool
• EAI techniques are used to connect between the SOAP
wrapper and legacy systems
• Once EAI objects are wrapped in SOAP they can easily
communicate with each other thus creating a cross
enterprise integration architecture
31
Web Services- Layers
Topic 16
Web Services and SOAP
ICS 123
Publish, Find, Use Services: UDDI
32
Service Interactions:
SOAP
Universal Data Format:
XML
Communications Layer:
HTTP
UDDI
Topic 16
Web Services and SOAP
• Universal Description, Discovery and Integration
ICS 123
• Platform-independent, open framework for describing
services, discovering businesses, and integrating business
services.
• A layer above SOAP, it is the glue that connects all the
SOAP objects into a Business Web
• DNS like model implementation of the directory
33
Topic 16
Web Services and SOAP
UDDI
ICS 123
4.
SW companies, standards
bodies, and programmers
populate the registry with
descriptions of different types
of services
1.
Marketplaces, search
engines, and business apps
query the registry to
discover services at other
companies
2.
UDDI Business Registry
5.
Businesses
populate
the registry with
descriptions of
the services they
support
34
Business
Registrations
3.
Service Type
Registrations
UBR assigns a programmatically unique
identifier to each service and business
registration
Business uses this
data to facilitate easier
integration with each
other over the Web
Registry Data
Topic 16
Web Services and SOAP
ICS 123
• Businesses register public
information
about themselves
• Standards bodies,
Programmers, Businesses
register information
about their Service Types
35
Topic 16
Web Services and SOAP
ICS 123
• Business name
• Text description
– List of multi-language text strings
• Contact info
– Names, phone numbers,
fax numbers, web sites…
• Known identifiers
– List of identifiers that a business may be known by - DUNS,
Thomas, other
36
Topic 16
Web Services and SOAP
ICS 123
• Business categories
– 3 standard taxonomies in V1
» Industry: NAICS (Industry codes - US Govt.)
» Product/Services: UN/SPSC (ECMA)
» Location: Geographical taxonomy
– Implemented as name-value pairs to allow any valid taxonomy identifier
to be attached to the business white page
37
Topic 16
Web Services and SOAP
ICS 123
• New set of information businesses use to
describe how to “do e-commerce” with them
– Nested model
» Business processes
» Service descriptions
» Binding information
– Programming/platform/
implementation agnostic
– Services can also be categorized
38
Service Type Registration
Topic 16
Web Services and SOAP
ICS 123
• Pointer to the namespace where service type is
described
– What programmers read to understand how to use the service
• Identifier for who published the service
• Identifier for the service type registration
– called a tModelKey
– Used as a signature by web sites that implement those services
39
Business Registration
• XML document
• Created by end-user
company (or on their
behalf)
• Can have multiple service
listings
• Can have multiple
taxonomy listings
ICS 123
businessEntity
businessKey
name
URL
description
contacts
businessServices
identifierBag
categoryBag
keyedReference
keyedReference
tModelKey
tModelKey
keyName
keyName
keyValue
keyValue
40
Topic 16
Web Services and SOAP
Contact
Contact
Phone
Phone
Address
Address
Email
Email
businessService
businessService
serviceKey
Key
tModelKey
Name
Name
Description
Description
BindingTemplates
BindingTemplates
keyedReference
keyedReference
tModelKey
tModelKey
keyName
keyName
keyValue
keyValue
Topic 16
Web Services and SOAP
Talk Overview
• What’s really new about Web Services
• What a SOAP Router is
• What a SOAP Router can do for you
• How an SOAP Router works
• ALIN in theory
• ALIN in practice
44
ICS 123
Topic 16
Web Services and SOAP
Web Services are NOT New
ICS 123
• There is nothing new under the sun in distributed
(or parallel , or grid, …) computing
– These are all efforts to lash many systems together to act as one
• SOAP today is a distributed computing technology
– ‘Sprinkling angle-bracket pixie dust’ over plain ol’ client/server RPC
• Sure, it works across the public Internet, but only
by treating it as a very slow LAN
– Trade rags already lament the ‘gap’ at this boundary: real security,
reliability, and performance problems reveal the fallacy of assuming
TCP/IP ‘just works’
45
Topic 16
Web Services and SOAP
Web Services ARE new
ICS 123
• Decentralization does pose brand-new challenges
• Latency
– Services are no longer mere milliseconds away.
– We need middleware for seconds up to weeks
• Agency
– Services can be owned by other organizations
– We need middleware for translating semantics
• Because SOAP messages are self-contained, they
can be stored, analyzed, and replayed at will
46
SOAP is Simpler Than You
Think
Topic 16
Web Services and SOAP
ICS 123
• Starts w/point-to-point, synchronous invocations
– just as Web page servers are accessed today
• Main virtue is that it is not DCOM, IIOP, or RMI
– Encapsulates the messy details of cross-platform, cross-language
binding using standard marshalling formats
• Getting started requires very little re-education.
– On the other hand, straight RPC message exchange patterns (MEPs)
don’t inherently reduce the tight coupling between client and server
semantics.
47
SOAP is Cooler Than You
Think:
Topic 16
Web Services and SOAP
ICS 123
• The vastly underestimated power of SOAP is that its
model is genuinely asynchronous
– Today, we use it as a form of XML RPC
– Underneath, that’s merely a pair 1-way messages
• Furthermore, SOAP Actors are not hosts
– An actor is a much more general concept than a fixed service provider
machine and port
– One actor may be a proxy for many others
– Ideally, actors represent actual business models
48
Topic 16
Web Services and SOAP
What is ‘SOAP Routing’?
ICS 123
• Today, the canonical Web Services diagram directly
connects SOAP endpoints
• There will also be a complementary technology that
sits in the middle:
– Store-and-forward reliable messaging
– Filtering, transformation, & analysis of flows
– Intelligent “active proxy chaining” of actors
49
What Good is ‘SOAP
Routing’? 1/2
Topic 16
Web Services and SOAP
ICS 123
• Internet-scale Middleware
– Just as distributed objects needed message-oriented-middleware
(MOM)…
– … decentralized services will need mediators
» Wrappers: SOAP brokers to legacy systems
» Monitors: SOAP transaction correlation & audit
» Queues: decouples SOAP provider from requestor
» Pub/Sub: decouples SOAP actors entirely
• The difference is coping with firewalls and other
kinds of agency boundaries
50
What Good is ‘SOAP
Routing’? 2/2
Topic 16
Web Services and SOAP
ICS 123
• Add new ilities without modifying either end.
–
–
–
–
–
–
Security: Router can authenticate and authorize
Reliability: Router can store-and-forward msgs
Availability: Router can redirect to live servers
Scalability: Router can redistribute messages
Interoperability: Router can transform msgs
Interactivity: Router can update msgs ‘in-flight’
• Routing lets us reason about a “network of
services” -- not just pipelines, but graphs
51
How ‘SOAP Routing’ Works
1/2
Topic 16
Web Services and SOAP
ICS 123
• Consider a simple Bank ATM service
– In the old world, you would have fine-grained account objects and
setUserName() functions
– As Web Services, you have more flexible XML-document services like
getStatement()
• A client-side ATM program can easily call it directly
today by composing a SOAP message and POSTing
it to the Bank server
52
How ‘SOAP Routing’ Works
2/2
Topic 16
Web Services and SOAP
ICS 123
• First, it provides ‘simple middleware’
– Recall how CGI scripts directly invoked scripts.
– Later, Application Servers emerged to ‘encapsulate’ the effort of storing
state &c
• By passing the message through a SOAP router:
–
–
–
–
53
It can enforce the Bank’s security policy
It can store a copy of the message for audit logs
It can redirect requests across a server farm
…This is exactly what Web proxies do today
How ‘SOAP Routing’ Helps
1/2
Topic 16
Web Services and SOAP
ICS 123
• Second, it affords flexibility of connections
• What if you wanted to change banks?
– The router can filter the captured message stream so that messages
from the new bank ‘look like’ the old bank
• What if you wanted to change currencies?
– This is equivalent to routing ‘through’ another actor
– The router can transform messages;
with plugin scripts today, XML schemas tomorrow
• What if you wanted to know if cash is running out?
– The router can analyze messages for trends
54
How ‘SOAP Routing’ Helps
2/2
Topic 16
Web Services and SOAP
ICS 123
• Third, it allows 3rd & 4th-party extensions
• What if you wanted to check on your bonds, too?
– The router can multicast the balance inquiry to the Bank service and a
Broker service
• What if you wanted to buy more bonds?
– The router can monitor the joint transaction of a money transfer to
the broker and the purchase of the bonds
• What if you wanted your limit kids’ access?
– The router can delegate access control rights
55
Topic 16
Web Services and SOAP
ALIN in theory 1/3
ICS 123
• ALIN is NOT merely Actor intermediation (piping)
– Actor chains today are like UUCP !path addressing; it’s source-routing
only
– It’s the ability to infer semantic chains, decouple political control, etc.
• SOAP Routers directly address the twin novel
challenges of decentralized service integration:
– Latency
» by decomposing every communication into a one-way,
asynchronous message, routers force us to confront extreme
latency, rather than hiding it
– Agency
» by introducing a trusted 3rd-party between the original endpoints,
routers allow us to represent the interests of many other 4th-party
agencies
56
Topic 16
Web Services and SOAP
ALIN in theory 2/3
ICS 123
• The Internet itself is a result of addressing high
latency and diverse agency at Layer 3
– IP Routers store-and-forward packets across autonomous systems,
each containing many LANs
– IP-format packets are a generic interchange for LANs
• Consequently, “Internet-scale” services must:
– Scale Across Time: Interfaces must be stable for decades
– Scale Across Space: Protocols must handle >100ms delays
– Scale Across Organizations: Namespaces must ‘peer’
• This is merely an application-layer router for *TPs
57
Topic 16
Web Services and SOAP
ALIN in theory 3/3
ICS 123
• This architectural style builds on REST & C2
– Beyond merely transferring representation, now we can mediate them
– Can connect any graph, not just a strict lattice
• It generalizes other MOM styles
– Queues, sync/async, 1-1, N-N, transactional
– Pub/Sub, address-based and content-based
• The most contentious aspect of ALIN for MOM folks
is “best-effort”: it is still difficult to sell TCP over IP
– If your ‘bottom turtle’ is already transactional, it rules out scalability
and perf
– The very definition of agency boundary is that transactions can’t span
them!
58
Topic 16
Web Services and SOAP
ALIN in practice
ICS 123
• Lots of ink is being spilled on ‘missing infrastructure for Web
Services’
– Major software platform vendors are only beginning to market RPC-style Web
Services
• Existing middleware products are having difficulty “outside
the firewall”
– JMS, for example, is single-vendor, single-language
– High-performance EAI tools often assume multicast IP
• Early Internet-scale messaging startups include KnowNow,
Bang, Kenamea, SonicXQ, Grand Central, and others
• Microsoft has outlined its WS-Routing proposal
• IBM has Business Rule Beans and MQ Event Broker in WS5.0
59
Topic 16
Web Services and SOAP
Distributed vs. Decentralized
ICS 123
• We already have lots of techniques for decomposing
systems to run on many distributed processors
acting as one
• The genuinely novel potential of Web Services is
decentralization
– Marketplaces, rather than central brokers
• Routing is a new technique for integrating services
provided by 3rd and 4th parties
60
Topic 16
Web Services and SOAP
A Parting Thought
• The dirty little secret is that
the 7-Layer ISO Network
model is insufficient to
model decentralized
computing.
• It’s really all about
integration at Layer 8 & 9
ICS 123
Political
Economic
Application
Presentation
Session
Transport
Network
Link
Physical
61
You Are Here