Adapters and EAI
Technology, Tools and Trends
Presenter: Atul Apte
President and CEO, iConexio Technologies Canada Inc.
Copyright iConexio Technologies Canada Inc.
Agenda
• Session 1:
– Background on software integration
– Background on adapters
– Overview of adapter technology
• Session 2:
– Overview of trends in adapter technology
– J2CA, Web Services, Semantic Adapters
– Overview of adapter development tools
• E-Fusion product architecture
• Q&A
Background On Software Integration
•
•
•
•
Software integration is the #1 IT priority today
It is also the most complex and expensive problem to solve
About 30% - 40% of all IT costs are integration-related
Approximately 1/2 of all integration costs are adapterrelated
• Adapters represent the highest incremental cost of software
integration
• Each software system that needs to be integrated requires
an adapter
Background On Software Integration
• The primary focus of software integration is
information systems (or business applications)
– The two main areas of software integration are:
• Data
• Function (Web Services)
– The two main types of software integration environments are:
• Point to Point (Application Server based)
• Brokered (Integration Platform based)
– The extended focus of software integration is business processes
• Integrating information systems with business processes
• Automating the integrated business processes
Background On Software Integration
• Components of Software Integration Platform
– Foundation components
•
•
•
•
•
Host (Application Server, Integration Server, others)
Messaging bus (JMS, others)
Object brokers (OMG)
Database engines (RDBMS, ODBMS, others)
Transaction engines (XA compliant, others)
– Value add integration components
• Transformer
– Mapping, Object aggregation, Object decomposition
• Broker
– Routing, Filters
• Adapters
Current Perspective on EAI
• EAI is still a big, complex, expensive, tedious and
repeating problem faced universally by companies
of all sizes
– Approximately 1/3 of all IT costs are EAI related
– Approximately 1/2 of all EAI costs are adapter-related
• The definition of EAI is much broader today and includes
software systems within and outside the companies
firewalls
– The term “EAI” has been changed and misunderstood with the continuing
evolution of EAI technologies
Current Perspective on EAI
• EAI is an ongoing initiative and requires strategic
thinking, strong risk management, and realistic
goal oriented implementation plans
• Industry studies have reported that the success
rates of EAI project is very low < 5%
• Existing methodologies, project management
practices & processes are not enough to deal with
the unique and specific risk management
requirements of EAI
Current Perspective on EAI
• iConexio has a strong commitment to solve one of
the toughest problems of EAI namely
development of custom/customized adapters and
lowering the total cost of ownership
• The first step is to define EAI in a clear and
simple form easy to understand, comprehend, and
implement
Definition of EAI
EAI is the process of integrating applications and
database systems into a unified software system
capable of supporting:
– On-demand data exchange
– Collaboration of shared functions
– Consolidation & Transformation of data into information
– Automating data synchronization
– Automating business process
EAI Domains
• The EAI processes are grouped into a set of
logical processes (“Domains”) which simplifies
the definition of specific EAI processes
– Application To Application Domain (A2A - Domain)
• The process of integrating two or more business applications
– Application To Information Domain (A2I - Domain)
• The process of integrating a business application with information
producers (data warehouses), information access portals, etc.
– Application To Process Domain (A2P - Domain)
• The process of integrating a business application with one or more
work flows, EDI gateways, supply chain automation networks, mobile
workforce networks
EAI Domain Reference Models
• Every EAI Domain is supported by a context
sensitive reference model that can be used to
define, design, deploy a customized EAI solution
• Each EAI domain reference model contains:
– A logical architecture that defines the components and
relationships of the EAI process
– A set of integration patterns that define the different behavior and
interactions of the EAI process
– A technology stack containing layers of technologies and standards
that enable the EAI process
Application to Application (A2A) EAI
Business Application
Function
Adapter
Transformer
Database
Adapter
Business Application
Database
XML
Data
Integration
Broker
Adapter
SOAP
Web
Services
Broker
UDDI
Function
Adapter
Application to Information (A2I) EAI
Business Application
Function
Database
Adapter
EII
Server
ETL
Information Portal
Database
Data
Warehouse
Adapter
Portal
Server
Function
Application to Process (A2P) EAI
Business Application
Business Application
Function
Function
Database
Adapter
Partner / Supplier
Systems
Database
Database
Adapter
Adapter
XML
XML / EDIFACT
Workflow
Engine
Firewall
WFMC
EDI
Gateway
Function
A2A Patterns
Data Replication Pattern
Data
Change
Detector
Data Source
Data
Extractor
Data
Filter
Data
Loader
Other Patterns
Data Synchronization (Bi-directional Data Replication)
Composite Data Extraction
Collaboration of Shared Services
Data Target
A2I Patterns
Information Creation Pattern
Data
Retriever
Data Source
Meta-Data
Creator
Info
Aggregator
Other Patterns
Extract Transform Load (ETL)
Data Staging
Info
Presenter
Information Sink
A2P Patterns
Task Delegation Pattern
Workflow
Manager
Task
Assignment
Agent
Task to
Function
Map
Other Patterns
Business Process Management
Function
Manager
Business
Application
Generic EAI Solution Architecture
Broker
Transformer
Foundation (Host, JMS, CORBA, Transaction and Database Engines)
Adapter
Adapter
Adapter
Adapter
Adapter
Information
System
Information
System
Information
System
Information
System
Information
System
Adapter Technology Status Quo
• Adapters represent the highest incremental cost of
integration
– Each applications that needs to be integrated requires a minimum
of one to three adapters (A2A, A2I, A2P)
• The average Fortune 1000 company1 has 48
applications and 14 databases for a total of 62
potential adapters
• There are few specific tools and platforms
available for development and maintenance of
adapters
1ID
C
Definition of an Adapter
• There is no Single Definition of an Adapter
• But there is a common understanding of the
adapters behavior and role
• Adapters, Connectors, Plug-ins, Web Services,
Integration Components and other proprietary
terms are used in different context to mean
“Adapter Type Of Functionality”
• Adapter definitions tend to be context sensitive
Generic Definition Of Adapter
• An Adapter is a software component which
integrates two different instances of software
systems
– Typically the software systems integrated are business applications
(information systems)
• In a brokered integration environment a business application is
integrated with the broker by using an adapter
– Adapter facilitates integration between the integrated software
systems
• Provide connectivity between the systems
• Manage the interactions between the systems
• Enforce transformation and other business rules
Background On Adapters
• The concept of adapters was first introduced by
the Enterprise Application Integration (EAI)
vendors in the later half of 1990’s
• Adapters have become the most important aspect
of software integration
– Every software that needs to be integrated requires an adapter
– The average Fortune 1000 company has 48 different business
applications and 14 different databases¹
– Adapters represent the highest incremental cost of software
integration
¹ IDC
Background On Adapters
• Until recently there were no standards for
developing and supporting adapters
– J2EE:JCA is the first system level standard designed to support
adapters on the Java Application Server
• Major vendors are now supporting J2EE:JCA in their Java application
servers
• Adapters are becoming a commodity
– System standards for adapters will result in more focus on
integration functionality
– Multiple providers of the same adapter will give users opportunity
to compare adapter features
Background On Adapters
• Generally adapters are custom developed from
scratch
– Most information systems are proprietary or heavily customized
– The average cost of developing and maintaining an adapter for
packaged applications like SAP, PeopleSoft, etc. over a five year
period is over U$ 1 Million
• The adapter market has few tools available for
developers
– The biggest challenge in creating adapter tools is the expertise and
experience required to design, develop, and maintain complex
adapters
Adapter Interactions
• An adapter use case defines a specific interaction
between the two software systems integrated by
the adapter
– An adapter use case is a sequence of one or more functional steps
• There are two types of steps in an adapter use case
– Integration function (Connection, Parsing, Transformation, etc.)
– Support function (System Logging, Error handling, Audit Trail, etc.)
– Two types of use cases
• Simple Use Case
– Involves only one adapter (two software systems)
• Complex Use Case
– Involves two or more adapters (four or more software systems)
Adapter Scenarios
• An adapter scenario is a specific instance of an
adapter use case and is bound to a specific runtime
environment
– Atomic Scenario (Derived from simple use case)
– Collaborative Scenario (Derived from complex use case)
• It is possible for adapter scenarios to change
between atomic and collaborative scenarios
depending on the runtime environment
Simple Adapter Use Case
Simple Use Case (Atomic Scenario)
API
Client
SAP
Adapter
SAP
•Check customer
•Add new order
Complex Adapter Use Case
Complex Use Case (Collaborative Scenario)
Simple Use Case
SAP
Adapter
SAP
API
API
Client
Add new
order
Siebel
Adapter
Siebel
Check
customer
Adapter Technical Architecture
Technology Stack
Client Interface
Service Gateway
Responsible for providing unidirectional or
bidirectional access to adapter services
Responsible for defining and implementing service
contracts and access control
Document Manager
Responsible for parsing, filtering, and formatting data
and documents
Transformation Manager
Responsible for data type conversion, data mapping,
object aggregation and object decomposition
Scenario Manager
Information System Manager
Host Environment Manager
Responsible for managing single/multiple adapter
scenarios involving one or more information systems
Responsible for managing connectivity to the
information systems
Responsible for managing the lifecycle and
collaborating with the operating platform
Adapter Technical Architecture
Technology Stack
Client Interface
Service Gateway
Optional Layer
Document Manager
Transformation Manager
Scenario Manager
Information System Manager
Host Environment Manager
Optional Layer
Adapter Technical Architecture
Technology Stack
Client Interface
Service Gateway
SOAP/UDDI
Document Manager
XML / XSLT
Transformation Manager
Scenario Manager
Information System Manager
Host Environment Manager
J2EE:JCA
Simplifying Adapter Complexity
• How to manage the technical complexity of
adapters?
– Use model based software development techniques
• Pre-defined software models based on integration and design patterns
will provide a robust framework for developing adapters
– Developing multiple adapters for one application
• Encapsulate fewer interfaces into one adapter instance
– Collaborate with external transformers instead of developing
transformer functionality in each adapter
Adapter Models
Analysis, Architecture and Design
Implementation
Conceptual
Patterns
Functional
Patterns
Structura
l
Patterns
Integration
Patterns
Abstract Adapter
Model
Concrete Adapter
Model
Trends In Adapter Technologies
• Adapters are more sophisticated in functionality
– Web services enable application function level collaboration
– J2EE Connector Architecture enables plug-and-play support for
adapters inside application servers
– Adapters can function in a point-to-point or broker based
integration environment
• Standard adapter models will provide a consistent
architecture and design for adapters
• Adapter tools will enable faster, cheaper, and
easier development and maintenance
Impact of Web Services
• The most significant impact of web services is the
shift in software application architecture
• Service Oriented Architecture “SOA” is becoming
the foundation for new internet based business
applications
– The concept of “Composing” business applications from existing
software services
– Web services enables development of “SOA” compliant
applications
– Adapter have an even more critical role in fulfilling “Services”
Impact of Web Services
• Where is the Synergy between adapters and web
services?
– Web services provide a standardized platform for defining,
locating, and invoking business services
• WSDL, UDDI, SOAP
– Adapter provide a standardized component mechanism for
fulfilling the web services by accessing external software systems
• J2CA
– Both web services and adapters are moving towards a common
semantic environment enabling rapid “composition”
• UDEF, RDF
What is J2EE Connector Architecture?
• J2EE Connector Architecture (J2CA) is a subspecification of J2EE
• J2CA is a system level interface for hosting and
accessing connectors to non-J2EE applications
– J2CA connectors extend the J2EE servers ability to interface with
non-J2EE applications
– Most J2EE servers support J2CA specification
• IBM WebSphere, BEA WebLogic, Jboss, Oracle AS
– J2CA compliant adapters for popular applications like SAP,
PeopleSoft, are now available in the market place
What is J2EE Connector Architecture?
• J2CA Specification (Ver 1.5) defines system
contracts for managing security, transactions, and
connections between J2EE application
components and legacy systems
• Support meta-data interfaces and bi-directional
access between adapters and legacy systems
• Message driven access to J2CA connectors is now
possible
Moving Towards Semantic Adapters
• What is a Semantic Adapter?
– A Semantic Adapter provides an interface to the semantics of the
adapter
•
•
•
•
Adapter Behavior (Services provided by the adapter)
Meta-data of the adapted systems data structures
Meta-data of the adapted systems service interfaces
Configuration properties of the adapters runtime environment
– A Semantic Adapter uses an Ontology based approach to define the
semantics
• This involves associating commonly understood meaning to the
definition of adapter properties, behavior, configuration, and
associated meta-data
Adapter Development Tools
• Fundamental Requirements
– Enable just in time adapter development and maintenance
• Customers do not purchase adapters ahead of requirement
• End points (information systems) change and therefore pre-built
adapters need customization too
– Extend the model based design (MBD) paradigm to adapters
• Model based development significantly reduces development time
and cost, and increases adapter quality
– Differentiate and support abstract adapter models and concrete
adapter models
– Enables multiple implementations of a common abstract adapter model
Adapter Development Tools
• Fundamental Requirements
– Enable adapter developers to focus on the business requirements of
integrating two software systems
• Develop and maintain platform controllers which encapsulates the
operating platform system interfaces from adapter developers
– Eliminates the burden of learning and programming system level
interfaces and standards
– E.g: J2EE:JCA controller which encapsulates J2EE:JCA system contracts
from the adapter developer
• Generate adapter framework and component shell from pre-defined
adapter models
– Ensures design consistency, reduces bugs and enhances the quality of
adapters
E-Fusion Adapter Technology
• Objectives:
– Solve the root cause of high adapter development and maintenance
costs
• Provide the best adapter development tools and runtime platform
– Define the core technology architecture for adapters
– Define new adapter standards and methodologies
• Support relevant industry standards applicable to adapters
– Develop a single adapter technology framework capable of
supporting different adapter interactions, programming languages
and operating platforms
• Reduces the total cost of ownership for adapters
Overview of E-Fusion Adapter Tool
• iConexio has developed E-Fusion, a breakthrough, patent pending technology for Adapter
development and maintenance
• E-Fusion reduces the time & cost to develop and
maintain software adapters from 41% to 75%
• Rapid progression towards holistic integration of
information systems for improved ROI
– A2A
– A2I
– A2P
Benefits of E-Fusion
• E-Fusion extends the Model Based Development
(MBD) software paradigm and applies is to
adapters
• E-Fusion provides specific platform controllers
which eliminates 90% of the server/platform
coding
• E-Fusion’s Development and Testing methodology
significantly improves adapter quality
Benefits of E-Fusion
• E-Fusion sets new standards for adapters
– First to provide developers with design and implementation
standards for the business functions supported by adapters
– First full life cycle adapter development methodology
• E-Fusion provides a Adapter Platform which
includes all of the common adapter infrastructure
• E-Fusion enables developers to concentrate on the
elements unique to each adapter project,
independent of platform
Positioning E-Fusion
• E-Fusion provides a common adapter tooling
environment across application platforms
– IBM platforms
•
•
•
•
–
–
–
–
WebSphere Application Server
WebSphere Business Integration Server (Crossworlds)
MQSeries (Integrator, Workflow, Messaging)
Mainframe and AS400
J2EE Application Servers
CORBA ORB’s
.NET Platform
Legacy Operating Systems (UNIX, Linux)
Positioning E-Fusion
• E-Fusion provides a universal adapter hub across
enterprise integration environments
– Business Integration
• Process Integration
• Application Connectivity
– Enterprise Information Portals
• Information Integration
– EAI
• Data Integration
– Web Services
• Application Collaboration
E-Fusion Product Architecture
Integrated Collaboration Environment (ICE)
Adapter Development Environment (ADE)
Extends
Adapter
Models
Adapter
Repository
Adapter
Development Tool
Generate Adapter
Component Shell
From Models
Java
IDE
Assemble & Test
Adapter
Register Components
To E-Fusion
Define Scenarios
Code
Adapter Runtime Environment (ARE)
Configure Adapters
Monitor Adapter
Adapter
Deployment Tool
Adapter
Runtime Platform
Run Adapter Scenarios
Extends
Applicatio
n
Platform
E-Fusion Adapter Technology
• Architecture:
– Adapter architecture based on the concept of functional layers
• Each layer encapsulates specific adapter functionality accessible as
well defined services
• Each layer can be implemented as an isolated/distributed component
of the adapter
– Architecture supports three types of interactions for an adapter
• Synchronization
– Keeping distributed data models in sync with each other
• Collaboration
– Enabling software systems to collaborate functions
• Delegation
– Enabling software systems to execute tasks on behalf of other systems
E-Fusion Adapter Technology
• Adapter Model
– Is a collection of one or more adapter patterns managed by the EFusion framework Integrated Collaboration Environment (ICE)
– E-Fusion supports two types of Adapter Models
• Abstract adapter model
– Represents the architecture and design of an adapter
– Adapters cannot be created directly from abstract adapter models as
there are no implementation details defined
• Concrete adapter model
– Represents the implementation reference model of an adapter
– Includes one or more run-time scenarios
– Adapters can be created from concrete adapter models
Adapter Development Environment
• Adapter Development Environment (ADE)
supports developers throughout the full lifecycle
of adapter development and maintenance
– From design to maintenance
• Design, Develop, Assemble, Configure, Deploy, Maintain
• Develop pre-built adapter models and support custom adapter models
– Enable higher efficiencies in adapter development without
changing existing development workflow and processes
• Integrate E-Fusion with leading IDE tools and extend the tools
capabilities to support adapter development
• Define new adapter engineering and development methodologies
which can be applied with generic software development tools
Adapter Runtime Environment
• Adapter Runtime Environment (ARE) supports
distributed and centralized adapter deployments
across multiple platforms
– Support dynamic distribution of adapter components across the
platforms in the adapter runtime environment
– Support centralized administration, monitoring, and debugging of
adapters at runtime
– Support dynamic language (locale) translation for multi-lingual
runtime environments
– Support dynamic component code download and side-by-side
runtime environments (production and QA)
E-Fusion Adapter Runtime Platform
Execution
Manager
Execution
Manager
Controller
Controller
Repository
E-Fusion Adapter
Runtime Platform
CPU/OS
E-Fusion Adapter
Runtime Platform
Application Server
(J2EE / CORBA / Other)
CPU/OS
Summary
• Adapters are complex pieces of software
performing critical integration tasks
• The definition of adapters continues to expand and
support new application architectures (“SOA”)
• Adapter tools are now available for enabling rapid
adapter development and deployment
• The trend is towards developing semantically and
functionally rich adapters will continue to
strengthen the role of adapters
Q&A
Thank You