Web Service and Related
Technologies
in Directory Integrator (TDI)
Lak Sri
laksri@us.ibm.com
XML with TDI –Topic 1
• XML terminology
• XML Parsers available in TDI
– Shallow and Deep options
• XML Validation
• XML and XSL/XSLT
• Creating XML
– Using Parser
– Using Scripts
– Using EMF SDO FC
DSML over HTTP with TDI – Topic 2
• HTTP protocol overview
• DSMLv2 and IdM (Identity management)
• DSML Event Handler
– Sun setting in TDI 7.0
• HTTP based Alternative for Event Handler
• Strategies to overcome bottlenecks over HTTP
– Using Fire and Forget
– Using Sequencing
– Using RMI based adapter
• Security in the HTTP world and IDI’s role in EAI
Web Service with TDI – Topic 3
•
•
•
•
•
•
•
•
Why Web Services
Web Service Concepts and Standards
Web Service Suite in TDI
Setting up Web Services Server
Setting up Web Services Client
Handling Complex Types
Beyond Transport level security
Precautions to take in WS environemnt
XML with TDI –Topic 1
• XML terminology
• XML Parsers available in TDI
– Shallow and Deep options
• XML Validation
• XML and XSL/XSLT
• Creating XML
– Using Parser
– Using Scripts
– Using EMF SDO FC
XML Support in TDI
Topic 1
Axis Easy Web Service Server Connector
Axis Easy Web Service Invoke
Axis Java-to-Soap
Invoke Soap Web Service
Axis Soap-to-Java
Complex Types Generator
Wrap Soap
TDI 6.1.1 Server
JMX Connector
SNMP Connector
SNMP Server Connector
TCP Connector
TCP Server Connector
LDAP Connector
LDAP Server Connector
Tivoli Access Manager Connector
Windows Users and Groups Connector
SAP ALE IDoc Connector
SAP R/3 Business Object Repository
SAP R/3 User Registry
SAP R/3 RFC Functional Component
Active Directory Changelog Connector v2
IBM Directory Server Changelog Connector
Netscape/iPlanet Changelog Connector
zOS LDAP Changelog Connector
BTree Connector
JDBC Connector
Properties Connector
SystemStore Connector
RDBMS Changelog Connector
AssemblyLine Connector
Server Notifications Connector
AssemblyLine Function Component
Domino Change Detection Connector
Domino Users Connector
Lotus Notes Connector
Active Correlation Technology Connector
Generic Log Adapter Connector
RAC Connector
Entry to CommonBaseEvent Function
AssemblyLine
Script Connector
Generic Java Method
Parser FC
Scripted Function Component
Timer Connector
Remote Command Line Function Component
z/OS TSO/E Command Line Function Component
Command Line Connector
Memory Queue FC
MemQ Connector
Memory Stream Connector
File System Connector
Exchange Changelog Connector
Mailbox Connector
SendEMail Function Component
TIM DSMLv2 Connector
DSMLv2 SOAP Connector
DSML v2 SOAP Server Connector
Generic JNDI Connector
ITIM Agent Connector
EMF SDOToXML Function Component
EMF XMLToSDO Function Component
FTP Client Connector
URL Connector
HTTP Client
HTTP Server Connector
IBM MQ Series Connector
JMS Pub/Sub Connector
MQe Password Store Connector
System Queue Connector
JVM on Windows, Linux, AIX, iSeries, zOS, Sun, HP
CSV Parser
DSML v1 Parser
DSML v2 Parser
Fixed Record Parser
HTTP Parser
LDIF Parser
Line Reader/Writer
SOAP Parser
Script Parser
Simple Parser
XML Parser
XML Sax Parser
XSL based XML Parser
XML Sources and Dialects
• XML Data Sources
–
–
–
–
Files System, URL, Memory, Web Service
Messages: HTTP, JMS / WebSphere MQ
Repositories : LDAP directories, RDBMS
Applications: TAMeb, FIM, web pages
• XML dialects (Markup Languages)
– Identity management
• DSML
• SPML
– Web Services
• SOAP
– Security
• SAML
XML Technology Review
XML
• eXtensible Markup Language
• Metalanguage - used to create other languages
• Has become a universal data-exchange format
• You need to ensure that XML is
– Well-Formed: Structure follows XML syntax rules
– Valid: Structure conforms to a Schema
What is a XML Node ?
• An XML node is a tree, containing an open
tag, contents, and a close tag
– <foo id=“123”>This is <bar>an
element</bar></foo>
– Here, the tag named ‘foo’ encloses the
contents and attributes of the Node
– In this case node with the tag ‘foo’ contains a
node with the tag ‘bar’ also.
There is another notion used in XML literature called Element, that is skipped here to keep things simple
XML is a Nested Tree
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<IBM>
<eFruit>
<FruitSales>
<Order>
<OrderId OrderState="ReadOnly">E0001DU9</OrderId>
<Items>
<Fruit>
<FruitName>Grapes</FruitName>
<FruitCount>12</FruitCount>
</Fruit>
<Fruit>
<FruitName>Apples</FruitName>
<FruitCount>3</FruitCount>
</Fruit>
</Items>
</Order>
</FruitSales>
</eFruit>
</IBM>
Order
OrderID
Node Depicted
items
items
OrderState
fruit
fruitname
fruit
fruitcount
Nodes
Branch nodes contain children
Leaf nodes contain content
Attributes, Values, etc.
attribute
& value
element &
content
XML is a Nested Tree - 2
Result
Order@OrderId:E0001DU9
Order@OrderId#OrderState:ReadOnly
Order@Items@Fruit@FruitName:Grapes|Apples
Order@Items@Fruit@FruitCount:12|3
XML Syntax
• Tags properly nested
• Tag names case-sensitive
• All tags must be closed
– or self-closing
– <foo/> is the same as <foo></foo>
• Attributes enclosed in quotes
• Document consists of a single (root)
element
Advantages of XML
•
•
•
•
Machine-readable (easy to parse)
Standard format for data interchange
Possible to validate
Extensible
– can represent any data
– can add new tags for new data formats
• Hierarchical structure (nesting)
• Manageable with tools
Alternatives in TDI for
Handling XML
TDI’s XML related support
• Parsers
–XML Sax Parser
–XML DOM Parser
–XSL based XML Parser
• DOM Scripting with JAXP
• system object methods [xslTransform()]
• EMF SDO FC
Using the SAX Parser
XML
Document
Driving
Code
Processing
code
(handlers)
StartElement
Characters
etc.
Parser
Using the SAX Parser -2
Using the SAX Parser -3
Using the SAX Parser -4
Using the SAX Parser -5
Using DOM XML Parser
TDI DOM Parser
Default TDI
Code performs
two levels by
default
Custom Script
to perform n
level tree
traversal
n-Node Tree
Using DOM XML Parser -2
Using DOM XML Parser -3
Using DOM XML Parser -4
Using DOM XML Parser -5
DOM vs Sax
• Data persists in
memory
• “Random access”
– can go forwards and
backwards
• Can read and write
• In-place updates
• Small memory usage
• Does not read the
whole document, just
the parts needed
• Read only
• Can parse a non-well
formed document
Using xmldom object
to Create XML
Using xmldom object -2
Using xmldom object -3
<Fruit>
<FruitName>Mango</FruitName>
<FruitCount>12</FruitCount>
</Fruit>
Validating XML
(example using SAX parser)
XML Schemas
• “Schemas” is a general term--DTDs are a form
of XML schemas
• When we say “XML Schemas,” we usually mean
the W3C XML Schema Language
– This is also known as “XML Schema Definition”
language, or XSD
• DTDs, XML Schemas, and RELAX NG are all
XML schema languages
Referring to a schema
• To refer to a DTD in an XML document, the reference goes before
the root element:
– <?xml version="1.0"?>
<!DOCTYPE rootElement SYSTEM "url">
<rootElement> ... </rootElement>
• To refer to an XML Schema in an XML document, the reference goes
in the root element:
– <?xml version="1.0"?>
<rootElement
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
(The XML Schema Instance reference is required)
xsi:noNamespaceSchemaLocation="url.xsd">
(This is where your XML Schema definition can be found)
...
</rootElement>
Sample XML with DTD and XSD Schema
XML
<?xml version="1.0"?>
…… or ….
<ste>
<presenter>Lak Sri</presenter>
<audience>All</audience>
<topic>IDI</topic>
<theme>how to</theme>
</ste>
DTD
<!ELEMENT ste (audience, presenter,
topic, theme)>
<!ELEMENT presenter (#PCDATA)>
<!ELEMENT audience (#PCDATA)>
<!ELEMENT topic (#PCDATA)>
<!ELEMENT theme (#PCDATA)>
Note: PCDATA - parsed character data
XSD
<?xml version="1.0"?>
<xs:schema
xmlns:xs="http://www.w3.org/2001/XMLSchema"
targetNamespace=http://www.ibm.com
xmlns=http://www.ibm.com
elementFormDefault="qualified">
<xs:element name=“ste">
<xs:complexType>
<xs:sequence>
<xs:element name="presenter"type="xs:string"/>
<xs:element name="audience" type="xs:string"/>
<xs:element name=“topic" type="xs:string"/>
<xs:element name=“theme" type="xs:string"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>
Using parser for doc validation -2
Using parser for doc validation -3
Transforming the XML
using XSL
With system object
Transforming the XML using XSL - 1
Transforming the XML using XSL - 2
Example Output
Transforming the XML using XSL - 3
XSL Transformation
XML Data
Transforming the XML
using XSL
XSL based XML Parser
Transforming the XML using XSL (thru config) - 4
Result
Species:
Great Tinamou.|
Highland Tinamou.|
Little Tinamou.|
Thicket Tinamou.|
Slaty-breasted
Tinamou.|
Choco Tinamou.,
Family:TINAMIDAE,
Order:TINAMIFORMES
Species:
Red-throated Loon.|
Arctic Loon.|
Pacific Loon.|
Common Loon.|
Yellow-billed Loon.,
Family:GAVIIDAE,
Order:GAVIIFORMES
EMF SDO
(SDO to XML and XML to SDO)
SDO in a nutshell
•
•
•
•
•
Self-contained object graph
Objects can be passed remotely
One API for XML and non-XML data
Generated statically-typed API can be used
DOM-like access possible
Previous TDI version: SDO Technology is not available in previous versions of TDI.
The EMF XMLToSDO and SDOToXML Function Components are also
not compatible with the TDI 6.0 Castor Function Components.
Any solution which uses the Castor Function Components needs to be
re-implemented in order to work with the EMF XMLToSDO and
EMF SDOToXML Function Components.
SDO to XML Config
SDO to XML Config Output
Step 1 – Run the HTTP Server to server the XSD files referred
Step 2 – Run the AL for SDO to XML
06:04:26 [getxml] CTGDIS505I The 'conn' object
06:04:26 [getxml] CTGDIS003I *** Start dumping Entry
06:04:26
Operation: generic
06:04:26
Entry attributes:
06:04:26
DocRoot@person#sex (replace):
'Male'
06:04:26
DocRoot@person#name (replace):
'Lak Sri'
06:04:26 [getxml] CTGDIS004I *** Finished dumping Entry
06:04:26 [getxml] CTGDIS506I The 'work' object
06:04:26 [getxml] CTGDIS003I *** Start dumping Entry
06:04:26
Operation: generic
06:04:26
Entry attributes:
06:04:26 [getxml] CTGDIS004I *** Finished dumping Entry
06:04:29 [getxml] CTGDIA012I The XML Schema has been
successfully parsed and an Ecore model has been created.
06:04:29 [getxml] CTGDIA013I The Entry Attributes have
been successfully applied to the data structure.
06:04:29 [getxml] CTGDIA018I The XML that results from
conversion from the DOM Element to XML is:
<?xml version="1.0" encoding="UTF-8"?>
<person xmlns="http://www.ibm.com" sex="Male">
<name>Lak Sri</name>
</person>.
DSML with TDI – Topic 2
•
•
•
•
HTTP protocol overview
Security in the HTTP world and IDI’s role in EAI
DSMLv2 and IdM (Identity management)
DSML Event Handler
– Removal of DSMLv2 EH in TDI 7.0
• HTTP based Alternative for Event Handler
• Strategies to overcome bottlenecks over HTTP
– Using Fire and Forget
– Using Sequencing
– Using RMI based adapter
HTTP Technology Review
As a DSMLv2 Transport protocol
HTTP Protocol - Information Flow
HTTP Request Example
GET /blog/ HTTP/1.1 􀀛
Host: ibm.com:80 􀀛
User-agent: firefox􀀛
1
Request from HTTP Client
REQUEST / RECEIVE
2
HTTP Response Example
HTTP/1.1 200 OK 􀀛
Date: Wed, 10 Oct 2007 16:07:59 GMT 􀀛
Server: WebSphere 􀀛
Connection: close 􀀛
Content-Type: text/html 􀀛
􀀛
<html>………….</html>
Example
Body
Respone from HTTP Server
LISTEN / RECEIVE / REPOND
HTTP Protocol - Methods
• GET
• Returns a representation of a resource
• Safe method – no significant side effects
• HEAD
• Returns only headers of what GET would return
• Useful for testing validity, accessibility, metadata
PUT/POST/DELETE - Subject to access rights
• PUT
• Replaces or creates a resource
• DELETE
• Removes a resource
• POST
• Submits data to a resource for any processing
• Replace or add data, manipulate other resource
HTTP Protocol and State Information
• Each new request to a server starts up a brand
new thread to response
• Remember that each HTTP request is unrelated
to any other (as far as the Web server is
concerned).
• Providing intelligent feeling of sessions requires
keeping state information (and is done via hidden fileds or
cookies)
HTTP Protocol – Headers Info
Header Contains
• Content-type
• Media type of the body entity
• Content-length
• Content-encoding
• Compression algorithm
• Last-modified
• Expires
• Info for caches
Header and Cookies
• Server gives client a cookie:
Set-Cookie: NAME=VALUE;
expires=DATE;
path=PATH;
domain=DOMAIN_NAME;
• Client can send request with cookie:
Cookie: NAME1=STRING1;
NAME2=STRING2 ...
Header and Chunked transfer-coding
• Unknown content length
• Entity headers at the end of data
• Usually for generated data
HTTP Status Codes
Success 200
• 2xx – request recognized and accepted
• 200 OK
• 202 Accepted to be processed later
• 204 No content
• 206 Partial content (on partial GET)
Bad 400
• 400 Bad, malformed request
• 401 Unauthorized
• 407 Proxy authentication required
• 404 Not found
• 410 Gone
Server Errors 500
• 500 Internal server error
• 501 Not implemented
• 503 Temporarily unavailable
• 505 HTTP version not supported
Configuring HTTP Server - 1
Configuring HTTP Server - 2
HTTP
Requests
HTTP
Response
HTTP Server
BIG PICTURE
Input Output
Http Server Returns back when
the AL Flow is complete
Configuring HTTP Server (Response in Browser) - 3
Server Log
0:12:28
10:12:28
10:12:28
10:12:28
Entry attributes:
http.status (replace):
'OK'
http.content-type (replace):'text/html'
http.body (replace):
'C:\Documents and Settings\Administrator\My
Documents\idi\index.html'
10:12:28 [httpserver] CTGDIS004I *** Finished dumping Entry
10:12:28 [httpserver] CTGDIS506I The 'work' object
10:12:28 [httpserver] CTGDIS003I *** Start dumping Entry
10:12:28
Operation: generic
10:12:28
Entry attributes:
10:12:28
http.Accept (replace):
'text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5'
10:12:28
http.base (replace):
'/'
10:12:28
http.Host (replace):
'localhost'
10:12:28
http.User-Agent (replace): 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7'
10:12:28
http.method (replace):
'GET'
10:12:28 [httpserver] CTGDIS004I *** Finished dumping Entry
Browser Result
HTTP and Security
In the context of TDI
HTTP Protocol BASIC Authentication
• When challenged, Client must be able to
respond with username/password
• 2x messages: can preemptively authenticate:
– Protection space (authentication scope): all paths at
or below current URI
– (server:port/path/*, Realm, authentication scheme
• Problem: password is clear text
Configuring HTTP Server Authentication - 1
The maximum queue length for incoming connections.
If a connection request arrives when the queue is full,
the connection will be refused.
The basic-realm sent to the client in
case authentication is requested.
Enable it
Configuring HTTP Server Authentication - 2
Script based verification of username and password
Browser will
Prompt for
password
OR
Or u will have to pass username
and password from client end
EAI authentication service for TAM with TDI
TDI
AssemblyLine ”EAI server”
Access Manager
Http connector
”any” connector
WebSeal
Policy
API
TAM
LDAP
Optional
Authentication
Connector / logic
LDAP connector
Optional TAM API
code
TDI provides a generic authentication server for TAM, where the AssemblyLine can lookup
multiple sources, as well as format and transform data. The AssemblyLine above does not
illustrate any specific scenario, but illustrates how the integration with the TAM EAI
(External Authentication Interface) service works.
HTTPS or SSL over HTTP Protocol
Short for Secure Sockets Layer is a protocol developed in 1996 by
Netscape for transmitting private documents via the Internet.
By convention, URLs that require an SSL connection start with https
Public-Key Cryptography is typically used to protect the session key
used by asymmetric encryption algorithm. The Public Key is used to
encrypt the session key (by the client), which in turn is used to
encrypt some data, and the Private Key (by the server) is used for
decryption.
Benefits
• Confirms that client and server are who they are
The domain for which the certificate was issued.
The legal owner of the certificate.
The validity dates of the certificate.
• Encrypts information sent to and from http client and server
• Information exchanged between server and client is private
and is protected from being viewed or tampered with
Step on TDI for enabling SSL over HTTP
Server Side
• Step 1. Generate a keystore for IDI server. The keystore must be in
JKS (Java Key Store) format.
• Step 2. Create a new self signed certificate for the IDI http server.
• Step 3. Extract the Server’s certificate for client use
• Step 4. Edit the global/solution.properties file.
• Step 5. Edit the file java.security
• Step 6. Change the Http Server component configuration to support
SSL connection and change port number to an appropriate
value
Client Side
• Step 1. Generate or update the trustStore for the client.
• Step 2. Enable Http client to recognize the 'https' protocol in the URL
when making a request to IDI.
What is DSMLv2 ?
Directory Services Markup
Language
DSMLv2 - Directory Services Standard
•
Directory Services Markup Language
•
A Working Group - Technical Committee
–
(specialised XML)
Microsoft, Sun, IBM, Novell, Bowstreet, Oracle, Access 360, Netscape etc.
•
DSMLv1 approved in Dec’ 1999
•
DSMLv2 approved in Nov’ 2001
– Representing LDAP Operations in XML
– Design Goals:
•
•
•
Transport Independence
Full fidelity with LDAP
NOT a superset of DSMLv1
DSMLv2 Request Example
<batchRequest onError="exit" processing="sequential"
responseOrder="sequential" xmlns="urn:oasis:names:tc:DSML:2:0:core"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<searchRequest derefAliases="neverDerefAliases"
dn="o=ibm,c=us" scope="wholeSubtree" sizeLimit="0"
timeLimit="0" typesOnly="false">
<filter>
<present name="sn"/>
</filter>
<attributes>
<attribute name="cn"/>
<attribute name="sn"/>
</attributes>
</searchRequest>
</batchRequest>
Note: This is protocol Independent, meaning it does not have to be transported
Over HTTP. But since ITIM uses it over HTTP out of the box, we will discuss it.
Example DSMLv2 Message Over SOAP
<se:Envelope xmlns:se="http://schemas.xmlsoap.org/soap/envelope/">
<se:Body xmlns="urn:oasis:names:tc:DSML:2:0:core">
<batchRequest onError="exit" processing="sequential"
responseOrder="sequential" xmlns="urn:oasis:names:tc:DSML:2:0:core"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<searchRequest derefAliases="neverDerefAliases"
dn="o=ibm,c=us" scope="wholeSubtree" sizeLimit="0"
timeLimit="0" typesOnly="false">
<filter>
<present name="sn"/>
</filter>
<attributes>
<attribute name="cn"/>
<attribute name="sn"/>
</attributes>
</searchRequest>
</batchRequest>
</se:Body>
</se:Envelope>
Note: DSMLv2 Soap binding is not used by ITIM, but is supported by many LDAP Servers
Adapter framework for ITIM (Tivoli Identity Manager)
TDI
Adapter AssemblyLines
TDI logic and
connectors
Target
system
TIM services
TIM DSML
Event Handler
TDI logic and
connectors
Target
system
TIM manages life cycle
of users in the
enterprise IT systems
TDI logic and
connectors
Target
system
DSMLv2 Event Handler is removed in TDI 7.0
Note: All event handlers are removed in TDI 7.0
From DSMLv2 EH to DSMLv2 Soap Connector Server
Setup simulatable DSMLv2 Client Operations for testing
1.
2.
3.
4.
Configure the Http client in call reply mode
Configure a DSMLv2 Parser on top
Setup the output map (sends request/call)
Setup the input map (receives reply)
DSMLv2 Soap connector Server to replace the DSMLv2 EH
5.
6.
7.
8.
9.
Configure the DSMLV2 Soap Server
Have a Switch based on DSMLv2 Operations
Create Case component as needed (including default)
Prepare Response (include AL connector if needed)
Check Client / Server by running them
Note: ITIM does not use Soap based client Binding
Transitioning over to DSMLv2 Soap Connector (Client for testing) - 1
Transitioning over to DSMLv2 Soap Connector (Client for testing) – 2,3,4
DSMLv2 Soap Server Connector (Connection Configuration) - 5
DSMLv2 Soap Server Connector (Operation Switch) – 6,7
DSMLv2 Soap Server Connector (Response Config for Search)- 8a
DSMLv2 Soap Server Connector (Response Log for Search)- 9a
Search Request XML
<?xml version="1.0" encoding="UTF-8"?>
<batchRequest xmlns="urn:oasis:names:tc:DSML:2:0:core"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<searchRequest
derefAliases="neverDerefAliases"
dn="cn=lak,o=ibm,c=us"
scope="wholeSubtree"
sizeLimit="0"
timeLimit="0"
typesOnly="false">
<filter>
<present name="sn"/>
</filter>
<attributes>
<attribute name="cn"/>
</attributes>
</searchRequest>
</batchRequest>.
Search Response XML
10:26:59 ****** [http.Content-Type:text/xml, http.body:<?xml version="1.0"
encoding="UTF-8"?>
<batchResponse xmlns="urn:oasis:names:tc:DSML:2:0:core"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<searchResponse>
<searchResultEntry dn="C=US">
<attr name="objectclass">
<value>country</value>
<value>top</value>
</attr>
<attr name="c">
<value>us</value>
</attr>
</searchResultEntry>
<searchResultEntry dn="o=ibm,C=US">
<attr name="objectClass">
<value>top</value>
<value>organization</value>
</attr>
<attr name="o">
<value>ibm</value>
</attr>
</searchResultEntry>
<searchResultEntry dn="ou=Groups,o=IBM,C=US">
<attr name="ou">
<value>Groups</value>
</attr>
<attr name="objectclass">
<value>top</value>
<value>organizationalUnit</value>
</attr>
</searchResultEntry>
………AFTER SEVERAL SUCH searchResultEntry(s)
<<searchResponse>
DSMLv2 Soap Server Connector (Response Config for Add)- 8b
DSMLv2 Soap Server Connector (Response Log for Add)- 9b
Add Request XML
<?xml version="1.0" encoding="UTF-8"?>
<batchRequest xmlns="urn:oasis:names:tc:DSML:2:0:core"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<addRequest dn="cn=lak,o=ibm,c=us">
<attr name="objectclass">
<value>person</value>
</attr>
<attr name="sn">
<value>Sri</value>
</attr>
<attr name="cn">
<value>Lak</value>
</attr>
<attr name="telephoneNumber">
<value>1234567890</value>
</attr>
</addRequest>
</batchRequest>.
Add Response XML
<?xml version="1.0" encoding="UTF-8"?>
<batchResponse
xmlns="urn:oasis:names:tc:DSML:2:0:core"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchemainstance">
<addResponse matchedDN="cn=lak,o=ibm,c=us">
<resultCode code="0" descr="success"/>
</addResponse>
</batchResponse>'
Note: The Code for Add Response on the DSMLv2 Server side does not
do anything with Add..you will need to perform add operation or whatever
Logic and respond back with success / failure etc.
Using DSMLv2 Soap Client with Microsoft Active Directory
• DSML Services for Windows (DSFW)
• Microsoft’s DSMLv2 implementation
– Implemented as a SOAP Listener
– Requires MSXML 4.0 SP1
• Runs on
– Windows 2000 Server
– Windows Server 2003
• Free download from
http://www.microsoft.com/windowsserver2003/downloads/featurepacks
Difficulties in Handling simulataneous requests
– Long-lived Server side operations
– Where blocking is not acceptable.
– Limitations on number connector that can be initialized
simulataneously
[See if connection pooling can help]
• Sometimes API based connections have limit
• Using JNI based non re-entrant connections
– Limitation in performance due to competition for CPU or memory
Managing HTTP Protocol based bottlenecks
TDI
Incoming
Http Request
Fire &
Forget
HTTP Receiving
1
Thread
TDI Thread
Or Process
Return to client
JMS
2
MemQ
3
This AssemblyLine could receive incoming HTTP messages, use that information Fire action,
Queue up in External JMS system, or sequence using then merge in data from an LDAP
directory, and finally respond back to the event originator with the requested data.
Server AssemblyLine Sequencing by Work Entry to Mem Queue
Server Service Thread 1
Wait for Task Status
for given time and
Respond
if Status 1 good
Server Service Thread N
Wait for Task Status
for given time and
Respond
if Status N good
TA
S
K
Request 1
Request N
1
N
Work
S
TA
T
U
S
Work
1
N
Mem Queue
Read Mem Queue and Process
Creating Simulataneous Requests -1
Connection used for Simulataneous Requests -2
Receiving Simulataneous Requests -3
Queuing Simulataneous Requests - 4
Setting the TAG of current Thread to Queue - 5
Ensure that the Queue to READ is running if not start it- 6
Wait for the Current Task’s Entry to be Processed and Respond- 7
Read Queue that has Multiple Task’s Entry- 8
If Read Queue completes, set Status for Server to Respond - 9
Task’s Thread is waiting (see step 7)
What we covered in Topic 2
•
•
•
•
HTTP protocol overview
Security in the HTTP world and IDI’s role in EAI
DSMLv2 and IdM (Identity management)
DSML Event Handler
– Sun setting in TDI 7.0
• HTTP based Alternative for Event Handler
• Strategies to overcome bottlenecks over HTTP
– Using Fire and Forget
– Using Sequencing
– Using RMI based adapter
Web Service with TDI – Topic 3
•
•
•
•
•
•
•
•
Why Web Services
Web Service Concepts and Standards
Web Service Suite in TDI
Setting up Web Services Server
Setting up Web Services Client
Handling Complex Types
Beyond Transport level security
Precautions to take in WS environemnt
Why are Web Services (and SOA) getting popular ?
www.ibm.com/soa
“Simple” TDI Enterprise AssemblyLine – Reads Data, Performs action
AL Component
Action Point
Connector
/ Script
/ Mapping etc.
Logic Flow
Feed
Intial Work Unit
Unit of Completed
Work
But…imagine a information driven organization where
the dependencies within and outside the enterprise are many
and is changing fast..
Possible Solution : Service Oriented Architecture
• Enterprise Logic is divided into servicable end points
supported by immediate business knowledge (Model)
• Define the service as the contract and hide rest of the
logic from the outside world (Assemble)
• Together the collection of Services adhere to a
communications agreement which is based on business
perspective (Deploy)
• Services maintain a relationship that minimizes
dependencies and maintain minimal awareness of each
other within the communications agreement (Manage)
Web Services
Technology Review
Questions that arise in distributed computing
Waiting for
requests
Communication protocol, Data format
(known location,
known port)
Sending
requests,
getting
results
• Basic questions are:
– How to transmit arbitrary data type over the wire ?
– What kind of protocol to use ?
– How do we know what services are provided and
where ?
A possible resolution
Waiting for
requests
Data in XML format
(known location,
known port)
Sending
requests,
getting
results
– Data represented in XML format
– Transport Agnostic
• HTTP is the most used, because they are firewallfriendly, but SMTP, JMS or RMI/IIOP can be used
– Allow for Services Description and Discovery
without hard coding all the functionality
SOAP – the well defined XML Exchange protocol
• Simple Object Access Protocol
•
•
•
XML based protocol for exchange
of information
– Encoding rules for datatype
instances
– Style1 to encapsulate RPC calls
using the extensibility and
flexibility of XML
– Style 2 to deliver a whole
document without any method
calls encapsulated
SOAP Message
Structure
[Header element]
Header
Entries
Body Element
Transport independent
SOAP with Attachments allow
arbitrary data to be packaged.
[Fault Element]
SOAP
Envelope
WSDL – Web Services Definition
• Web Services Definition Language
• An XML-based language for
describing Web Services
– what the service does (description)
– how to use it (method signatures)
– where to find the service
WSDL Document
Structure
[Types]
{Messages}
{Port Types}
• It does not depend on the
underlying protocol
• WSDL descriptions can be made
available from an UDDI registry
{Bindings}
{Services}
Web Services with UDDI
UDDI
Registry
2
1
Query
Requestor
Entity
3
Interact
SOAP Messages
Publish
Provider
Entity
Review of the Web Services Stack
Web Service Support in TDI
The Web Services suite consists of the following :
Core Web Service Components
• WSReceiverServerConnector
• InvokeSoapWS FC
SOAP to Work and Work to SOAP Helper Components
• AxisJavaToSoap FC
• AxisSoapToJava FC
Utility Components
• ComplexTypesGenerator FC
• WrapSoap FC
Axis Easy Components to try Web Services (Not covered in this presentation)
• AxisEasyWSServerConnector
• AxisEasyInvokeSoapWS FC
See
http://www.tdi-users.org/twiki/pub/Integrator/HowTo/WS101.pdf for more info
Using Key TDI Web Service Components
WSDL File
Work Entry
AL Operations
AxisJavaToSoap FC Request Mode
Create WSDL
WSReceiverServerConnector
SOAP Request
SOAP Request
InvokeSoapWS FC
SOAP Response
AxisSoapToJava FC Request Mode
Work
AxisSoapToJava FC Response Mode
AxisJavaToSoap FC Response Mode
Work Entry
SOAP Response
Web Services Service Provider in TDI
Step 1 – Create an AL with WebService Server component
Step 2 – Create Operations & Expose Functionality
Step 3 – Create the WSDL file
(Repeat Step 2 and 3 and fix changes to Client side if needed)
Step 4 – Define WebService Server Logic Flow
a – Return WSDL File for WSDL request
b – If Not a WSDL File process request
c – Set up Switch Case for all exposed operations
d – For each Operation, use the SOAP to Java FC and get work from SOAP
e – Process Work
f – Set Respose SOAP from work attribute via the Java to Soap FC
Step 1 – Create an AL with WebService Server component
Step 2 – Create AssemblyLine Operations including Default
Step 3 – Create the WSDL file from the Config TAB of the Web Service Server
Web Services Service Provider in TDI
Step 4 – Define WebService Server Logic Flow
a – Return WSDL File for WSDL request
b – If Not a WSDL File process request
c – Set up Switch Case for all exposed operations (and default)
d – For each Operation, use the SOAP to Java FC and get work from SOAP
e – Process Work
f – Set Respose SOAP from work attribute via the Java to Soap FC
Step 4 – Define WebService Server Logic Flow
a – Return WSDL File for WSDL request
Step 4 – Define WebService Server Logic Flow
b – If Not a WSDL File process request
Step 4 – Define WebService Server Logic Flow
c – Set up Switch Case for all exposed operations
Step 4 – Define WebService Server Logic Flow
For Operation “getFullName”
Step 4 – Define WebService Server Logic Flow
d – For each Operation, use the SOAP to Java FC
and get work from SOAP
Step 4 – Define WebService Server Logic Flow
e – Process Work
Step 4 – Define WebService Server Logic Flow
f – Set Respose SOAP from work attribute via the Java to Soap FC
Creating the Web Services
Consumer in TDI
Ensure that the AL has Work attributes set to the right values
For example an Attribute Map can be used:
Step 5 – Create SOAP Request from Work
Step 6 – Send SOAP Request and get SOAP Response via
WebService Client
Step 7 – Convert the Soap Response to Work
Step 5 – Create SOAP Request from Work
Step 6 – Send SOAP Request and get SOAP Response via WebService Client
Step 7 – Convert the Soap Response to Work
Print out the Results from the Work after calling Web Service
Testing the Web Services Provider and Consumer in TDI
Step 11 – Start the Web Service Server
Step 12 – Test WSDL file from a Browser
Step 13 – Start the Web Service Client
Step 14 – Check Java to SOAP log on the Client Side
Step 15 – Check the Call and Reply SOAP Messages
Step 16 – Check Soap to Java log on the Client Side
Step 17 – Check the Server logs for statistics
Step 18 – Check Performance of the Server
Step 11 - Start Web Service Server
Step 12 – Test WSDL file from a Browser
Step 13 - Start Web Service Client
Step 14 – Check Java to SOAP log on the Client Side (Consumer)
Step 15 – Check the Call and Reply SOAP Message
Step 16 – Check SOAP to Java log on the Client Side (Consumer)
Step 17 – Check the Server logs to review the path
ComplexTypesGenerator FC
Used for converting (especially complex types of) WSDL into Java Classes necessary for Soap Binding
Jar File
Complex type to Java Source
Archive into
Jar
Class Files
Compile
ComplexTypesGenerator FC
• The Apache Axis library (on which the TDI Web Service components
are based Axis 1.3 for TDI 6.1.1) provides a tool called WSDL2Java
• WSDL2Java takes a WSDL Web service definition to generate the
corresponding Java classes. This will generate only those bindings
necessary for the client. Axis follows the JAX-RPC specification
when generating Java client bindings from WSDL.
• Using this both a Web service client and a Web service provider for
both WS-Provisioning and WS-Trust applications have been
successfully used.
• Java JDK is needed to successfully compile the Generated Java
Classes.
Using Advanced options like Soap Parser or WrapSoap FC
var e = system.newEntry();
e.setAttribute ("soap_call", "updateIdentity");
e.setAttribute ("uid", "johnd");
e.setAttribute ("pwd", "secret");
// Retrieve the XML document as a string
var soap = system.getParser ("ibmdi.SOAP");
soap.initParser();
var soapxml = soap.getXML ( e );
main.logmsg(soapxml);
<SOAP-ENV:Envelope
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsd="http://www.w3.org/1999/XMLSchema"
xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance">
<SOAP-ENV:Body>
<updateIdentity SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<uid xsi:type="xsd:string">johnd</uid>
<pwd xsi:type="xsd:string">secret</pwd>
</updateIdentity>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
Web Service Server Mode Performance
A. Is there a limit on the number of bind requests that TDI can handle concurrently?
The global system property com.ibm.di.server.maxThreadsRunning can be
used to reduce the maximum number of threads started by the server.
This property can either be set in the global.properties or solution.properties.
B. Any suggested values for this parameter?
You can monitor the number threads by dumping before deciding optimal level for your
needs.
main.logmsg ("Number of threads: " + java.lang.Thread.activeCount());
Refer to :
http://publib.boulder.ibm.com/infocenter/tivihelp/v2r1/topic/com.ibm.IBMDI.doc_6.1.1/use
rsguide28.htm#howtocontrolthenumberofthreads
http://publib.boulder.ibm.com/infocenter/tivihelp/v2r1/topic/com.ibm.IBMDI.doc_6.1.1/use
rsguide37.htm#conpool
Web Service Server Mode Performance
In some environments, there is a high volume of incoming events and the
AssemblyLine is not fast enough to fully utilize the CPU.
In this situation, you can increase the number of threads for the
AssemblyLine by modifying the AssemblyLine pool settings.
See this url for additional details
http://publib.boulder.ibm.com/infocenter/tivihelp/v2r1/topic/com.ibm.IBMDI.doc_6.1/usersguide119.htm#alpool
Web Services Vulenerability
•
Web Services interfaces open up information to underlying systems.
•
Playback Attack (even digitally signed can be recorded and played
back). Hence consider using Timestamp, Sequence number, or even
Expiration information to handle replay attacks in the Application level.
•
Bad input data like SQL injection attack can be packaged in a SOAP
message
•
Difficulty in recognizing Application level attacks at the network level
(since traffic is often encrypted)
Web Services Security Review
•
Web Services is a Web-based technology, so it inherits all
the security benefits and risks of the Web.
•
HTTPS transport can be used to provide a very fast and
secure transport of Web services
•
Provides authentication through either HTTP Basic or
Client certificates (X.509)
•
Lastly using a Transport level security can ensure that
message is safe between point to point, and cannot
ensure saftey from end to end. ( Can be avoided by
Message level security : WS-Security – Refer to TFIM).
What we Covered in Topic 3
•
•
•
•
•
•
•
•
Why Web Services
Web Service Concepts and Standards
Web Service Suite in TDI
Setting up Web Services Server
Setting up Web Services Client
Handling Complex Types
Beyond Transport level security
Precautions to take in WS environemnt