Front cover
Managed File Transfer for SOA using IBM WebSphere MQ File Transfer Edition
Redguides
for Business Leaders
Martin Keen
David Ward
Ran Gu
Eugene Kuehlthau
Leonard McWilliams
Discover the benefits of managed file
transfer
Learn how IBM WebSphere MQ File Transfer
Edition can help to solve business problems
Integrate file transfer across
heterogeneous platforms and systems
Executive overview
For many organizations, the exchange of files between business systems remains a common
and important integration methodology. Files are the simplest unit of data to exchange and
often represent the lowest common denominator to connect disparate systems for an
enterprise infrastructure.
A managed file transfer system introduces control, management, and auditabilty to address
problems that arise when ad-hoc file transfers are used to integrate or connect business
systems in the enterprise.
IBM® WebSphere MQ File Transfer Edition provides an enterprise-grade managed file
transfer capability that is both robust and easy to use. WebSphere MQ File Transfer Edition
exploits the proven reliability and connectivity of WebSphere MQ to transfer files across a
wide range of platforms and networks.WebSphere MQ File Transfer Edition takes advantage
of existing WebSphere MQ networks, and you can integrate it easily with existing file transfer
systems.
This IBM Redguides™ publication provides a business overview of WebSphere® MQ File
Transfer Edition. Figure 1 illustrates the concepts of WebSphere MQ File Transfer Edition.
Configure
Track
Audit
File
WebSphere MQ File Transfer Edition
Figure 1 Overview of WebSphere MQ File Transfer Edition
© Copyright IBM Corp. 2009. All rights reserved.
1
WebSphere MQ File Transfer Edition offers the following benefits:
򐂰 Provides reliable managed file transfer (MFT) using WebSphere MQ
򐂰 Enables consolidation of messaging and the file transfer infrastructure onto single
backbone
򐂰 Provides bulk transfer of files, regardless of size
򐂰 Offers a record log of transfer activity for audit purposes
򐂰 Offers automation and control of file movement between IT systems
򐂰 Extends the WebSphere MQ Explorer graphical user interface (GUI) for configuring
transfers and monitoring progress remotely
򐂰 Provides scripting support for programmatic control of transfers
򐂰 Is a service-oriented architecture (SOA) ready approach that enables files to be
transferred onto Enterprise Service Buses (ESBs)
This IBM Redguide publication provides an introduction to IBM WebSphere MQ File Transfer
Edition and describes how it relates to existing file transfer technologies as well as other
business systems. Later sections of this guide describe the management, auditing, and
reporting capabilities of WebSphere MQ File Transfer Edition.
The need for managed file transfer and WebSphere MQ File
Transfer Edition
The exchange of files throughout an enterprise is a common integration requirement.
Although the exchange of files between systems is conceptually simple, doing so in the
enterprise remains a challenge to manage and audit. This difficulty is brought into clear focus
when an organization needs to perform file transfer with a different business organization,
perhaps using a different physical network, with different security requirements, and perhaps
a different governance or regulatory framework.
Despite a plethora of new technologies that include Web services, Web 2.0, and many robust
existing technologies such as Enterprise Messaging, file transfer remains a common, almost
ubiquitous method of integrating business systems. WebSphere MQ File Transfer Edition is
designed to address the needs of organizations that rely on the transfer of files as well as
organizations that want more control, manageability, and auditability.
Managed File Transfer for SOA using IBM WebSphere MQ File Transfer Edition
2
Basic FTP
File transfer has a long history. There are many existing tools that support it in some form.
The simplest and best known tool for file transfer is the File Transfer Protocol (FTP), which
was first made available in UNIX® systems in the 1970s. Today, the broad availability of FTP
on almost all platforms makes it an easy choice when the need to exchange files arises.
However, performing mission-critical file transfers using FTP does have issues:
򐂰 Limited reliability
FTP-based designs do not generally handle network or other errors well. Large file
transfers that are interrupted often cannot be resumed. FTP is not transactional, so
sending a set of files in an all-or-nothing way is not done easily.
򐂰 Limited recoverability
Because FTP deals in basic file units, it cannot know easily that a file transfer is
incomplete and cannot recover and restart the transfer.
򐂰 Limited security
FTP often requires user IDs and passwords, and these are typically sent in clear-text form
over the wire.
򐂰 Limited auditability
Typically, FTP-based transfer cannot be monitored, logged, or audited easily.
Addressing IT challenges using WebSphere MQ File Transfer Edition
WebSphere MQ File Transfer Edition can address many IT challenges:
򐂰 Assure integrity of file contents
WebSphere MQ File Transfer Edition uses a strong message digest algorithm to verify that
each file is delivered completely and correctly.
򐂰 Improve reliability of file transfer infrastructure
WebSphere MQ File Transfer Edition takes advantage of the reliability and delivery
assurances of WebSphere MQ. WebSphere MQ File Transfer Edition uses WebSphere
MQ to reliably deliver files to any WebSphere MQ supported platform.
򐂰 Consolidate and re-use existing MQ infrastructure
Because WebSphere MQ File Transfer Edition uses and relies on WebSphere MQ for file
transfer, a new network or other infrastructure is often not required. WebSphere MQ File
Transfer Edition can be configured with or without a local WebSphere MQ queue manager.
򐂰 Take advantage of an existing SOA infrastructure
WebSphere MQ File Transfer Edition is part of the IBM WebSphere MQ family and can be
deployed in concert with a new or existing SOA infrastructure.
򐂰 Reduce dependency on IT specialists
Using WebSphere MQ File Transfer Edition the benefits of managed file transfer can be
realized without programming or other specialized skills. Existing WebSphere MQ skills
can be redeployed and used to enable enterprise-grade managed file transfer.
򐂰 Zero-programming requirement
WebSphere MQ File Transfer Edition can provide all the benefits of visibility, management,
and auditability without the need to develop programs or complex scripts.
Managed File Transfer for SOA using IBM WebSphere MQ File Transfer Edition
3
File transfer using WebSphere MQ File Transfer Edition
WebSphere MQ File Transfer Edition provides enterprise-grade managed file transfer using a
network of WebSphere MQ queue managers. WebSphere MQ File Transfer Edition is the
newest member of the WebSphere MQ family and takes advantage of the proven reliability
and wide connectivity choices of WebSphere MQ.
WebSphere MQ File Transfer Edition provides the following features:
򐂰 Delivers robust solutions for managed file transfer
–
–
–
–
Enables control of all aspects of file movement between IT systems
Provides file delivery reliability
Optimized for both small and massive files
Provides audit trail of transfers
򐂰 Designed to integrate with the IBM SOA portfolio
– Enables files to be delivered to WebSphere Message Broker for file processing
򐂰 Provides secure file transfers using WebSphere MQ security mechanisms.
Relationship to WebSphere MQ
WebSphere MQ File Transfer Edition uses WebSphere MQ queue managers to manage file
transfers and to carry file data around the network. Figure 2 illustrates the relationship
between WebSphere MQ File Transfer Edition and WebSphere MQ.
MQ Network
Queue Manager
Agent
Agent
Queue Manager
Queue Manager
MQ Explorer
Command-line
Database Logger
Figure 2 Managed File Transfer using WebSphere MQ File Transfer Edition
Coordination queue manager
The WebSphere MQ File Transfer Edition coordination queue manager is the central
command and control point in the WebSphere MQ File Transfer Edition network. The
coordination queue managers receives status and other updates from WebSphere MQ File
Transfer Edition agents and publishes information to interested subscribers.
Managed File Transfer for SOA using IBM WebSphere MQ File Transfer Edition
4
Agent queue managers
The WebSphere MQ File Transfer Edition server agents can use their local queue manager or
a remote queue manager to send and receive files into the MQ network. WebSphere MQ File
Transfer Edition client agents use a remote queue manager to send and receive files.
Command queue managers
The command-line tools and the WebSphere MQ File Transfer Edition GUI use the command
queue manager to communicate with agents and the coordination queue manager.
Platforms and environments
WebSphere MQ File Transfer Edition works with WebSphere MQ at Version 6 or later on the
following platforms:
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
򐂰
Linux® on X86 Intel®
Linux for IBM System z®
Microsoft® Windows®
IBM AIX®
IBM z/OS®
Sun™ Solaris™
Hewlett Packard HP/UX
The WebSphere MQ File Transfer Edition coordination queue manager must be using
WebSphere MQ Version 7 or later.
For more information about supported platforms and environments, refer to:
http://ibm.com/webspheremq/filetransfer/requirements
WebSphere MQ File Transfer Edition Client
WebSphere MQ File Transfer Edition Client is used when there is no local WebSphere MQ
queue manager. The WebSphere MQ File Transfer Edition Client can connect to a remote
WebSphere MQ. It allows you to establish client transport mode connections to the
WebSphere MQ network.
WebSphere MQ File Transfer Edition Server
WebSphere MQ File Transfer Edition Server installs a file transfer agent and includes
WebSphere MQ Version 7.0 CDs. It can be used with a local or remote queue managers.
WebSphere MQ File Transfer Edition for z/OS
WebSphere MQ File Transfer Edition for z/OS can be used with a local z/OS queue manager
and brings managed file transfer to the zOS platform.
WebSphere MQ File Transfer Edition Remote Tools and Documentation
WebSphere MQ File Transfer Edition Remote Tools and Documentation contains a complete
set of documentation along with the WebSphere MQ Explorer graphical configuration tool, the
WebSphere MQ File Transfer Edition remote commands, the Ant scripting tool, the database
logger, and message schemas.
Managed File Transfer for SOA using IBM WebSphere MQ File Transfer Edition
5
Managing and auditing file transfers
WebSphere MQ File Transfer Edition is a platform for performing file transfers and auditing
those transfers.
Performing file transfers
Using WebSphere MQ File Transfer Edition, you can initiate a file transfer using either the
WebSphere MQ File Transfer Edition command-line tools or the WebSphere MQ Explorer
GUI. You can also initiate a file transfer from an Ant script or by sending a suitable MQ
request message to the agent’s queue.
Figure 3 shows how to use WebSphere MQ Explorer to initiate and monitor WebSphere MQ
File Transfer Edition file transfer.
Figure 3 Administration using WebSphere MQ Explorer
Managed File Transfer for SOA using IBM WebSphere MQ File Transfer Edition
6
You can also use WebSphere MQ File Transfer Edition to initiate file transfers using
commands. Example 1 uses the fteCreateTransfer command to create a new file transfer.
Example 1 Initiating a file transfer using a command
fteCreateTransfer
-sa WASH.AGENT
-sm WASHQM
-da NY.AGENT
-dm NYQM
-df /u/dailyfincollection/daily.financial.WASH.report.txt
-t text
-de overwrite
-sd leave
-gt WASH.TS.xml
c:\u\dailyfin\daily.financial.WASH.report.txt
Using resource monitors
Resource monitoring is a feature that allows WebSphere MQ File Transfer Edition to start a
file transfer when the contents of a directory meets certain conditions. Resource monitoring
allows a directory to be actively monitored by WebSphere MQ File Transfer Edition. When a
specified file (or files) exist, a file transfer is started automatically.
A resource monitor can poll a directory, or a nested directory tree, looking for the existence of
files with a certain name or for names that match a pattern. You can create resource monitors
using either WebSphere MQ Explorer or the WebSphere MQ File Transfer Edition
command-line tools.
Figure 4 shows how to use WebSphere MQ Explorer to create a new resource monitor.
Figure 4 Using MQ Explorer to create a new resource monitor
Managed File Transfer for SOA using IBM WebSphere MQ File Transfer Edition
7
Resource monitors can be used to initiate a file transfer between two systems or to call a
program or script at a system when the monitor’s conditions are met.
Auditing and logging
WebSphere MQ File Transfer Edition logs the status of all file transfer requests to its transfer
log in the form of stored WebSphere MQ messages. You can view these messages using the
WebSphere MQ Explorer interface. Figure 5 shows the WebSphere MQ Explorer Transfer
Log.
Figure 5 WebSphere MQ Explorer transfer log
Reporting
WebSphere MQ File Transfer Edition includes the ability to send transfer log messages to an
external database for reporting and post-analysis. The WebSphere MQ File Transfer Edition
database logger can process WebSphere MQ File Transfer Edition transfer log messages and
can send their contents to either a DB2® or Oracle® database.
Managed File Transfer for SOA using IBM WebSphere MQ File Transfer Edition
8
Integrating WebSphere MQ File Transfer Edition
WebSphere MQ File Transfer Edition uses the WebSphere MQ network. This feature enables
many integration opportunities with other WebSphere MQ based technologies, such as
enterprise messaging, JMS, and Web 2.0, as shown in Figure 6.
Managed File
Transfer
Enterprise
Messaging
Java Message
Service
HTTP and
Web 2.0
IBM WebSphere MQ Family
Publish and
Subscribe
SOAP and
Web Services
Low-Latency
Messaging
Mobile and
Telemetry
Figure 6 Integrating managed file transfer as part of a WebSphere MQ network
WebSphere MQ File Transfer Edition can perform managed file transfers between systems
that are running WebSphere MQ File Transfer Edition agents, and you can also use it to
integrate with existing file transfer systems.
You can integrate WebSphere MQ File Transfer Edition with existing business systems using
resource monitors, Apache Ant tasks that are available with WebSphere MQ File Transfer
Edition tasks, and pre- and post-processing tasks.
Resource monitors
WebSphere MQ File Transfer Edition can use its resource monitor feature to initiate a file
transfer based on the contents of a directory. You can use resource monitors to ensure that a
file transfer begins only when certain files exist. After a resource monitor is defined,
WebSphere MQ File Transfer Edition monitors the specified directory and initiates a transfer
when a predefined file is created in it.
Apache Ant tasks
Apache Ant is an XML-based scripting tool, released by the Apache Software Foundation,
that is widely used for building Java™-based software suites. Although its original purpose
was to manage the building of Java software, Ant is becoming popular as a general-purpose
scripting tool. WebSphere MQ File Transfer Edition can integrate its file transfer functions
using scripts that are run by Ant.
WebSphere MQ File Transfer Edition offers a set of Ant tasks that allow an Ant script to initiate
WebSphere MQ File Transfer Edition file transfers at the local or a remote system or call an
command or program at any system that has a WebSphere MQ File Transfer Edition agent.
File transfer pre-processing and post-processing tasks
When configuring WebSphere MQ File Transfer Edition to send and receive files, WebSphere
MQ File Transfer Edition can run a program or script either before or after the transfer occurs.
Pre-processing tasks are executed before the file transfer and post-processing tasks are
executed after the file transfer. Sophisticated integration needs can be addressed using a
combination of pre-processing and post-processing tasks.
Managed File Transfer for SOA using IBM WebSphere MQ File Transfer Edition
9
Summary
This guide provided an overview of WebSphere MQ File Transfer Edition, its relationship to
WebSphere MQ, and how it addresses the business requirements for robust managed file
transfer.
Other resources for more information
For more information about this topic, consult the following resources:
򐂰 For a technical overview of WebSphere MQ File Transfer Edition refer to IBM WebSphere
MQ File Transfer Edition Solution Overview, REDP-453232.
򐂰 For guidance about implementing WebSphere MQ File Transfer Edition solutions,
including step-by-step instructions and a sample scenario, refer to Getting Started with
IBM WebSphere MQ File Transfer Edition V7, SG24-7760.
The team who wrote this guide
This guide was produced by a team of specialists from around the world working at the
International Technical Support Organization (ITSO), Raleigh Center.
David Ward is Senior Software Engineer with IBM Software Group in the U. S. He has 30
years of experience in software architecture and design as well as network and systems
development. His areas of expertise include enterprise messaging and software development
methodologies.
Martin Keen is a Consulting IT Specialist at the ITSO, Raleigh Center. He writes extensively
about WebSphere products and SOA. He also teaches IBM classes worldwide about
WebSphere, SOA, and ESB. Before joining the ITSO, Martin worked in the EMEA
WebSphere Lab Services team in Hursley, U. K. Martin holds a bachelor’s degree in
Computer Studies from Southampton Institute of Higher Education.
Ran Gu has a Bachelor degree of Computer Science and Technology from TsingHua
University, China. After Ran graduated from the Chinese Academy of Sciences in 2004 with a
Master degree of Software Engineering, he joined IBM China as an IT Specialist working for 5
years in WebSphere technical sales support. Ran’s responsibility is helping customers build
ESB and data transfer solutions based on IBM products of WebSphere MQ File Transfer
Edition, WebSphere Message Broker, WebSphere Partner Gateway, and WebSphere
Transformation Extender.
Eugene Kuehlthau is a Software IT Specialist in the U. S. He has 35 years of experience in
the field of data processing. His areas of expertise include WebSphere MQ, WebSphere MQ
Workflow, and WebSphere Process Server. He has developed WebSphere MQ and Workflow
courses.
Leonard McWilliams is a Consulting IT Specialist on the WebSphere Federal sales team
working primarily with WebSphere MQ and WebSphere Message Broker in classified
accounts. He has 35 years of IT industry consulting and application development experience
including messaging, database management, and geographical information systems. He has
a B.S. in Biological Science, Philosophy, and Music from the University of Kansas and a M.A.
in Education from Antioch University.
Managed File Transfer for SOA using IBM WebSphere MQ File Transfer Edition
10
Notices
This information was developed for products and services offered in the U.S.A.
IBM may not offer the products, services, or features discussed in this document in other countries. Consult
your local IBM representative for information on the products and services currently available in your area. Any
reference to an IBM product, program, or service is not intended to state or imply that only that IBM product,
program, or service may be used. Any functionally equivalent product, program, or service that does not
infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to
evaluate and verify the operation of any non-IBM product, program, or service.
IBM may have patents or pending patent applications covering subject matter described in this document. The
furnishing of this document does not give you any license to these patents. You can send license inquiries, in
writing, to:
IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A.
The following paragraph does not apply to the United Kingdom or any other country where such
provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION
PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR
IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of
express or implied warranties in certain transactions, therefore, this statement may not apply to you.
This information could include technical inaccuracies or typographical errors. Changes are periodically made
to the information herein; these changes will be incorporated in new editions of the publication. IBM may make
improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time
without notice.
Any references in this information to non-IBM Web sites are provided for convenience only and do not in any
manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the
materials for this IBM product and use of those Web sites is at your own risk.
IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring
any obligation to you.
Information concerning non-IBM products was obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not tested those products and cannot confirm the
accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the
capabilities of non-IBM products should be addressed to the suppliers of those products.
This information contains examples of data and reports used in daily business operations. To illustrate them
as completely as possible, the examples include the names of individuals, companies, brands, and products.
All of these names are fictitious and any similarity to the names and addresses used by an actual business
enterprise is entirely coincidental.
COPYRIGHT LICENSE:
This information contains sample application programs in source language, which illustrate programming
techniques on various operating platforms. You may copy, modify, and distribute these sample programs in
any form without payment to IBM, for the purposes of developing, using, marketing or distributing application
programs conforming to the application programming interface for the operating platform for which the sample
programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore,
cannot guarantee or imply reliability, serviceability, or function of these programs.
© Copyright IBM Corp. 2009. All rights reserved.
11
This document, REDP-4533-00, was created or updated on August 3, 2009.
®
Trademarks
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of
International Business Machines Corporation in the United States, other countries, or
both. These and other IBM trademarked terms are marked on their first occurrence in
this information with the appropriate symbol (® or ™), indicating US registered or
common law trademarks owned by IBM at the time this information was published. Such
trademarks may also be registered or common law trademarks in other countries. A
current list of IBM trademarks is available on the Web at
http://www.ibm.com/legal/copytrade.shtml
Redbooks®
The following terms are trademarks of the International Business Machines Corporation in the United States,
other countries, or both:
AIX®
DB2®
IBM®
Redguides™
Redbooks (logo)
System z®
®
WebSphere®
z/OS®
Oracle, JD Edwards, PeopleSoft, Siebel, and TopLink are registered trademarks of Oracle Corporation and/or
its affiliates.
Other company, product, or service names may be trademarks or service marks of others.
Managed File Transfer for SOA using IBM WebSphere MQ File Transfer Edition
12