Front cover z/OS: WebSphere Business Process Management V6.2 Production Topologies Creating production topologies for WebSphere Process Server Incorporating WebSphere Business Services Fabric Integrating WebSphere Business Monitor John Gates John Hutchinson Thomas McManus Mohamed ShamsEldin Salem Martin Keen ibm.com/redbooks International Technical Support Organization z/OS: WebSphere Business Process Management V6.2 Production Topologies June 2009 SG24-7733-00 Note: Before using this information and the product it supports, read the information in , “Notices” on page xiii. First Edition (June 2009) This edition applies to Version 6.2 of WebSphere Process Server, WebSphere Business Services Fabric, and WebSphere Business Monitor. © Copyright International Business Machines Corporation 2009. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv The team that wrote this book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii Part 1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Chapter 1. Business Process Management production topologies for z/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.1.1 WebSphere Application Server for z/OS architecture . . . . . . . . . . . . . 5 1.1.2 Hardware considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.1.3 HTTP servers, LDAP servers, edge servers, proxies, external security managers, and so forth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.1.4 Non-functional requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.1.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.2 WebSphere Process Server for z/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.2.1 Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.2.2 Service integration buses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.2.3 Messaging engines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.2.4 WebSphere MQ for z/OS queue sharing . . . . . . . . . . . . . . . . . . . . . 15 1.2.5 Business Process Choreographer . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.2.6 Common Event Infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.2.7 WebSphere Process Server Applications . . . . . . . . . . . . . . . . . . . . . 16 1.3 Deployment environments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.3.1 Single-cluster topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 1.3.2 Two-cluster topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 1.3.3 Custom topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 1.4 Selecting an appropriate topology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 1.4.1 The single cluster pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 1.4.2 The two cluster pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 1.4.3 Custom patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 1.4.4 Condensed pattern selection criteria. . . . . . . . . . . . . . . . . . . . . . . . . 28 1.5 BPM reference topology for z/OS and how we will use it in this book. . . . 30 1.6 Incorporating WebSphere Business Services Fabric for z/OS into the BPM reference topology for z/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 © Copyright IBM Corp. 2009. All rights reserved. iii 1.6.1 WebSphere Business Services Fabric for z/OS modules . . . . . . . . . 33 1.6.2 WebSphere Business Services Fabric for z/OS . . . . . . . . . . . . . . . . 33 1.7 Monitoring the BPM environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 1.7.1 Monitoring the BPM reference topology for z/OS using WebSphere Business Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Part 2. Building production topologies for WebSphere Process Server . . . . . . . . . . . . . . 37 Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 2.1.1 Starting with a standalone server . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 2.1.2 Outlining the steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 2.1.3 Extending production topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 2.2 Planning the production network deployment topology. . . . . . . . . . . . . . . 44 2.2.1 Choosing the topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 2.2.2 Using a spreadsheet to plan names for configuration objects . . . . . 45 2.3 Preparing the z/OS system components. . . . . . . . . . . . . . . . . . . . . . . . . . 53 2.3.1 Planning for hardware requirements for these topologies. . . . . . . . . 53 2.3.2 Planning for software requirements . . . . . . . . . . . . . . . . . . . . . . . . . 56 2.3.3 Using SMP/E to install WebSphere Process Server for z/OS . . . . . . 59 2.3.4 Planning z/OS dataset names and aliases . . . . . . . . . . . . . . . . . . . . 61 2.3.5 Planning UNIX (UNIX System Services) file systems . . . . . . . . . . . . 63 2.3.6 Using intermediate symbolic links . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 2.3.7 Planning security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 2.3.8 Planning RACF user IDs, groups and EJBROLE profiles . . . . . . . . . 70 2.4 Preparing DB2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 2.4.1 DB2 terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 2.4.2 Planning DB2 names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 2.4.3 Choosing currentSchema or currentSQLID . . . . . . . . . . . . . . . . . . . 83 2.4.4 Using GRANT statements in DB2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 2.4.5 Using the started task ID to connect to the database . . . . . . . . . . . . 91 2.4.6 Deciding how you will create and execute the DDL . . . . . . . . . . . . . 93 2.4.7 Creating a DB2JccConfiguration.properties file . . . . . . . . . . . . . . . . 96 2.4.8 Customizing .profile and .setup files for users . . . . . . . . . . . . . . . . . 98 2.4.9 Verify Java and the DB2 Universal JDBC Driver . . . . . . . . . . . . . . 101 2.5 Creating a standalone WebSphere application server . . . . . . . . . . . . . . 102 2.5.1 Task overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 2.5.2 Using the spreadsheet to plan the server configuration . . . . . . . . . 104 2.5.3 Using the zPMT to create the installation jobs . . . . . . . . . . . . . . . . 105 2.5.4 Running the configuration jobs to create the standalone server . . . 108 2.5.5 Defining user IDs, group IDs, and EJBROLE profiles . . . . . . . . . . . 110 2.5.6 Starting the application server and verifying the configuration . . . . 110 iv z/OS: WebSphere Business Process Management V6.2 Production Topologies 2.5.7 Customizing the new server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 2.5.8 Backing up the configuration ZFS . . . . . . . . . . . . . . . . . . . . . . . . . . 113 2.6 Adding WebSphere Process Server for z/OS . . . . . . . . . . . . . . . . . . . . . 114 2.6.1 Running the zSMPInstall script . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 2.6.2 Preparing the standalone profile response file . . . . . . . . . . . . . . . . 117 2.6.3 Running the zWPSConfig.sh script . . . . . . . . . . . . . . . . . . . . . . . . . 120 2.6.4 Configuring the SCA, CEI and BPC components . . . . . . . . . . . . . . 121 2.6.5 Business Space powered by WebSphere . . . . . . . . . . . . . . . . . . . . 122 2.7 Creating the DB2 databases and tables . . . . . . . . . . . . . . . . . . . . . . . . . 122 2.7.1 Using the sample createDB.sh script . . . . . . . . . . . . . . . . . . . . . . . 123 2.7.2 Customizing the sample createDB.sh script . . . . . . . . . . . . . . . . . . 123 2.7.3 Executing createDB.sh to generate the DDL . . . . . . . . . . . . . . . . . 126 2.7.4 Modifying the generated DDL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 2.7.5 Creating the database and storage group. . . . . . . . . . . . . . . . . . . . 129 2.7.6 Executing the DDL with the createDB script . . . . . . . . . . . . . . . . . . 130 2.7.7 Executing the DDL with DSNTEP2 batch job . . . . . . . . . . . . . . . . . 132 2.7.8 Using the DB2 command line processor . . . . . . . . . . . . . . . . . . . . . 133 2.7.9 Granting table privileges to the J2C auth alias user ID . . . . . . . . . . 135 2.7.10 Verifying the DB2 table creation . . . . . . . . . . . . . . . . . . . . . . . . . . 136 2.8 Starting the standalone application server . . . . . . . . . . . . . . . . . . . . . . . 138 2.8.1 Review any error messages and repair . . . . . . . . . . . . . . . . . . . . . 138 2.9 Testing sample applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 2.9.1 Installing a simple BPC application (WPSEcho.ear) . . . . . . . . . . . . 142 2.9.2 Installing a long-running BPEL application . . . . . . . . . . . . . . . . . . . 144 2.10 Testing the vehicle loan process application . . . . . . . . . . . . . . . . . . . . . 146 2.10.1 Installing the sample application . . . . . . . . . . . . . . . . . . . . . . . . . . 146 2.10.2 Updating the SCA import URLs . . . . . . . . . . . . . . . . . . . . . . . . . . 147 2.10.3 Testing the sample application . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 Chapter 3. Creating a WebSphere Process Server for z/OS network deployment cell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 3.1.1 Planning for the network deployment cell . . . . . . . . . . . . . . . . . . . . 153 3.2 Creating the deployment manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 3.2.1 Task overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 3.2.2 Using the spreadsheet to plan the deployment manager . . . . . . . . 155 3.2.3 Using the zPMT to create the installation jobs . . . . . . . . . . . . . . . . 155 3.2.4 Running configuration jobs to create the deployment manager . . . 155 3.2.5 Starting deployment manager and verifying the configuration . . . . 156 3.2.6 Reviewing and fixing any errors . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 3.2.7 Backing up the deployment manager’s ZFS data set . . . . . . . . . . . 158 3.3 Adding WebSphere Process Server for z/OS to the deployment manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 Contents v 3.3.1 Running the zSMPInstall script . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 3.3.2 Preparing the DmgrDB2.rsp response file . . . . . . . . . . . . . . . . . . . 161 3.3.3 Backing up the deployment manager’s ZFS . . . . . . . . . . . . . . . . . . 162 3.3.4 Running the zWPSConfig script . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 3.3.5 Starting the deployment manager . . . . . . . . . . . . . . . . . . . . . . . . . . 164 3.3.6 Verifying the deployment manager runtime configuration . . . . . . . . 164 3.3.7 Backing up the HFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 3.4 Federating the standalone server into this ND cell . . . . . . . . . . . . . . . . . 165 3.4.1 Updating the primary node’s node agent ports . . . . . . . . . . . . . . . . 166 3.4.2 Backing up the HFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 3.5 Testing the WebSphere Process Server-configured server . . . . . . . . . . 169 3.5.1 Starting the node agent. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 3.5.2 Starting the server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 3.5.3 Testing sample applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 3.5.4 Backing up the ZFSes in this ND cell . . . . . . . . . . . . . . . . . . . . . . . 170 3.6 Creating a cluster from the server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 3.6.1 Checking the virtual host alias ports and short names . . . . . . . . . . 173 3.6.2 Fixing the CEI messaging engine . . . . . . . . . . . . . . . . . . . . . . . . . . 174 3.6.3 Restarting the cluster member . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 3.6.4 Backing up the ND cell containing the clustered server . . . . . . . . . 177 3.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 Chapter 4. Creating additional WebSphere Process Server nodes . . . . 179 4.1 Review and overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 4.2 Creating an empty node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 4.2.1 Using the spreadsheet to plan the server configuration . . . . . . . . . 182 4.2.2 Using the zPMT to create the installation jobs . . . . . . . . . . . . . . . . 182 4.2.3 Preparing the new z/OS system . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 4.2.4 Running the configuration jobs to create an empty node . . . . . . . . 184 4.2.5 Customizing the servant and adjunct procedures for DB2 . . . . . . . 185 4.3 Configuring the new node for WebSphere Process Server for z/OS . . . 186 4.3.1 Running the zSMPInstall script . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 4.3.2 Tailoring the response file for the empty node . . . . . . . . . . . . . . . . 187 4.3.3 Running zWPSConfig script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188 4.3.4 Backing up the configuration ZFS . . . . . . . . . . . . . . . . . . . . . . . . . . 188 4.4 Federating the new node into the network deployment cell . . . . . . . . . . 188 4.5 Creating an additional cluster member . . . . . . . . . . . . . . . . . . . . . . . . . . 189 4.6 Validating the new cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 4.6.1 Starting the node agent. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 4.6.2 Starting the new cluster member . . . . . . . . . . . . . . . . . . . . . . . . . . 196 4.6.3 Looking at the final topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 4.6.4 Testing sample applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 vi z/OS: WebSphere Business Process Management V6.2 Production Topologies Part 3. Extending production topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 Chapter 5. WebSphere Business Services Fabric for z/OS . . . . . . . . . . . 201 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 5.1.1 Planning and preparing the Fabric installation . . . . . . . . . . . . . . . . 204 5.1.2 Software pre-requisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 5.1.3 Installing WebSphere Business Services Fabric for z/OS in SMP/E. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 5.1.4 Sample JCL and shell scripts used in this chapter . . . . . . . . . . . . . 205 5.1.5 Creating the deployment directory . . . . . . . . . . . . . . . . . . . . . . . . . 205 5.2 Install WebSphere Business Services Fabric for z/OS in a standalone node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206 5.2.1 Copy Fabric artifacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 5.2.2 Configure security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 5.2.3 Configure databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 5.2.4 Augment WebSphere Process Server for z/OS standalone node profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220 5.2.5 Verify the augmentation process succeeded . . . . . . . . . . . . . . . . . 225 5.2.6 Post augmentation tasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 5.2.7 Verify the WebSphere Business Services Fabric for z/OS installation and configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252 5.2.8 Create symlink for fabric product registration . . . . . . . . . . . . . . . . . 258 5.3 Optional configuration steps for stand-alone server . . . . . . . . . . . . . . . . 259 5.3.1 Configure WebSphere Application Server for z/OS servants for 64-bit addressing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260 5.3.2 Configure message engines to use MQ shared queues . . . . . . . . . 265 5.3.3 Define HTTP servers or proxies . . . . . . . . . . . . . . . . . . . . . . . . . . . 278 5.3.4 Configure LDAP security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280 5.4 Troubleshooting the WebSphere Business Services Fabric for z/OS installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298 5.5 WebSphere Business Services Fabric events . . . . . . . . . . . . . . . . . . . . 298 Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299 6.1 Configuring WebSphere Business Services Fabric for z/OS in a clustered environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300 6.2 Install WebSphere Business Services Fabric for z/OS in the deployment manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301 6.2.1 Augment WebSphere Process Server for z/OS deployment manager profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302 6.2.2 Verify the augmentation process succeeded . . . . . . . . . . . . . . . . . 305 6.2.3 Post augmentation tasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316 6.3 Install WebSphere Business Services Fabric for Contents vii z/OS in a managed node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327 6.3.1 Augment WebSphere Process Server for z/OS managed node profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328 6.3.2 Verify the augmentation process succeeded . . . . . . . . . . . . . . . . . 332 6.3.3 Post augmentation tasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338 6.4 Verify the WebSphere Business Services Fabric for z/OS basic clustered node operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339 6.4.1 Start cluster node agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340 6.4.2 Start clustered servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342 6.5 Complete cluster configuration for WebSphere Business Services Fabric for z/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343 6.6 Creating a WebSphere Business Services Fabric for z/OS cluster configuration with empty managed nodes . . . . . . . . . . . . . . . . . . . . . . . 345 6.6.1 Running the post augmentation WebSphere Business Services Fabric for z/OS python configuration scripts . . . . . . . . . . . . . . . . . . . . . . . 345 6.6.2 Run fabricSIBConfig.py. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346 6.6.3 Verify fabricSIBConfig.py completed successfully . . . . . . . . . . . . . 347 6.6.4 Run fabricAppDeploy.py . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354 6.6.5 Verify fabricAppDeploy.py completed successfully . . . . . . . . . . . . . 355 6.7 Configuring a WebSphere Business Services Fabric for z/OS cluster from a stand-alone node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361 6.8 Verify basic server operation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397 Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403 7.1 Install and test the sample application . . . . . . . . . . . . . . . . . . . . . . . . . . 405 7.1.1 Update the endpoint URLs of ITSOBankCBAPrj . . . . . . . . . . . . . . 406 7.1.2 Change endpoint URLs by editing the .owl file . . . . . . . . . . . . . . . . 406 7.1.3 Import the FCA files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407 7.1.4 Assign MDFABADM to the ITSOBankOrg team . . . . . . . . . . . . . . . 408 7.1.5 Create an external LDAP federation project . . . . . . . . . . . . . . . . . . 409 7.1.6 Set enrollments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416 7.1.7 Install the sample application’s .ear files . . . . . . . . . . . . . . . . . . . . . 420 7.1.8 Map modules to the Webserver . . . . . . . . . . . . . . . . . . . . . . . . . . . 422 7.1.9 Update the SCA import URLs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423 7.1.10 Test the sample application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424 7.2 Errors we encounters while testing the Fabric sample application . . . . . 427 7.3 Verify WebSphere Business Services Fabric for z/OS functionality using Business Space powered by WebSphere . . . . . . . . . . . . . . . . . . . . . . . 433 7.3.1 Enable Business Space widgets for WebSphere Business Services Fabric for z/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433 7.3.2 Edit required endpoints files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434 7.3.3 Configure the Fabric Rest Services gateway . . . . . . . . . . . . . . . . . 435 viii z/OS: WebSphere Business Process Management V6.2 Production Topologies 7.3.4 Clear the OSGI cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439 7.3.5 Verify WebSphere Business Services Fabric for z/OS Business Space functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439 7.4 Configuring the WebSphere Business Services Fabric for z/OS Content Packs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451 7.4.1 The Banking Payments Content Pack . . . . . . . . . . . . . . . . . . . . . . 452 7.4.2 . . . . . . . . . . . . . . . . . . . . . . . The Healthcare Payer Content Pack454 7.4.3 The Insurance Property and Casualty Content Pack . . . . . . . . . . . 457 7.4.4 The Product Lifecycle Management Content Pack . . . . . . . . . . . . . 459 7.4.5 The Telecom Operations Content Pack . . . . . . . . . . . . . . . . . . . . . 461 Chapter 8. Monitoring the Business Process Management environment465 8.1 Overview of WebSphere Business Monitor. . . . . . . . . . . . . . . . . . . . . . . 466 8.2 Incorporating WebSphere Business Monitor on Linux on System z with WebSphere Process Server for z/OS and the BPM reference topology for z/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467 8.3 Considerations for WebSphere Business Monitor installation. . . . . . . . . 468 8.3.1 Prerequisite software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 468 8.3.2 Installation overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469 8.4 Installation of WebSphere Business Monitor distributed topology . . . . . 470 8.4.1 Installing WebSphere Business Monitor deployment manager. . . . 470 8.4.2 Creating the WebSphere Business Monitor databases . . . . . . . . . 471 8.4.3 Installing WebSphere Business Monitor nodes. . . . . . . . . . . . . . . . 475 8.5 Building WebSphere Business Monitor profiles . . . . . . . . . . . . . . . . . . . 476 8.5.1 Preparing to build profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476 8.5.2 Create WebSphere Business Monitor custom profiles . . . . . . . . . . 477 8.6 Creating the WebSphere Business Monitor clusters . . . . . . . . . . . . . . . . 479 8.7 Configuring the WebSphere Business Monitor infrastructure . . . . . . . . . 483 8.7.1 Deploy CEI Service on WebSphere Business Monitor support cluster . 484 8.7.2 Create a Datasource for the Monitor Messaging Engine . . . . . . . . 486 8.7.3 Using the WebSphere Business Monitor configuration panel . . . . . 487 8.8 Installing WebSphere Business Monitor support applications. . . . . . . . . 490 8.8.1 Deploy Action Manager. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 490 8.8.2 Deploy database service scheduler . . . . . . . . . . . . . . . . . . . . . . . . 491 8.8.3 Deploy REST API service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492 8.8.4 DB2 Alphablox installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492 8.8.5 Deploying Business Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 500 8.8.6 Configure Business Space for dashboard widgets . . . . . . . . . . . . . 502 8.9 Monitoring the BPM reference topology for z/OS using WebSphere Business Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505 8.9.1 Configuring security for linking WebSphere Business Monitor to WebSphere Process Server for z/OS . . . . . . . . . . . . . . . . . . . . . . . 505 Contents ix 8.9.2 8.9.3 8.9.4 8.9.5 Securing WebSphere Business Monitor . . . . . . . . . . . . . . . . . . . . . 506 Securing the BPM reference topology for z/OS . . . . . . . . . . . . . . . 506 Security considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 506 Configuring WebSphere Business Monitor to use a remote CEI server 521 8.9.6 Configuring bus destination roles . . . . . . . . . . . . . . . . . . . . . . . . . . 524 8.10 Additional Monitor SI Bus configuration for the WebSphere Process Server for z/OS cell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527 8.10.1 Monitor SI Bus database configuration . . . . . . . . . . . . . . . . . . . . . 527 8.10.2 Monitor Foreign Bus configuration . . . . . . . . . . . . . . . . . . . . . . . . 534 8.11 Verifying WebSphere Business Monitor functions with the BPM reference topology on z/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537 8.11.1 Preparing the business process for monitoring . . . . . . . . . . . . . . . 537 8.11.2 Preparing the monitor model in the toolkit. . . . . . . . . . . . . . . . . . . 537 8.11.3 Monitor model deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537 8.11.4 Inspecting the monitor model and business process functionality 541 8.11.5 Monitor dashboards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 542 Chapter 9. Creating an integrated user experience using Business Space powered by WebSphere . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547 9.1 Create a data source for Business Space . . . . . . . . . . . . . . . . . . . . . . . . 549 9.2 Enable widgets in Business Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 549 9.3 Run the Business Space Configuration wizard . . . . . . . . . . . . . . . . . . . . 551 9.4 Prepare the Business Space DDL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553 9.5 Execute the Business Space DDL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553 9.6 Define EJBROLE profile for role Administrator . . . . . . . . . . . . . . . . . . . . 554 9.7 Start the applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555 9.8 Enable business rules in the Business Space. . . . . . . . . . . . . . . . . . . . . 555 9.9 Configure local Business Space help . . . . . . . . . . . . . . . . . . . . . . . . . . . 557 9.9.1 Clear the OSGI cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 557 9.9.2 Verify Business Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 558 Part 4. Problem determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 569 Chapter 10. Problem diagnosis and prevention . . . . . . . . . . . . . . . . . . . . 571 10.1 Overview of problem diagnosis and prevention . . . . . . . . . . . . . . . . . . 572 10.2 Problem prevention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573 10.2.1 Good management practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573 10.2.2 Assemble required resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574 10.2.3 Acquire appropriate skills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575 10.3 Tools and techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575 10.3.1 Parallel testing cells . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575 10.3.2 TSO techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575 10.3.3 Scripting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 578 x z/OS: WebSphere Business Process Management V6.2 Production Topologies 10.3.4 UNIX system services: Tools and techniques . . . . . . . . . . . . . . . . 580 10.3.5 Using telnet, vi and ISPF 3.17. . . . . . . . . . . . . . . . . . . . . . . . . . . . 580 10.3.6 UNIX System Services tools on z/OS . . . . . . . . . . . . . . . . . . . . . . 581 10.3.7 DB2 tools. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 583 10.3.8 Tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 588 10.4 Configuration problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 590 10.4.1 Methodologies for determining configuration problems . . . . . . . . 590 10.4.2 Top ten problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 593 10.4.3 Configuring WebSphere Process Server for z/OS . . . . . . . . . . . . 598 10.4.4 Configuring WebSphere Business Services Fabric for z/OS . . . . 603 10.4.5 Configuring Business Space powered by WebSphere . . . . . . . . . 603 10.4.6 DB2 problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603 10.5 Runtime problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 607 Part 5. Appendixes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613 Appendix A. Additional material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 615 Locating the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 616 Using the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 616 How to use the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 616 SampleWpsHtmApps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 617 Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 617 zos_wbm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 617 zos_wps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 618 zos_wbsf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 618 zos_LDAP_config . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 619 zos_planning_spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 620 zos_wpswork. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 620 zPMT_ResponseFiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 621 Installing the sample JCL and DDL on a z/OS system. . . . . . . . . . . . . . . . . . 622 Installing the sample shell scripts on a z/OS system . . . . . . . . . . . . . . . . . . . 623 Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 625 Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 627 IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 627 Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 628 Techdocs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 628 Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 630 How to get Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631 Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631 Contents xi xii z/OS: WebSphere Business Process Management V6.2 Production Topologies Notices This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. © Copyright IBM Corp. 2009. All rights reserved. xiii Trademarks IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. These and other IBM trademarked terms are marked on their first occurrence in this information with the appropriate symbol (® or ™), indicating US registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at http://www.ibm.com/legal/copytrade.shtml The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both: AlphaBlox® DB2 Connect™ DB2 Universal Database™ DB2® Domino® DRDA® IBM® Language Environment® Lotus® RACF® Redbooks® Redbooks (logo) ® System z® SystemPac® Tivoli® WebSphere® z/OS® z9® zSeries® The following terms are trademarks of other companies: ACS, and the Shadowman logo are trademarks or registered trademarks of Red Hat, Inc. in the U.S. and other countries. EJB, J2EE, Java, JDBC, JMX, JNI, JSP, JVM, ZFS, and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Excel, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. xiv z/OS: WebSphere Business Process Management V6.2 Production Topologies Preface This IBM® Redbooks® publication addresses the configuration, administration, and security of the key runtime environments in business process management: WebSphere® Process Server V6.2 and WebSphere Business Services Fabric V6.2 for the z/OS® platform, and WebSphere Business Monitor V6.2 integration to the z/OS platform. Part 1 of this Redbooks publication introduces production topology concepts and terminology and explores the differences between production topologies on distributed platforms and z/OS. In Part 2, through a series of step-by-step instructions you will learn how to create a production topology environment for WebSphere Process Server for z/OS. Part 3 extends the production topology for WebSphere Process Server, describing step-by-step how to add WebSphere Business Services Fabric for z/OS into the topology. This section also addresses integration with WebSphere Business Monitor, running on a z/Linux® partition for monitoring business process management solutions. Additionally, this section shows how Business Space powered by WebSphere can provide an integrated user experience for this topology. Part 4 describes problem diagnosis and prevention guidance that will prove invaluable in creating your own production topologies. A separate publication covering distributed platforms, WebSphere Business Process Management V6.2 Production Topologies, SG24-7732 is also available. © Copyright IBM Corp. 2009. All rights reserved. xv The team that wrote this book This book was produced by a team of specialists from around the world working at the International Technical Support Organization, Raleigh Center. Figure 1 From left to right: John Gates, John Hutchinson, and Martin Keen John Gates is a Senior System z® IT Architect. He has almost 30 years experience working on large systems in all areas of design, development, and test. In his current position, he works with large customers creating cross platform, complex, integrated solutions to e-business problems. He writes extensively and is a frequent speaker at industry events such as Impact and the Share User group conference. He attended Marist College in Poughkeepsie, New York and holds a BS degree in Computer Information Systems. John Hutchinson is an Executive Consulting IT Specialist at the Washington Systems Center (WSC) in Gaithersburg, Maryland. He has over 40 years of experience in mainframe operating systems and middleware field. He has written extensively on JES2, WebSphere Application Server for z/OS, and WebSphere Process Server for z/OS. He holds a degree in Industrial Engineering from the University of California, Berkeley and joined IBM in 1967. Thomas McManus is a Senior Software Engineer with IBM SWG Business Partner Technical Strategy and Enablement. He has ten years of experience deploying, administering, and securing middleware topologies. Tom is an IBM Certified SOA Solution Architectural Designer, IBM Certified Administrator for SOA Solutions—WebSphere Process Server V6.0, and IBM Certified Solution Developer—Web Services Development.. Mohamed ShamsEldin Salem is a Senior IT Specialist with IBM software group in Cairo Technology and Development Center (C-TDC) Egypt. He has 6 years working in WebSphere Business Monitor information development, development, and SWAT teams. Mohamed has experience in installation, configuration, and security for the WebSphere product stack. He provides technical support for xvi z/OS: WebSphere Business Process Management V6.2 Production Topologies WebSphere products in Europe and Africa regions in critical customer situations. He is a certified software solution developer for WebSphere Business Monitor, WebSphere Integration Developer, and WebSphere Portal Server. Mohamed holds a bachelor’s degree in Computer Engineering from Cairo University Egypt. Martin Keen is a Consulting IT Specialist at the ITSO, Raleigh Center. He writes extensively about WebSphere products, and SOA. He also teaches IBM classes worldwide about WebSphere, SOA, and ESB. Before joining the ITSO, Martin worked in the EMEA WebSphere Lab Services team in Hursley, UK. Martin holds a bachelor’s degree in Computer Studies from Southampton Institute of Higher Education. Special thanks to the authors of z/OS: WebSphere Business Process Management V6.1.2 Production Topologies, SG24-7703 Kevin Senior IBM Software Group Lab Services, Italy Ryan Malynn WebSphere Techology Institute, Raleigh, USA Thanks to the following people for their contributions to this project: Naveen Balani IBM Mumbai, MH India Rich Conway International Technical Support Organization, Raleigh Center, USA Russ Heald IBM Hursley, UK Valentin Mahrwald, WPS for z/OS Development, IBM Hursley, UK Mike Poirier, Dave Bonaccorsi IBM Middletown, RI, USA Kelly J Brennan, IBM Middletown Mike Kearney, Mike Loos, Mike Cox, Jennie Liang, Lee-Win Tai, Steve Matulevich, John Goodyear, Brian Pierce IBM Washington Systems Center, Gaithersburg, MD, USA Mitch Johnson, IBM Raleigh Preface xvii Joerg-Ulrich Veser, IBM Germany Saori Tanikawa, IBM Japan Uday Pillai, IBM SWG, USA Mohamed Saeed, Hui Ming Zhong IBM WebSphere Business Monitor SWAT team Luis Sanchez, John Alcorn IBM WebSphere Business Monitor development William Bratton DB2® Alphablox support team Become a published author Join us for a two- to six-week residency program! Help write a book dealing with specific products or solutions, while getting hands-on experience with leading-edge technologies. You will have the opportunity to team with IBM technical professionals, Business Partners, and Clients. Your efforts will help increase product acceptance and customer satisfaction. As a bonus, you will develop a network of contacts in IBM development labs, and increase your productivity and marketability. Find out more about the residency program, browse the residency index, and apply online at: ibm.com/redbooks/residencies.html Comments welcome Your comments are important to us! We want our books to be as helpful as possible. Send us your comments about this book or other IBM Redbooks in one of the following ways: Use the online Contact us review Redbooks form found at: ibm.com/redbooks Send your comments in an e-mail to: redbooks@us.ibm.com xviii z/OS: WebSphere Business Process Management V6.2 Production Topologies Mail your comments to: IBM Corporation, International Technical Support Organization Dept. HYTD Mail Station P099 2455 South Road Poughkeepsie, NY 12601-5400 Preface xix xx z/OS: WebSphere Business Process Management V6.2 Production Topologies Part 1 Part 1 Overview © Copyright IBM Corp. 2009. All rights reserved. 1 2 z/OS: WebSphere Business Process Management V6.2 Production Topologies 1 Chapter 1. Business Process Management production topologies for z/OS This chapter introduces WebSphere Process Server for z/OS components. It also introduces topology patterns. Topology patterns for z/OS are compared to topology patterns for non-z/OS, described in IBM Redbooks publication WebSphere Business Process Management V6.2 Production Topologies, SG24-7732. This chapter contains the following sections: 1.1, “Introduction” on page 4 1.2, “WebSphere Process Server for z/OS” on page 12 1.3, “Deployment environments” on page 17 1.4, “Selecting an appropriate topology” on page 24 1.5, “BPM reference topology for z/OS and how we will use it in this book” on page 30 1.6, “Incorporating WebSphere Business Services Fabric for z/OS into the BPM reference topology for z/OS” on page 32 1.7, “Monitoring the BPM environment” on page 34 © Copyright IBM Corp. 2009. All rights reserved. 3 Two topology patterns are described in 1.3, “Deployment environments” on page 17: Single-cluster topology Two-clusters topology Reasons for choosing one pattern over another are discussed in 1.4, “Selecting an appropriate topology” on page 24. For WebSphere Process Server on non-z/OS platforms, topology choices have been given names like bronze, silver, or gold. With WebSphere Process Server for z/OS we have avoided using names that imply that one choice is better than another. Instead we discuss the advantages and disadvantages of a single-cluster topology compared to the two-clusters topology, to help you select a production topology that best meets your requirements. Using the above information, we discuss our recommended topology for a BPM production deployment on z/OS in 1.5, “BPM reference topology for z/OS and how we will use it in this book” on page 30. We refer to this deployment model throughout this book. 1.1 Introduction A WebSphere Process Server topology is the physical layout of the deployment environment required to meet your business needs for capacity, availability, and scalability. When discussing topologies for WebSphere Process Server on non-z/OS platforms, the choices are usually described as follows: Single Cluster topology, or bronze topology Remote Messaging topology, or silver topology Remote Messaging and Remote Support, gold topology, or the ND7 topology Custom topology The names bronze, silver, and gold come about because performance studies have shown that the scalability depends on the topology. Therefore gold is better for performance than silver, which in turn, is better than bronze. With WebSphere Process Server for z/OS, names like this are misleading because it is not so clear what makes one topology better than another on z/OS. The choice of topology depends on the particular circumstances of each customer. In most cases it will not be necessary to split the components of WebSphere Process Server for z/OS across multiple clusters to achieve significant capacity, availability, and scalability. 4 z/OS: WebSphere Business Process Management V6.2 Production Topologies This difference between WebSphere Process Server for z/OS and WebSphere Process Server on non-z/OS platforms stems from the different architecture of the underlying WebSphere Application Server for z/OS. It is possible to have a mixed-stack approach with some parts of your infrastructure on a non-z/OS platform and some on z/OS. A project that created such a topology and compared it with a pure z/OS topology is described in the IBM White Paper The Mixed Platform Stack Project: Deploying a secure SOA solution into z/OS, WP101300, which can be obtained from the following Web page: http://www.ibm.com/support/techdocs In this Redbooks publication we describe only pure z/OS topologies. You should not, however, approach your choice of production topology in a dogmatic manner. There are good reasons to deploy on non-z/OS platforms, on z/OS, and on a mixed-stack. The correct choice for your organization depends on your unique requirements. These solutions are platform-independent, so they give you the flexibility to choose. 1.1.1 WebSphere Application Server for z/OS architecture A WebSphere Application Server for z/OS server has an architecture (Figure 1-1 on page 6) that includes a controller region, one or more servant regions, and a separate address space called the control region adjunct (or simply, the adjunct), which hosts the message engines. By providing an adjunct address space with a separate Java™ Virtual Machine (JVM™) for the message engines, the architecture of a single-cluster in WebSphere Application Server for z/OS is analogous to the two-clusters silver topology of a non-z/OS WebSphere Application Server described in Redbooks publication WebSphere Business Process Management V6.2 Production Topologies, SG24-7732. Chapter 1. Business Process Management production topologies for z/OS 5 n One WebSphere z/OS Server Servant Region Servant Region Servant Region HTTP Embedded HTTP Server Controller Region Messaging EJB container Servlets EJBs JSPs zWLM Adjunct Region SIB Web container Messaging Engine Web Services Engine WPM dispatch Data replication Dynamic cache Name server Security Figure 1-1 Architecture of a server in WebSphere Application Server for z/OS The simplest WebSphere Process Server for z/OS production topology comprising one cluster starts at the silver topology. It is not normally necessary to split the messaging engines into a separate cluster to achieve performance and scalability. The WebSphere Process Server for z/OS single-cluster topology is described in more detail in 1.3.1, “Single-cluster topology” on page 18. 1.1.2 Hardware considerations A WebSphere Process Server for z/OS production topology does not normally need to concern itself with the zSeries® hardware, except that more clusters will imply more JVMs and greater memory requirements. As documented in our experiments in this book, we use 64-bit WebSphere Application Server for z/OS servers. We do this primarily so we can have access to more JVM heap memory. When estimating CPU capacity it is important to remember that WebSphere Process Server for z/OS processing is primarily Java. When running in our WebSphere Application Server for z/OS-based environment where processes 6 z/OS: WebSphere Business Process Management V6.2 Production Topologies were deployed, we measured zAAP offload in the range of 60–70%, depending on workload. If CPU consumption on general purpose processors is a concern for you, consider deploying zAAP processors. Ensure that the zSeries machines in your parallel sysplex have enough capacity to support the topology you require. Specific CPU and memory requirements are best achieved through experimentation and rigorous benchmarking activities. The program directories for the products being deployed generally give good advice on where to start. 1.1.3 HTTP servers, LDAP servers, edge servers, proxies, external security managers, and so forth A production topology will include other non-WebSphere Process Server services such as one or more HTTP servers (for Web content) and firewalls. With WebSphere Process Server for z/OS, load balancing of HTTP requests across an HTTP Server running on z/OS is performed by Sysplex Distributor and is not something that needs to be considered in your topology design. You will, however, have to define one or more Distributed VIPAs to exploit the Sysplex Distributor function. If you are using a LocalOS, SAF-based user registry such as RACF®, the user registry is not something you need to consider in your WebSphere Process Server for z/OS production topology. It should already be configured with a RACF database shared across the parallel sysplex. However, if you decide to use LDAP, you must plan for high availability and disaster recovery if the LDAP is not running on z/OS. In an Internet environment it is normal to install a proxy (such as WebSeal) in the De-Militiarized Zone (DMZ), and to authenticate using an external security manager such as Tivoli® Access Manager. You will need to plan for the scalability and availability of any external authentication infrastructure. 1.1.4 Non-functional requirements This section addresses the following topics: Performance 64-bit addressability Scalability Recovery Availability Servicability Chapter 1. Business Process Management production topologies for z/OS 7 Performance Performance of the deployed architecture is certainly a factor in designing the topology to be deployed. Some of the factors involved are the same as those that drive other aspects of the design process. In general, on z/OS, we want to drive the utilization of a given CPU to as high a level as possible. We use various techniques to do this. One of the factors to consider is the performance profile of the application to be deployed. Can the application in the best possible configuration drive the CPU utilization to machine capacity? In our experience, the answer in the case of WebSphere and Java based products is that it depends. With good application design and implementation, it is possible to see good CPU utilization and overall good performance. In addition, the inherent design of the WebSphere Application Server for z/OS runtime promotes maximum CPU utilization with minimal configuration and manual intervention. More on this in “Scalability” below. Memory is also a concern in WebSphere-based processing. Many Java processes are by their nature, memory intensive. We therefore need to pay attention to how much physical/virtual memory we dedicate to a particular LPAR/WebSphere configuration. In many distributed architectures, it may make sense to base a design around a topology that spans LPARs/servers to fully exploit the installed memory of the servers involved. On z/OS the technology is robust, and memory management is not a concern. Because z/OS is designed to utilize all configured memory, the primary concern is not whether the machine uses the memory, but if there is enough installed memory available for all processes involved. While the major driving force behind memory utilization is Java heap size, we must also consider the memory requirements of the base operating system and address spaces. We also need to consider the physical machine architecture. Most modern WebSphere-based products use large amounts of Java heap. When running in 31-bit addressing mode, unlike distributed platforms, the System z architecture is limited to 31 bits. In this mode, the maximum available memory for a single z/OS address space is 2 GB. All memory for the address space must be served from this 2 GB address range. In practice, available heap in a 31-bit address space is approximately 768 MB. In many cases, 31-bit servers may be sufficient to run all of the deployed business processes. Additional memory utilization can be achieved through the deployment of additional servants unless the application sets themselves consume more memory than can fit in a single 31-bit JVM. In these cases it may be necessary to deploy 64-bit servants, as described in “64-bit addressability”. 8 z/OS: WebSphere Business Process Management V6.2 Production Topologies WLM workload classification is another way to isolate large applications into specific WebSphere servants. This technique would allow applications with large Java memory requirements potentially to be isolated in fewer 64-bit JVMs. 64-bit addressability Modern machine architectures are 64 bit. System z is no exception. Modern system z processors use a 64-bit architecture. WebSphere and all of the BPM products we will be discussing in this IBM Redbooks publication are fully enabled for 64-bit through their use of a 64-bit JVM. In some cases, deploying large business processes or large numbers of smaller business processes may force the need for a 64-bit servant to be defined. Because 64-bit processing uses more system resources than 31-bit processing, careful analysis of the deployed workloads should take place before deciding on 64-bit servants. Note: 64-bit servers are not the default for WebSphere Application Server for z/OS V.6.1. In our examples we first deployed 31-bit servers. Because our environment is not constrained in terms of CPU or memory, we decided to switch our servants to 64-bit addressing mode. We left all other server components in 31-bit addressing mode. Scalability Scalability of WebSphere-based architectures is achieved on distributed platforms by building vertical clusters. By adding additional cluster members on the same physical machine, distributed architectures can approach 100% CPU utilization on a single platform. Scalability of WebSphere on z/OS is achieved by adding more servants to a single deployed WebSphere Application Server. Rather than adding more cluster members across physical nodes, WebSphere in conjunction with the z/OS operating system automatically adds and subtracts JVMs and servant capacity on demand. This is the normal practice when scaling WebSphere-based processes on a z/OS platform and is an entirely automatic process. You should consider this default behavior when designing your WebSphere Process Server for z/OS topology. Recovery Recovery of architecture components should be a factor in designing a BPM topology. The z/OS operating system and installed infrastructure components do a good job of exploiting the underlying hardware and software facilities for recovery and first failure data capture. The design of the WebSphere Application Server for z/OS also helps. Chapter 1. Business Process Management production topologies for z/OS 9 On z/OS, the basic tenant is to keep the operating system alive at all costs. In the name of OS survivability, we will sacrifice almost everything. The BPM portfolio products that run on top of WebSphere Application Server for z/OS inherit all of the QoS of the base WebSphere design: WebSphere Application Server for z/OS is designed with a clear separation of concerns with respect to where code runs. Operating system code and code that has access to the operating system runs authorized in a controller address space. Application code. By definition, WebSphere, WebSphere-based products, and anything that runs in a JVM runs in its own execution environment, called a servant address space. Code running in the controller is completely isolated from code running in the servants. Communication between the processes is through WLM-managed work queues. In the event of an application error, a particular servant may end abnormally. It will, in most cases, be restarted automatically by the controller in place. Various additional recovery actions are possible based on individual requirements. Failures associated with an application are therefore isolated to the servant where the error occurred. In the event of a controller failure, z/OS policy-based recovery, managed by a z/OS component called the Automatic Restart Manager, can restart the controller in place, or on another system in the z/OS sysplex. In the unlikely event of a failure of the z/OS operating system, additional recovery options for surviving members of the z/OS sysplex are available. Robust WebSphere-based architectures consider the factors listed above in the application of technology in the z/OS environment. The bottom line recommendation is that architectures on z/OS do not have to be overly complex to deal with recovery actions and system failures that might occur on other platforms. Availability In general, you only need multiple cluster members on z/OS to provide greater availability. It is natural to think that two is enough. However, it is better to use three than two. This is known as the Rule of Three. The reason for preferring three is as follows. When you run multiple cluster members, you are protecting against the failure of the entire logical partition (LPAR), or of some singleton address space on which the cluster member depends. If the controller region fails, you lose the capacity of that cluster member. When you lose one cluster member, the remaining cluster member must take up the entire load of the failing cluster member in 10 z/OS: WebSphere Business Process Management V6.2 Production Topologies addition to its own. A sudden increase in load from 50% capacity to 100% may cause the remaining cluster member to fail too. When there are three cluster members, however, and one fails, there are two remaining cluster members to absorb 33% of the workload from the failed cluster member. This means a growth of only about 17% in the workload of each cluster member, which may not cause a problem. On z/OS, when workload suddenly shifts to another cluster member, the effect is not as bad as it would be on a non-z/OS platform, because the re-assignment of resources to the remaining cluster by zWLM helps it deal with the extra workload. If you have configured the cluster so the number of servants vary with the workload, the impact of one cluster member failing may not cause a problem for one remaining cluster member. However there will still be one controller region processing all the communications. You can run into a bottleneck with the number of SSL threads in the controller, for example. For this reason, although you may never encounter a situation where two cluster members is not adequate, if you have the flexibility in your system configuration, we recommend three cluster members for maximum availability and stability, even on z/OS. Note: The focus of this IBM Redbooks publication is on configuration and deployment. Therefore, in our examples throughout this IBM Redbooks publication we will use a two node configuration. Servicability Reality is that in many cases, debugging complex problems will require additional resources and in some cases, problem recreates. A successful architecture will therefore be adaptable to service requirements and components and allow for flexible data capture and problem resolution. Chapter 1. Business Process Management production topologies for z/OS 11 1.1.5 Summary When planning your WebSphere Process Server for z/OS production topology, consider the following factors: The CPU required and the memory required to support all the JVMs. Consider whether to use 64-bit versus 31-bit to allow for sufficient Java heap. The number of clusters and cluster members required to support your business and availability requirements. For situations where continuous availability is required, clusters with three cluster members are best. The number of servants per cluster member required to support the workload. If using LDAP, the network communication, high availability configuration, and disaster recovery requirements of the LDAP server. The number and location of any HTTP servers. Whether you should use WebSphere MQ for z/OS as a JMS provider. Servicability requirements and additional capacity required to accommodate recreates for data capture. 1.2 WebSphere Process Server for z/OS A number of components are created and used when configuring a WebSphere Process Server for z/OS topology. These components are discussed in this section. 1.2.1 Databases A production WebSphere Process Server for z/OS cluster should use DB2 for z/OS rather than Derby to hold, store, and track information. You can choose to define all the tables for all components of WebSphere Process Server for z/OS in one database, or create databases for each component. Unlike WebSphere Process Server on non-z/OS platforms, you do not need to use multiple databases for performance reasons. When using DB2 for z/OS, it is more a matter of administrative convenience to do this. There is no need to consider the physical location of databases when selecting your production topology because they are all in DB2 for z/OS. 12 z/OS: WebSphere Business Process Management V6.2 Production Topologies Whether you choose one or two clusters, you must configure DB2 for z/OS in data sharing mode so that all cluster members can access the same data. For our BPM reference topology for z/OS in MD cell we defined databases as follows: Common database (MPWPSDB) Used as a repository for various components in WebSphere Process Server, this database needs to be created prior to starting WebSphere Process Server. It persists information regarding the following components: – – – – – – Application Scheduler Business Rules Mediations Recovery Relationships Selectors Business Process Choreographer (BPC) database (MDBPCDB) Used by the Business Flow Manager and the Human Task Manager, this database needs to be created prior to starting BPC components. Messaging engine database (MDSIBDB) These databases are used by the SCA system and application buses, the CEI bus, the BPC bus, and the intercell bus used by Monitor for cross cell communication of business events. We created each bus in a separate database because their Data Definition Language (DDL) uses the same table space names. It also allowed us to assign different bufferpools to each bus if required more easily. Event database (MDCEIDB) This database persists information regarding the Event Service such as Common Based Events and key performance indicators. Business Space database (MDBSPDB This database is used by the Business Space component for configuration information associated with deployed widgets. Fabric database (MDFABDB This database is used by the WebSphere Business Services Fabric product for RESTful services and Fabric widgets. Chapter 1. Business Process Management production topologies for z/OS 13 1.2.2 Service integration buses A service integration bus is a managed communication mechanism that supports service integration through synchronous and asynchronous messaging. A bus consists of interconnecting messaging engines. WebSphere Process Server makes use of the following service integration buses: SCA system bus This bus is used to host queue destinations for SCA modules. The SCA runtime uses these queue destinations to support asynchronous interactions between components and modules. SCA application bus This bus supports the asynchronous communication between WebSphere Business Integration Adapters and other SCA components. Common Event Infrastructure (CEI) bus This bus is used to transmit common base events asynchronously to a CEI server. BPC bus This bus is used for transmitting messages internally in the Business Flow Manager. Fabric Bus This bus is used for transmitting messages internally to the Fabric Performance Monitor. Monitor CEI bus This bus is used for communicating business events to WebSphere Business Monitor running in another cell. On non-z/OS platforms, experience has shown that running the message engines in the same cluster as the rest of WebSphere Process Server can cause a bottleneck. Therefore, a lot of emphasis is placed on ensuring that the messaging infrastructure runs in its own cluster. However, as discussed in 1.1.1, “WebSphere Application Server for z/OS architecture” on page 5, the fact that the message engines run in adjunct regions obviates the need to configure them in a separate cluster. 14 z/OS: WebSphere Business Process Management V6.2 Production Topologies 1.2.3 Messaging engines Messaging engines are servers used to provide asynchronous messaging support for your applications and for the internal messaging needs of the WebSphere Process Server components, such as the internal navigation queues used by long running business processes. On non-z/OS systems there are well-known advantages to configuring messaging engines in a separate cluster, but with WebSphere Process Server for z/OS there is probably no reason to do this. Message engines already run in separate adjunct regions, which is like running them in a separate cluster. Messaging engines can use two different types of message store: File-based Database Our z/OS implementation always uses database message stores backed in DB2 z/OS. 1.2.4 WebSphere MQ for z/OS queue sharing A unique feature of WebSphere Process Server for z/OS is the ability to use WebSphere MQ for z/OS as a JMS provider. In a clustered environment, the ability to use shared queues with WebSphere MQ for z/OS allows you to run multiple instances of a message engine without the disadvantages of partitioning the queues. If your messaging workload exceeds the capacity of one adjunct region, you can consider configuring a WebSphere MQ for z/OS JMS provider using the bindings mode for optimal performance and security. 1.2.5 Business Process Choreographer Business Process Choreographer (BPC) is an enterprise workflow engine that supports both business processes and human tasks. The core of the BPC configuration consists of the following components: Business Flow Manager The Business Flow Manager provides services to run business processes within an application server. Human Task Manager The Human Task Manager provides services to run human tasks within an application server. Chapter 1. Business Process Management production topologies for z/OS 15 Applications that make use of Business Flow Manager or Human Task functions must be deployed in a cluster where BPC has been configured. In the two-clusters topology, BPC is configured in the AppTarget cluster. This was the mdcl02.AppTarget cluster in our MD cell. 1.2.6 Common Event Infrastructure Common Event Infrastructure (CEI) is an embeddable technology intended to provide event management services to applications that require those services. For service component event points that you monitor, events can be published to the CEI server and stored in the CEI database. Depending on the extent CEI is used by applications, it may be necessary to define CEI in a separate cluster. In the two-clusters topology CEI is configured in the support and messaging cluster, which is remote from the cluster hosting the applications. That is probably enough separation. In the BPM reference topology for z/OS, all bus are defined in the same cluster. For additional capacity, rather than build another cluster, it might make more sense to define the message engines using MQ shared queues. 1.2.7 WebSphere Process Server Applications WebSphere Process Server for z/OS provides a variety of Web-based application tools, such as the following: BPC Explorer BPC Explorer implements a generic user interface for interacting with business processes and human tasks. It is typically used to initiate and test business processes. Business rules manager (BRM) BRM assists business analysts in browsing and modifying business rule values. In addition to these WebSphere Process Server-specific applications, Business Space powered by WebSphere can be used to interact with WebSphere Process Server for z/OS. Business Space is a browser-based, graphical interface included in WebSphere Process Server for z/OS that allows authorized users to create, manage, and integrate Web interfaces across the IBM BPM Suite. 16 z/OS: WebSphere Business Process Management V6.2 Production Topologies These applications are all support functions, and therefore, in a distributed implementation, would be deployed into a support and messaging cluster in the two-clusters topology. On z/OS we are using a single cluster topology and will deploy these functions there in the single cluster. 1.3 Deployment environments When using WebSphere Process Server on non-z/OS platforms, a deployment environment can be created using IBM-supplied deployment environment patterns. The deployment environment patterns included in the Integrated Solutions Console and the profile management tool represent the most common deployment environments that customers require. Each of the patterns centers around the number of WebSphere Process Server clusters and cluster members. Think of deployment environment patterns as wizards that define a chosen topology. With WebSphere Process Server for z/OS, the deployment environment patterns are not useful because the patterns they create are probably not what you want to use on z/OS. Instead, configure each component of WebSphere Process Server for z/OS into the appropriate cluster using scripts or the wizards located on the cluster configuration panels. WebSphere Process Server for z/OS contains three basic sets of functions that together form a complete production environment. Each of these functions can be separated into individual, dedicated clusters, but on z/OS it is best to start by assuming they are deployed in the same cluster. The three sets of functions in the WebSphere Process Server environment are as follows: Application deployment target The application deployment target is the set of servers to which you install your applications (human tasks, business processes, mediations, and so forth). Supporting infrastructure The supporting infrastructure is the CEI and other infrastructure services used to support your environment, such as the BPC Explorer, Business Rules Manager and Business Space. Messaging infrastructure The messaging infrastructure is the set of servers used to provide asynchronous messaging support for your applications and for the internal messaging needs of the WebSphere Process Server components, such as the internal navigation queues used by long running business processes. Chapter 1. Business Process Management production topologies for z/OS 17 On non-z/OS systems there are well-known advantages to configuring the messaging infrastructure in a separate cluster but with WebSphere Process Server for z/OS, there is probably no reason to do this. Message engines already run in separate adjunct regions, which is like running them in a separate cluster. With WebSphere Process Server for z/OS, the main topology decision you face is whether it is worth splitting major application functions, like WebSphere Business Services Fabric for z/OS, into an AppTarget cluster, while leaving the majority of the WebSphere Process Server for z/OS function deployed in the support and messaging cluster. The single-cluster and two-clusters topologies are described in more detail in 1.4, “Selecting an appropriate topology” on page 24. The characteristics of these topologies are compared to help you choose between them. 1.3.1 Single-cluster topology The single-cluster topology pattern provides one cluster for all the functional components. The user applications, messaging infrastructure, CEI, and support applications are all configured in the same cluster. On non-z/OS platforms, this topology is referred to as the bronze topology. Typically, the bronze topology is suggested for testing, proofs of concept, and demonstration environments. However, on z/OS, a single-cluster topology is a viable production topology. This is because of the different architecture of WebSphere Application Server for z/OS. As mentioned in 1.1.1, “WebSphere Application Server for z/OS architecture” on page 5, the fact that the message engines run in a separate adjunct region means that the drawbacks of the bronze topology do not apply to WebSphere Process Server for z/OS. A single-cluster topology on z/OS has characteristics that are analogous to those of the non-z/OS silver topology. There are other operational advantages, however, such as zWLM-managed servant regions, which make a single-cluster on z/OS what we call a polished silver topology. A single-cluster topology sample configuration for WebSphere Process Server is shown in Figure 1-2 on page 19. 18 z/OS: WebSphere Business Process Management V6.2 Production Topologies Controller MDSR1A Controller MDSR1B Servant MDSR1AS (WPS) Servant MDSR1BS (WPS) BPC (HTM and BFM) CEI BPC Explorer BRM Business Space BPC (HTM and BFM) CEI BPC Explorer BRM Business Space CEI ME BPC ME mdnodeb CEI Bus mdnodea SCA.SYSTEM Bus Adjunct MDSR1BA BPC Bus Adjunct MDSR1AA SCA.APPLICATION Bus SCA APP SCA SYS ME ME Single Cluster mdsr1 Figure 1-2 WebSphere Process Server for z/OS single-cluster topology Note the following aspects of this example: All of the components are configured in a single cluster. In our single-cluster topology cell MD this cluster is called mdsr1. The mdsr1 cluster is a member of all four of the required WebSphere Process Server buses: – – – – SCA.SYSTEM bus SCA.APPLICATION bus CEI bus BPC bus The message engines execute in the adjunct regions. Each cluster member has a Business Process and Human Task container. All of the supporting infrastructure applications are configured in the cluster: – – – – BPC Explorer Business Rules Manager CEI Business Space Chapter 1. Business Process Management production topologies for z/OS 19 Each cluster member is an application deployment target. In Figure 1-2 on page 19, the messaging engines are shown running in specific adjuncts, as shown in Table 1-1. Table 1-1 Messaging engines and their adjuncts, as shown in Figure 1-2 on page 19 Adjunct Messaging engine MDSR1AA SCA.SYSTEM and SCA.APPLICATION MDSR1BA CEI and BPC This is not the default configuration. By default, each cluster member is capable of running all four of the messaging engines, and the server that starts first automatically runs all four of the engines. You can configure different service integration bus policies to control where message engines run. For more information, see Redbooks publication WebSphere Business Process Management V6.2 Production Topologies, SG24-7732. Each server scales by running multiple servants, not by adding more cluster members. More than one cluster member is required for availability reasons (and three is a good number) but not for scalability reasons. The behavior of the messaging engines in a single-cluster topology is different than in a remote cluster. When the messaging engines and the applications are collocated, the default behavior is for message producers and consumers to use a local active messaging engine if one is available. Thus, if you have two applications deployed to each cluster member that need to communicate asynchronously, once each message producer places messages in the queues, the message consumer on the machine where the engine is local consumes all of the messages produced. The consuming application only processes messages on the server with the local messaging engine. Read and write local also creates a unique set of issues if you attempt to partition the destinations. When you create more than one active set of messaging engines, partitioning results. Each server’s active messaging engines contain a portion of the queues assigned to that engine. Unfortunately, although you can attain additional throughput if there are active messaging engines on each server, partitioning can create issues for your applications. If you partition destinations when the applications and messaging engines are in the same cluster, you will not be able to maintain message order. This is true even if you attempt to enable event sequencing in WebSphere Process Server. Partitioned destinations can also create unpredictable behavior if one or more messaging engines fails in a single-cluster topology. If you are prepared to 20 z/OS: WebSphere Business Process Management V6.2 Production Topologies endure unpredictable behavior and loss of message order, partitioning the destinations in a single-cluster topology may be acceptable. However, this configuration is discouraged. Note: For information about workload sharing with queue destinations, refer to the WebSphere Application Server for z/OS Information Center at the following Web page: http://publib.boulder.ibm.com/infocenter/wasinfo/v6r1/index.jsp?topi c=/com.ibm.websphere.pmc.zseries.doc/concepts/cjt0007_.html 1.3.2 Two-cluster topology The two-clusters topology pattern is analogous to the Remote Messaging and Remote Support topology pattern for WebSphere Process Server on non-z/OS platforms where it is also known as the gold topology. For non-z/OS platforms this is the preferred topology for production environments, but on z/OS there may be no need to define multiple clusters for performance reasons. We considering deploying this topology, ultimately deciding not to do so. Instead, we directed our focus towards the BPM reference topology for z/OS, highlighted in 1.5, “BPM reference topology for z/OS and how we will use it in this book” on page 30. We present the two-cluster topology here for completeness and to offer the reader an additional choice of deployment topologies to consider. Refer to 1.4, “Selecting an appropriate topology” on page 24, for more explicit considerations for choosing a deployment topology. Figure 1-3 on page 22 shows a MP cell, which is comprised of two clusters: mpsr2.WPS_SandM (providing the support and messaging function), and mpsr2.AppTarget (running BPC and the applications). Configuring this topology in WebSphere Process Server for z/OS is no different from configuring the single-cluster topology. It is largely a matter of invoking the appropriate wizard in the correct cluster. For our MD cell we only used two nodes. Figure 1-3 on page 22 shows three nodes to illustrate the possible assignment of the buses to specific adjuncts. Also, for availability reasons, we recommend using three nodes with three cluster members in “Availability” on page 10. Chapter 1. Business Process Management production topologies for z/OS 21 Controller MDSR2A Controller MDSR2B Servant MDSR2AS BPC (HTM and BFM) BPC Explorer Servant MDSR2BS BPC (HTM and BFM) BPC Explorer Adjunct MDSR2AA Adjunct MDSR2BA AppTarget Cluster mdsr1.AppTarget BPC ME SCA APP ME SCA SYS ME CEI ME Adjunct MDSR1AA Adjunct MDSR1BA mdnodea mdnodeb CEI Bus Servant MDSR1BS (WPS) CEI BRM Business Space SCA.APPLICATION Bus Servant MDSR1AS (WPS) CEI BRM Business Space Support and Messaging Cluster mdsr1.WPS_SandM SCA.SYSTEM Bus Controller MDSR1B BPC Bus Controller MDSR1A Figure 1-3 WebSphere Process Server for z/OS in the two-clusters topology Note the following aspects of this example: All of the applications are deployed to the mpsr2.AppTarget cluster. The BPC is configured in the mpsr2.AppTarget cluster, so each cluster member has a Business Process Container and a human task container. The mpsr2.WPS_SandM cluster is a member of three of the four required WebSphere Process Server for z/OS buses: – SCA.SYSTEM bus – SCA.APPLICATION bus – CEI bus The mpsr2.AppTarget cluster is a member of the BPC bus. 22 z/OS: WebSphere Business Process Management V6.2 Production Topologies The BPC bus can be defined locally in the same cluster as BPC because the BPC message engine runs in an adjunct, so there is no performance concern with running it in the same cluster. The supporting infrastructure applications are configured in the mpsr2.WPS_SandM cluster: – – – – BPC Explorer Business Rules Manager CEI Business Space Remote SCA must be configured in the mpsr2.AppTarget cluster. Note: The BPC Explorer would normally be deployed in the support and messaging cluster too, but the wizard only allows you to deploy BPC Explorer into a cluster that runs the Business Process Container. Therefore, it is shown deployed in mpsr2.AppTarget. In Figure 1-3 on page 22, the messaging engines are shown running in the specific adjuncts shown in Table 1-2. Table 1-2 Messaging engines and their adjuncts, as shown in Figure 1-3 on page 22 Adjunct Messaging engine MDSR01AA SCA.SYSTEM and SCA.APPLICATION MDSR01BA CEI and BPC This is not the default configuration. You can customize a Service Integration Policy to assign a preferred server for the active message engine. As with the single-cluster topology, each server scales by running multiple servants, not by adding more cluster members. More than one cluster member is required for availability reasons (and three is a good number) but not for scalability reasons. The two-clusters topology allows you to add servants to cluster members in the AppTarget cluster. This is to assist in adding capacity to the applications, while keeping the capacity of the support and messaging infrastructure the same (or vice versa). The behavior of the messaging engines in the two-clusters topology is different from the behavior when the messaging engines are collocated with the applications. Because the messaging engines are in a remote cluster with respect to the applications, there is no preference for the message producers and consumers to use a local messaging engine. Each member of the mdcl02.AppTarget cluster, for example, connects to the appropriate bus and uses the remote messaging engine for that bus. Chapter 1. Business Process Management production topologies for z/OS 23 As with a single-cluster topology, if you partition destinations when the applications and messaging engines are in separate clusters, you will no longer have the ability to maintain message order. Any time you partition destinations you lose message order. This is true even if you attempt to enable event sequencing in WebSphere Process Server for z/OS. In addition, partitioned destinations can create additional issues when the messaging engines are remote. By default, you have no control over which active messaging engine your applications will use at run time. This can create situations where two applications on the same server attach to two different messaging engines. If one application produces messages for one engine and the message consumer is using a different engine, stranded messages can result. For these reasons, partitioned destinations are strongly discouraged in the two-clusters scenario. 1.3.3 Custom topologies If neither of these two topologies is suitable to your needs, you can create a custom topology. However, we do not recommend this unless you have a good reason to depart from one of the two topologies presented here. There are performance advantages to be gained by placing everything in one cluster and it is always a good idea to keep it simple. Creating a custom topology with WebSphere Process Server for z/OS involves defining the number of clusters you want and deploying the components in the appropriate cluster. You must define remote SCA support in clusters that are remote from the SCA message engines. If you choose to implement a custom topology pattern, it is generally unwise to move support components into non-default locations, as this structure will likely impact application performance. 1.4 Selecting an appropriate topology Selecting an appropriate topology for your production environment depends upon several factors, including those listed below: Available hardware resources Application invocation patterns Types of business processes you plan to implement (interruptible versus non-interruptible) How heavily you intend to use the CEI Scalability requirements Administrative effort involved 24 z/OS: WebSphere Business Process Management V6.2 Production Topologies On non-z/OS platforms, where resources are typically not shared between servers because they are running on different machines, it is important to consider the characteristics of all the hardware when planning your topology. On z/OS however, CPU, memory, data, message queues, and security profiles can all be shared between the logical partitions of a parallel sysplex. Therefore, while you need to plan to have sufficient total capacity on your zSeries machines to support your workload, the topology of the hardware is not something that greatly influences your WebSphere Process Server for z/OS topology choice. If you are able to use WebSphere MQ as a message provider, the potential bottleneck of running one message engine in an adjunct disappears. For this reason, if your applications make heavy use of messaging, it is more advantageous to use WebSphere MQ as a message provider than to split the message engines into a separate cluster or consider partitioning. Our recommendation is to keep the number of clusters to a minimum and think carefully before using more than one. We did not describe the two-clusters topology in response to any known performance problem with the single-cluster topology in WebSphere Process Server for z/OS. Our motivation was related to availability and operability, as explained in 1.3.2, “Two-cluster topology” on page 21. Although the two-clusters pattern has some advantages, they come at the cost of remote access to the SCA buses. We were unable to perform performance tests to quantify this, and it is possible that the overhead of accessing SCA remotely offsets any benefits of having applications in a separate cluster. In addition, multiple clusters means more address spaces, which means greater CPU and memory requirements. As you plan for your production environment, consider the advantages and disadvantages of single-cluster compared to two-cluster. These topologies are now compared in more detail. 1.4.1 The single cluster pattern With WebSphere Process Server for z/OS, a single-cluster topology may be suitable for your production environment, but because each server instance must run both the infrastructure and your applications, the heap requirements of the servant JVMs is more likely to necessitate the use of the 64-bit mode than the two-clusters topology. With WebSphere Process Server for z/OS, you can configure zWLM so that asynchronous interactions are routed to different servants from those running synchronous requests, minimizing the impact of asynchronous requests without having to resort to a separate cluster. Thus, unlike non-z/OS systems, where a Chapter 1. Business Process Management production topologies for z/OS 25 single cluster topology is typically used for proof of concept, development, and testing environments, a single cluster is a practical production topology on z/OS, and is in fact the preferred topology choice. From an administrative and scalability perspective, the single-cluster topology has advantages. A single cluster where each member runs all the WebSphere Process Server components is easier to administer. Instead of several server instances in multiple clusters, you have a single cluster with fewer members. Remember that with WebSphere Process Server for z/OS, when the needs of your environment grow, you scale by running more servants. It is not necessary to configure additional nodes and cluster members. The number of servants can be set to vary dynamically depending on the workload, so it is not necessary to manually add capacity in response to increased workload. In a non-z/OS environment, when additional nodes or cluster members are added, all components are scaled at the same rate. For example, each additional cluster member adds more CEI processing whether you need it or not. If you have the messaging engines spread across server members using service integration policies, there is some additional administrative effort in creating and maintaining the policies. With WebSphere Process Server for z/OS, however, adding additional servants provides more capacity for the business workload without adding additional infrastructure overhead. Expanding the cluster beyond three cluster members has no affect on message processing capability because the one-of-n high availability policy limits message processing capacity to the adjunct that is running the active message engine. The other adjuncts host message engines that act as stand-bys. Therefore, use service integration policies to spread the active message engines across the three cluster members, as shown in Figure 1-2 on page 19. 1.4.2 The two cluster pattern As discussed in the opening comments of 1.4, “Selecting an appropriate topology” on page 24, we did not describe the two-clusters topology in response to any performance problem with the single-cluster topology. Our motivation was related to availability and operability, specifically the following two concerns: WebSphere Business Services Fabric for z/OS will add to the startup times of a cluster member. On our system, a cluster member in the single-cluster topology took six minutes to start, while the support and messaging cluster member took three minutes to start and the AppTarget cluster member took three minutes in the two-clusters topology. The applications will probably have different availability needs compared to the support and messaging infrastructure. By placing WebSphere Business Services Fabric for z/OS, BPC, and applications in a separate cluster, those cluster members can be restarted separately from the support and messaging infrastructure. 26 z/OS: WebSphere Business Process Management V6.2 Production Topologies Creating a second cluster, each with its own functions and applications, creates some additional administrative burden, but this is not excessive on z/OS. When you add cluster members and clusters on non-z/OS platforms, your performance tuning plan and the troubleshooting burden can expand greatly, but on z/OS you can classify different workloads using zWLM and obtain detailed performance data on each workload. The topology does not affect your ability to monitor your workloads and give them correct priorities. Spreading messaging engines across the members of the messaging cluster adds the administrative burden associated with creating and maintaining policies, but this is mostly a one-off activity. As was stated earlier, in WebSphere Process Server for z/OS you may not need to add a second cluster for scalability reasons, but having two clusters allows you to scale the cluster members hosting applications separately from those hosting the support messaging infrastructure. Expanding the support and messaging cluster beyond three cluster members has no affect on message processing capability. The scalability of message processing is limited by the workload that can be processed by one adjunct, so the best you can do is spread the active message engines across three cluster members. In summary, the two-clusters topology in WebSphere Process Server for z/OS offers all the advantages of the Remote Messaging and Remote Support topology on non-z/OS platforms. Because the application target cluster is only responsible for running your business integration applications, performance tuning and diagnostics are much simpler than in a single-cluster topology where one cluster runs everything. The two-clusters topology is also ideal for environments that make extensive use of CEI for monitoring and auditing, because CEI processing does not take place in the same cluster as the applications. 1.4.3 Custom patterns You can create a custom topology by defining as many clusters as you want and launching the wizards that configure each component of WebSphere Process Server for z/OS on the appropriate cluster. The process of creating such a topology is essentially the same as that for the two-clusters topology we describe in this Redbooks publication. You might want to depart slightly from the topologies we describe if your organization has no need for some of the WebSphere Process Server for z/OS components. If you know you will not use the CEI, for example, you can create a topology that does not include CEI support. Similarly, if your organization has Chapter 1. Business Process Management production topologies for z/OS 27 governance rules that prevent you from taking advantage of the Business Rules Manager, you can remove it from your topology. These decisions do not significantly affect the choice of topology, except that when you configure fewer components it tends to make the single-cluster topology more practical. In summary, consider a custom topology for good reasons. A single-cluster topology with WebSphere Process Server for z/OS is good, and it is possible to imagine reasons to adopt the two-clusters topology described in this Redbooks publication, but topologies with more clusters than this are animals that should be approached with extreme caution. 1.4.4 Condensed pattern selection criteria Table 1-3 provides a condensed list of the advantages and disadvantages to each of the topology patterns. Consider the information listed in Table 1-3 as a quick guide to selecting your production topology. Table 1-3 Topology selection considerations Consideration Single cluster topology Two-cluster topology Number of clusters to maintain One cluster for all components One cluster for applications One cluster for the support and messaging infrastructure Hardware requirements Minimum footprint for a high-availability production environment More address spaces means more JVMs and more real storage requirements 64-bit addressing None. Servers running in 64-bit addressing mode have access to more Java heap. Additional servers running in 64-bit addressing mode have access to more Java heap and thus require more physical memory to be configured to support the topology. Asynchronous interactions Use z/WLM to isolate synchronous and asynchronous interactions Use z/WLM to isolate synchronous and asynchronous interactions Cross-server communication overhead None. Everything is running in the same cluster members Some overhead in accessing remote SCA buses from the application cluster Long running processes, state machines and human tasks No issue. zWLM is used to assign appropriate priorities No issue. zWLM is used to assign appropriate priorities 28 z/OS: WebSphere Business Process Management V6.2 Production Topologies Consideration Single cluster topology Two-cluster topology Heavy CEI activity Heavy CEI use may impact applications Suitable environment for heavy CEI use including remote CEI connections for WebSphere Business Monitor Scalability Easy to scale by adding servant regions. zWLM manages the distribution of work across the servants and can dynamically control the number of servants in response to the workload. Easy to scale by adding servant regions. zWLM manages the distribution of work across the servants and can dynamically control the number of servants in response to the workload. Allows selective scaling of either the application cluster or the support and messaging cluster Chapter 1. Business Process Management production topologies for z/OS 29 1.5 BPM reference topology for z/OS and how we will use it in this book The BPM reference topology for z/OS uses as its base the single cluster pattern described in 1.4.1, “The single cluster pattern” on page 25 and referenced below in Figure 1-4. Controller MDSR1A Controller MDSR1B Servant MDSR1AS (WPS) Servant MDSR1BS (WPS) BPC (HTM and BFM) CEI BPC Explorer BRM Business Space BPC (HTM and BFM) CEI BPC Explorer BRM Business Space CEI ME BPC ME mdnodeb Figure 1-4 BPM reference topology for z/OS 30 z/OS: WebSphere Business Process Management V6.2 Production Topologies CEI Bus mdnodea SCA.SYSTEM Bus Adjunct MDSR1BA BPC Bus Adjunct MDSR1AA SCA.APPLICATION Bus SCA APP SCA SYS ME ME Single Cluster mdsr1 To this pattern we add several components as specified in Figure 1-5. HiperSockets & Virtual Networking and Switching Tivoli Management Console WebSphere Business Monitor Linux MQ SysAdmin Tools Apache Linux CMS Linux mdnodea mdnodeb WAS / WPS / Fabric DB2 MQ IHS WAS / WPS / Fabric DB2 MQ IHS TDS Sysplex Distributer z/OS z/OS z/VM Control Program I/O & Network Memory Memory IFL Processors LPAR Memory Processors LPAR LPAR Processor Resource / System Manager (PR/SM) Figure 1-5 BPM reference topology for z/OS The components and products we use in this book are as follows: WebSphere Business Services Fabric for z/OS We add the required components and facilities to enable Fabric functions on z/OS. We extend the ITSO supplied sample application to support business services deployed in the Fabric. WebSphere MQ shared queues We modify the Messaging Engines specified in the two cluster pattern to be backed by WebSphere MQ shared queues to provide better throughput and scalability. WebSphere Business Monitor We deploy WebSphere Business Monitor in a Linux for System z LPAR. We also enable additional CEI buses in our production WebSphere Process Chapter 1. Business Process Management production topologies for z/OS 31 Server for z/OS cell to enable cross cell communication to WebSphere Business Monitor. Tivoli Composite Application Manager We deploy the Tivoli Management Server in a Linux for System z LPAR. We add two ITCAM data collectors to our WebSphere Process Server for z/OS production cell. The Tivoli Composite Application Manager for WebSphere z/OS 6.1 data collector and the Tivoli Composite Application Manager for Response Time Tracking for z/OS provide management capabilities for our production cell. IBM HTTP Server We deploy an HTTP server in each of the WebSphere nodes and generate a plugin configuration so each node talks to its corresponding HTTP server. Apache Web server We configure an Apache Web server in a Linux for System z LPAR to act as a proxy to the HTTP servers on the WebSphere server nodes. Sysplex Distributer We configure Sysplex Distributer on each of the z/OS nodes to route requests to an available WebSphere node for processing. Federated security We initially configure security for our cell to be based on RACF and z/OS security. When we add our Linux based components, we must extend security to them. We do this by deploying a Tivoli Directory Server on z/OS. We use VMM to federate security using Tivoli Directory Server. We use the remainder of this book to explain the configuration steps that we undertook to build out the BPM reference topology on z/OS. 1.6 Incorporating WebSphere Business Services Fabric for z/OS into the BPM reference topology for z/OS A WebSphere Process Server for z/OS production topology can also include other products from the WebSphere business integration portfolio. Chapter 5, “WebSphere Business Services Fabric for z/OS” on page 201 presents information about configuring WebSphere Business Services Fabric for z/OS in the BPM reference topology for z/OS. 32 z/OS: WebSphere Business Process Management V6.2 Production Topologies 1.6.1 WebSphere Business Services Fabric for z/OS modules WebSphere Business Services Fabric for z/OS contains two software packs: Business Services Foundation Pack The Business Services Foundation Pack is a set of runtime services and facilities that provide the management and runtime environment for the WebSphere Business Services Fabric for z/OS application set. Business Service Tool Pack. The Business Services Tool Pack is a set of workstation-based tools that provide the design time tools and environment for creating and assembling applications for deployment in the WebSphere Business Services Fabric for z/OS environment. The tool pack provides tooling necessary for creating and deploying content into WebSphere Business Services Fabric for z/OS, but as it runs only on the workstation, we will only provide limited coverage of it here. WebSphere Business Services Fabric for z/OS also includes industry-specific solutions called Industry Content Packs. Five of these content packs are supported for z/OS: Banking Payments Content Pack Healthcare Payer Content Pack Insurance Property and Casualty Content Pack Product Lifecycle Management Content Pack Telecom Operations Content Pack Content packs provide pre-built industry solutions and content that can jumpstart projects allowing customers to create service-oriented assets. The runtime environment provided by the Business Services Foundation Pack is the focus of Chapter 5, “WebSphere Business Services Fabric for z/OS” on page 201. We use the topology outlined in Figure 1-6 on page 34 to further our discussion. 1.6.2 WebSphere Business Services Fabric for z/OS When WebSphere Business Services Fabric for z/OS is added to the single cluster topology, the MDSR1 cluster is a member of the WebSphere Business Services Fabric bus (fabricbus), and the WebSphere Business Services Fabric core applications are added to the Single Cluster.mpcl02.WPS cluster. This topology is illustrated in Figure 1-6. Chapter 1. Business Process Management production topologies for z/OS 33 Controller MDSR1A Controller MDSR1B Servant MDSR1AS (WPS) Servant MDSR1BS (WPS) BPC (HTM and BFM) CEI BPC Explorer BRM Business Space Fabric BPC (HTM and BFM) CEI BPC Explorer BRM Business Space Fabric Fabric ME Fabric Bus mdnodeb CEI Bus Adjunct MDSR1BA BPC Bus mdnodea BPC ME SCA.SYSTEM Bus Adjunct MDSR1AA CEI ME SCA.APPLICATION Bus SCA APP SCA SYS ME ME Single Cluster mdsr1 Figure 1-6 WebSphere Business Services Fabric for z/OS in the BPM reference topology for z/OS Chapter 5, “WebSphere Business Services Fabric for z/OS” on page 201, provides detailed instructions on how to add WebSphere Business Services Fabric for z/OS to the BPM reference topology for z/OS. 1.7 Monitoring the BPM environment WebSphere Business Monitor is a business-activity monitoring application that measures business performance, monitors real-time and completed processes, and reports on business operations. It measures business performance, monitors business processes, detects business situations, issues related alerts, and graphically presents business information. This helps the user identify business problems, correct exceptions, and change processes accordingly. 34 z/OS: WebSphere Business Process Management V6.2 Production Topologies 1.7.1 Monitoring the BPM reference topology for z/OS using WebSphere Business Monitor WebSphere Business Monitor is an application that relies heavily on the other components in the BPM suite. The WebSphere Business Monitor server does not run natively on z/OS. To exploit its capabilities in our environment, we deployed WebSphere Business Monitor on a SuSE 10.2 Linux LPAR on system z. We used System z hypersockets to interconnect the Linux LPAR and the z/OS LPAR. Finally, we constructed the proper CEI buses in both environments to allow event information to flow from the WebSphere Process Server for z/OS and WebSphere Business Services Fabric for z/OS installations on z/OS to the WebSphere Business Monitor infrastructure deployed in the Linux LPAR. Hybrid z/OS - Linux on System z configurations are well-tested and are a best practice when integrating SOA technologies on the z/OS platform. Figure 1-7 on page 36 outlines the critical components of this topology and how it fits with the BPM reference topology for z/OS. Chapter 1. Business Process Management production topologies for z/OS 35 Controller MDSR1A Controller MDSR1B Servant MDSR1AS (WPS) Servant MDSR1BS (WPS) BPC (HTM and BFM) CEI BPC Explorer BRM Business Space Fabric Monitor BPC (HTM and BFM) CEI BPC Explorer BRM Business Space Fabric Monitor SCA APP SCA SYS ME ME CEI ME Adjunct MDSR1AA BPC ME Member 1 (WPS) BPC (HTM and BFM) Member 2 (WPS) BPC (HTM and BFM) Application Target Cluster (AppTarget) Member 1 (WPS) CEI BPC Explorer BRM Buisness Spaces AlphaBlox Member 2 (WPS) CEI BPC Explorer BRM Business Spaces AlphaBlox Support Cluster (Support) Member 1 Moderator Model logic module Member 2 Moderator Model logic module Member 1 (Mon) Action services DM service CEI Rest Services Member 2 (Mon) Action services DM service CEI Rest Services Monitor Moderator Model logic Cluster (Monitor) Mon ME Adjunct MDSR1BA mdsr1 Cluster Monitor Support Cluster (Mon Support) Member 1 (WAS) mdnodea mdnodeb monNode01 Monitor bus Mon ME CEI Bus BPC ME SCA.APP Bus CEI ME SCA.SYSTEM Bus BPC Bus Monitor bus CEI Bus SCA.APP Bus SCA.SYSTEM Bus BPC Bus SCA APP SCA SYS ME ME Messaging Cluster (Messaging) Member 2 (WAS) monNode02 Figure 1-7 BPM reference topology for z/OS hybrid topology with WebSphere Business Monitor deployed in a Linux for System z Cluster Refer to Chapter 8, “Monitoring the Business Process Management environment” on page 465 for details on how to build this environment. 36 z/OS: WebSphere Business Process Management V6.2 Production Topologies Part 2 Part 2 Building production topologies for WebSphere Process Server © Copyright IBM Corp. 2009. All rights reserved. 37 38 z/OS: WebSphere Business Process Management V6.2 Production Topologies 2 Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server This chapter describes how to plan and configure a WebSphere Process Server for z/OS in a standalone server. This will be used as the foundation for a production topology ND cell when we federate it with a Deployment Manger cell as described in Chapter 3, “Creating a WebSphere Process Server for z/OS network deployment cell” on page 151. The standalone server is a fully-functional WebSphere Process Server for z/OS (although not suitable for production). It can be useful for application testing, but will not have a Deployment Manager, multiple nodes, or clusters that are required for high availability and management. It will also not perform as well as an network deployment (ND) configuration because the single server must also support and run the Integrated Solutions Console application. We describe how to add additional nodes on other z/OS systems (LPARs) required for our production topology in Chapter 4, “Creating additional WebSphere Process Server nodes” on page 179. © Copyright IBM Corp. 2009. All rights reserved. 39 This chapter contains the following sections: 2.1, “Overview” on page 40 2.2, “Planning the production network deployment topology” on page 44 2.3, “Preparing the z/OS system components” on page 53 2.4, “Preparing DB2” on page 75 2.5, “Creating a standalone WebSphere application server” on page 102 2.6, “Adding WebSphere Process Server for z/OS” on page 114 2.7, “Creating the DB2 databases and tables” on page 122 2.8, “Starting the standalone application server” on page 138 2.9, “Testing sample applications” on page 142 2.10, “Testing the vehicle loan process application” on page 146 2.1 Overview Configuring WebSphere Application Server for z/OS and WebSphere Process Server for z/OS requires an understanding of the z/OS runtime and the various components that make up these products. Two good sources of this information can be found in the following documents: z/OS Getting Started: WebSphere Process Server and WebSphere Enterprise Service Bus V6, SG24-7378 z/OS Technical Overview: WebSphere Process Server and WebSphere Enterprise Service Bus, REDP-4196 Also, see “Abbreviations and acronyms” on page 625. There are three basic ways to approach the task of configuring a WebSphere Process Server for z/OS cluster: Start with WebSphere Process Server for z/OS in a stand-alone server, which is described in this chapter. Then, create a deployment manager, augment it for WebSphere Process Server for z/OS, and federate the standalone server. This is described in Chapter 3, “Creating a WebSphere Process Server for z/OS network deployment cell” on page 151. Finally, create additional empty nodes, augment them, and federate them into the cell as described in Chapter 4, “Creating additional WebSphere Process Server nodes” on page 179. This information is based on the White Paper WP101253, WebSphere Process Server for z/OS V.6.1 “Easy” Network Deployment Configuration. The White Paper can be obtained from the IBM Technical Sales Library at the following Web page: http://www.ibm.com/support/techdocs 40 z/OS: WebSphere Business Process Management V6.2 Production Topologies Configure a deployment manager and empty nodes and augment them with WebSphere Process Server for z/OS. Then, federate them into a network deployment cell and configure the various components (SCA, CEI, BPC, BS) with wizards using the Integrated Solutions Console. This scenario was described in the z/OS: WebSphere Business Process Management V6.1.2 Production Topologies, SG24-7703. Configure a deployment manager and empty nodes, add WebSphere Process Server for z/OS to the nodes, and federate them into a network deployment cell. Then use the Deployment Environment “wizard” to configure the various components. At the time of the writing this Redbooks publication, we did not get the deployment environments working, but hope to do so with version 6.2.0.1 of WebSphere Process Server for z/OS. 2.1.1 Starting with a standalone server An advantage of the stand-alone server approach is that it is easier and quicker. When you run the zWPSConfig.sh as part of augmenting the stand-alone server, it configures almost all components of WebSphere Process Server for z/OS at that time. You must still configure WebSphere Business Services Fabric for z/OS as a separate task, however. The disadvantages of starting with a stand-alone server are as follows: All the components are in one server, which means that after federating it into a network deployment cell you will end up with a single-cluster topology. To create the two-clusters topology, you need the flexibility to control where each component is deployed. For this, you can use the one of the other approaches listed above, which both start with empty nodes. The names of buses include the cell name of the server in which they were created. When the buses are transferred into a network deployment cell during federation of the stand-alone server, they retain their original names. Therefore, after federation the names of the buses will contain the cell name of the old stand-alone server and not the cell name of the network deployment cell. This is not a serious issue but one of which you need to be aware. This approach cannot be used for creating additional clustered configurations. It may only be employed for creating the initial cluster in a cell. Subsequent clusters must be created with empty nodes. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 41 2.1.2 Outlining the steps The procedure for configuring a standalone server, described in this chapter, is as follows: 1. Plan your target configuration. This is discussed in 2.2, “Planning the production network deployment topology” on page 44. We describe how to use a spreadsheet to plan the cell, The spreadsheet generates the response file input to the zProfile Management Tool (zPMT), which can be used to create the standalone server and augment it for WebSphere Process Server for z/OS. 2. Prepare your z/OS system. This is discussed in 2.3, “Preparing the z/OS system components” on page 53, which covers configuring catalog aliases, UNIX® system services environment, symbolic links, security, DB2, and other products. 3. Plan and prepare your DB2 subsystem configuration. DB2 is an integral part of any production topology, and deserves careful planning and preparation. You should work with your DBA in this part. This is discussed in 2.4, “Preparing DB2” on page 75. 4. Create a standalone WebSphere application server. This is discussed in 2.5, “Creating a standalone WebSphere application server” on page 102. We describe how to use the zPMT to define the application server’s node that will eventually be the base of our ND topology. 5. Augment WebSphere Process Server to the standalone server. This is discussed in 2.6, “Adding WebSphere Process Server for z/OS” on page 114. In this step you augment the server with the zSMPInstall and zWPSConfig shell scripts. 6. Create the database tables required for WebSphere Process Server. This is discussed in 2.7, “Creating the DB2 databases and tables” on page 122. In this step you create the WebSphere Process Server databases in DB2 z/OS and execute the Data Definition Language (DDL) to create all the tables, indexes, and views used by WebSphere Process Server for z/OS. Note: The tables cannot be created until the DDL is generated by the zWPSConfig script in the previous step. 7. Start up the augmented server and correct any problems. 42 z/OS: WebSphere Business Process Management V6.2 Production Topologies This is discussed in 2.7.10, “Verifying the DB2 table creation” on page 136. In this step you will start the server, examine the logs, and correct any settings using the Integrated Solutions Console. 8. Verify that the Business Process and Human Task containers, and related tools like the BPC Explorer, are functioning normally. See 2.9, “Testing sample applications” on page 142. Note about the procedure: At this point you have a fully functional WebSphere Process Server for z/OS, which can be used for application deployment and testing. You can use this standalone server as the basis for your production topology. 9. Configure a network deployment cell. In Chapter 3, “Creating a WebSphere Process Server for z/OS network deployment cell” on page 151, we describe how to configure a deployment manager for WebSphere Process Server for z/OS, and federate this standalone server into a network deployment configuration. 10.Add another node to your cell. To get to your desired production topology, you will add another node on a separate LPAR. This is described in Chapter 4, “Creating additional WebSphere Process Server nodes” on page 179, where we show how to create another node and create clusters spread across multiple systems. 11.Define HTTP servers or proxies. This is discussed in 5.3.3, “Define HTTP servers or proxies” on page 278. You will likely require one or more HTTP Servers or external proxies in front of your WebSphere Process Server for z/OS clusters. In this step we describe the options. 2.1.3 Extending production topologies At this point the WebSphere Process Server for z/OS configuration is complete, but can be enhanced by the following extensions. Part 3, “Extending production topologies” on page 199 discusses the following topics: Examine Business Space powered by WebSphere and verify that it is working properly. This is discussed in Chapter 9, “Creating an integrated user experience using Business Space powered by WebSphere” on page 547. After completing the tasks in this chapter you can add WebSphere Business Services Fabric for z/OS to the topology, as described in Chapter 5, “WebSphere Business Services Fabric for z/OS” on page 201. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 43 We also use the BPC Explorer to verify WebSphere Business Services Fabric for z/OS functions. See 7.1.10, “Test the sample application” on page 424. You can also incorporate the Business Monitor into this configuration, which is described in Chapter 8, “Monitoring the Business Process Management environment” on page 465. This Redbooks publication aims to help you to avoid the human errors that may occur during configuration of WebSphere Process Server for z/OS. With this in mind, Chapter 10, “Problem diagnosis and prevention” on page 571 details most problems we experienced, with their solutions. If you have experience with WebSphere Application Server for z/OS, and access to someone with good DB2 for z/OS skills, you can configure WebSphere Process Server for z/OS in a few days. This assumes that you do not lose too much time resolving problems and have planned properly. 2.2 Planning the production network deployment topology A great deal of planning is crucial to the successful implementation of a BPM topology. This planning will pay for itself in more efficient configuration, operations, systems management, and trouble-shooting. This section describes the following planning areas: Configuration topologies Hardware requirements (CPUs, memory, disk storage.) Names for cells, nodes, servers, clusters, user IDs, group IDs, and so forth. z/OS dataset names and space allocations Security identities, aliases, roles, and so forth DB2 names for databases, storage groups, schemas, buffer pools, and so forth UNIX file system directories, and files (ownerships and permissions), 2.2.1 Choosing the topology Before configuring WebSphere Process Server for z/OS, you must know the target topology you are aiming for. Topologies are discussed in detail in Chapter 1, “Business Process Management production topologies for z/OS” on page 3. We started with the single-cluster topology in a cell with an MD prefix. The spreadsheet that we describe in 2.2.2, “Using a spreadsheet to plan names for configuration objects” on page 45 can be used to help plan both topologies. 44 z/OS: WebSphere Business Process Management V6.2 Production Topologies 2.2.2 Using a spreadsheet to plan names for configuration objects The long and short names of cells, nodes, servers, and clusters deserve careful consideration to identify the meaningful use of these components, and allow for growth with additional servers, clusters, nodes, and LPARS. Note: For the short name of the server, we used MDSR1A and MDSR1 for the cluster transition name. The spreadsheet creates these names as MDSR01A and MDSR01 respectively, which is the recommended naming convention. We apologize for any confusion this may cause. The White Paper WP101030,WebSphere z/OS V6.1 - A Top Down Configuration Approach explains how to use a spreadsheet to create a WebSphere Application Server for z/OS V6.1 cell. You can obtain that paper at the following Web page: http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101030 We downloaded the PRS1331 - zPMT in AST 6.1.1 WebSphere V6.1 Configuration.xls spreadsheet from the following Techdocs Web page: http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/PRS1331 We made some enhancements to the spreadsheet to plan a cell that contains WebSphere Process Server for z/OS and WebSphere Business Services Fabric for z/OS. One of the enhancements was to generate response files compatible with the new Workstation Customization Tool (WCT) rather than the older Application Server Toolkit (AST). Single-cluster topology spreadsheet: The spreadsheet we used to plan our single-cluster topology MD cell is called MD - zPMT WPS V6.2 Configuration.xls. It can be found in the zos_planning_spread sheet directory in the additional material that accompanies this Redbooks publication. See Appendix A, “Additional material” on page 615. We made several changes to the standard WebSphere Application Server for z/OS spreadsheet to add support for WebSphere Process Server for z/OS. These changes are listed on the documentation worksheet in MD - zPMT WPS V6.2 Configuration.xls. You do not have to use a spreadsheet to plan your WebSphere Process Server cluster, but we suggest it as a quality-control tool. Many errors that occur when configuring WebSphere Process Server for z/OS come from poor or inconsistent choices, or simple typing errors. By using the spreadsheet you will avoid those errors. The spreadsheet becomes a record of the choices you made and what you did. It becomes your audit trail. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 45 The naming convention used in the spreadsheet follows that proposed by the IBM Washington Systems Center. If you do not want to follow that naming convention, do not give up on the spreadsheet. Experience has shown that it is better to change the formulae in the spreadsheet so they match your naming convention, and use the generated response files for input into zPMT. This is more likely to result in a successful installation than working from ad-hoc notes or typing all the input into zPMT manually. There are good spreadsheets available from Techdocs PRS1331, and in the additional files with this Redbooks publication. This is an enhanced version to the spreadsheet provided in z/OS: WebSphere Business Process Management V6.1.2 Production Topologies, SG24-7703.) How to use the spreadsheet This section explains how to use the extensions we made to the WebSphere Application Server for z/OS spreadsheet to add support for WebSphere Process Server for z/OS. We do not describe in detail how to use the spreadsheet for configuring the underlying WebSphere Application Server for z/OS nodes because that is already covered in White Paper WP101030,WebSphere z/OS V6.1 - A Top Down Configuration Approach. However, the checklist worksheet provides a summary of the steps involved. It guides you through customizing the Variables worksheet, copying the response-file worksheets from the spreadsheet, and importing them into zPMT. This allows you to generate the .CNTL and .DATA datasets that you use to configure the cell and each node. This is the same process that is documented in WP101030. However, there are some important differences compared to the standard spreadsheet provided in PRS1331 on the Techdocs web site. 46 z/OS: WebSphere Business Process Management V6.2 Production Topologies The worksheet names have been slightly changed and there are separate worksheets for configuring the primary and secondary empty nodes. Table 2-1 on page 71 is a list of the worksheets that can be accessed for many activities in this IBM Redbooks publication. Figure 2-1 A description of the worksheets to help configure your cell You can access these worksheets by clicking on the tabs (Figure 2-2) along the bottom. Figure 2-2 Tabs The checklist worksheet summarizes the tasks so you can check them off as you complete them. The checklist contains any additional tasks that we suggest you complete before starting to execute the BBO* jobs generated by the zPMT. Figure 2-3 on page 48 is an example from the spreadsheet. The tasks listed in the figure are described in 2.3.6, “Using intermediate symbolic links” on page 65. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 47 Prepare the primary node Check and run job MDNAALIA from the JCL_P worksheet to define ICFCAT aliases for the Deployment Manager and the Primary node. Check and run job MDNASYML from the JCL_P worksheet to define intermediate symbolic links to the current product mount points. Create directory /wasv61config/mdcell/ before running this job. Figure 2-3 Checklist tasks to define ICFCAT aliases and intermediate symbolic links Start by completing the Variables worksheet. Then, follow the checklist worksheet from top to bottom. Each major heading or task in the checklist has a matching section in this Redbooks publication so you can locate additional information about any step by consulting this document. Many fields also have comments that provide help. Just below the top of the Variables worksheet are fields in which you choose the LPARs that will host the primary and a secondary node. You also assign a single character qualifier to each node, as shown in Figure 2-4. Figure 2-4 Variables controlling the node topology Set the names of the LPARs that will host nodes and choose a single character node identifier. The values in the primary node column are used in the EmptyNode_P worksheet that generates the response file you will use when defining the primary node. The values in the Secondary Node column are used to generate the response file on the EmptyNode_S worksheet. You will use that when defining a secondary node. If you plan to create a cluster across more than two nodes, create the first secondary node, and return to the Variables worksheet to alter the LPAR name and node identifier in the secondary node column so the values apply to the third LPAR. The EmptyNode_S worksheet will then contain values relating the third node. You can import that response file into zPMT to define the third node. The sections in the checklist that apply to a secondary node will contain customized instructions for the third node. Below the node topology there are fields that control the cluster topology. To the right of the Choose WPS Topology (single-cluster | two-clusters) field there is a list box with two choices. When you choose the single-cluster topology only one cluster name is generated, as shown in Figure 2-5 on page 49. 48 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 2-5 Single cluster topology When you choose the two-clusters topology it causes different cluster names to be generated. The cluster name includes the 2-digit cluster identifier you decided to use, as in Figure 2-6 Figure 2-6 Two cluster topology There is nothing special about the cluster names and you can change the formulae to generate any cluster name you like. Note that instructions in the checklist worksheet differ depending on the choice of topology. Note: For the short name of the server, we used MDSR1A and MDSR1 for the cluster transition name. The spreadsheet creates these names as MDSR01A and MDSR01 respectively, which is the recommended naming convention. We apologize for any confusion this may cause. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 49 Further down the Variables worksheet, the 2-digit cluster identifiers are used to assign a unique port range to each cluster, as shown in Figure 2-7. Figure 2-7 Unique port range assigned to each cluster Specify the details of the DB2 for z/OS system that hosts your WebSphere Process Server for z/OS databases and plan out the database names, schemas, volumes, and bufferpools you will use. You can also decide whether to share a single database for all components or not. 50 z/OS: WebSphere Business Process Management V6.2 Production Topologies Part of the table used to plan these DB2 names is shown in Figure 2-8. Figure 2-8 Part of table used to plan database names The numeric part of the schema and database names of the SIBs is derived from the cluster identifier. This allows you to potentially create more than one cluster running BPC with its own SIB, for example. The light green fields contain generated defaults that you can change by altering the formulae, which you would not normally update when configuring a new cell. The dark green fields, however, must be updated to reflect your requirements. The bufferpool fields will not allow you to choose BP0, BP8K0, or BP16K0. Use some other bufferpools such as BP1, BP8K1, or BP16K1 The WPS_rsp tab in the spreadsheet contains generated response files for the deployment manager and empty nodes as well Do not use response files from WebSphere Process Server V6.1.2. These will not work with V6.2, because the parameters have changed. The LDAP variables shown in Figure 2-9 on page 52 are used on the WPS_LDAP worksheet to generate the sample ldif file. We used a Tivoli Directory Server for z/OS LDAP server with WebSphere Business Services Fabric for z/OS. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 51 Figure 2-9 The LDAP variables on the Variables worksheet After completing the Variables worksheet you can start to build the cell by following the Checklist tab. You might want to look at the WPS_RACF and WPS_LDAP worksheets before you start. Note: We used AUTOGID and AUTOUID features of RACF, but you can specify specific GIDs and UIDs in your configuration. We highly recommend using AUTOGID and AUTOUID. The WPS_RACF worksheet is where all the user IDs and groups that are used by WebSphere Process Server for z/OS and WebSphere Business Services Fabric for z/OS are planned. Some of the fields on the WPS_RACF worksheet are dark green, meaning that you are expected to set your values in those fields (for example, the password values on the WPS_RACF worksheet). Values on this worksheet are used in the xxRACF job on the JCL_P worksheet. That job creates all the user IDs required with WebSphere Process Server for z/OS. If you plan to use an LDAP user registry for the WebSphere Process Server for z/OS cell rather than RACF, review the WPS_LDAP worksheet and ensure that the user IDs and groups are added to your LDAP server. The user IDs and groups on the WPS_LDAP worksheet are derived from those on the WPS_RACF worksheet because you must define the cell’s started task user IDs, groups, and administrator user ID to the LDAP server. Because these have to be defined in RACF, even when the cell is using LDAP user registry for authentication and authorization, it is less confusing if the same names are used in both registries. 52 z/OS: WebSphere Business Process Management V6.2 Production Topologies 2.3 Preparing the z/OS system components The latest information about the supported hardware and software configurations for WebSphere Process Server or z/OS can be found on the following Web page: http://www.ibm.com/support/docview.wss?&uid=swg27013442 2.3.1 Planning for hardware requirements for these topologies Once you have planned the topology of your servers, you must allocate sufficient resources for the configuration you have chosen to assure adequate performance, availability, growth and systems management characteristics. CPUs There are capacity planning tools that can help you size your processor requirements for WebSphere Process Server for z/OS. Contact your IBM Representative to have a Capacity Planning or sizing study run for your environment. They can provide estimates of System z processor capacity for J2EE™ applications running under WebSphere Process Server (WPS). Note: We recommend a minimum of two processors per LPAR to minimize the overhead of switching between address spaces in a WebSphere Application Server Network Deployment for z/OS environment. IBM System z Application Assist Processors (zAAPs) zAAPs are lower-priced specialized processing units that provide a strategic z/OS Java execution environment for customers who desire cost-effective processing engines and traditional qualities of service of the IBM mainframe platform. zAAPs are processors out of the general processor pool that have special licensing terms to allow Java to run on z/OS with lower TCO. They run at full-rated processor speed and are not capped according to machine type and rating. They take advantage of the technology dividend (at no cost) as machine designs progress. For example, zAAP on z10 is 35% faster than comparable zAAP on z9®, at no additional cost to customers. WebSphere Application Server for z/OS and WebSphere Process Server for z/OS offers full support for zAAPs. zAAPs are designed to operate asynchronously with the general purpose processors when executing Java programming under control of the IBM JVM. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 53 zAAP can help you integrate and run new Java-based Web applications and lower the overall cost of computing for running Java technology-based workloads on the platform. It would be foolish not to use zAAP engines in these environments. For more details, see zSeries Application Assist Processor (zAAP) Implementation, SG24-6386. Real memory The real memory requirements for WebSphere Application Server for z/OS and WebSphere Process Server for z/OS depends largely on the size and number of JVM heaps in your servers. They do not tolerate paging well. (If the installation has a robust paging subsystem, then a small amount of demand paging can usually be tolerated.) 31-bit versus 64-bit Virtual storage demands also affect the real storage requirements. z/OS has 31-bit addressing modes and 64-bit addressing modes, not 32-bit and 64-bit modes. This means that when in 31-bit mode, the useful addressability is greatly reduced to less than a one gigabyte address space. Most z/OS installations cannot get more than 768 megabytes of virtual storage for their private address spaces. Any customer that needs a JVM with a heap greater than one gigabyte needs a 64-bit JVM, as there simply is not enough room in a z/OS address space for a JVM with that heap requirement. (Remember -Xms means nothing with respect to the amount of space acquired for the JVM heap. The -Xmx specification determines the amount of heap space acquired in one chunk for the use of the JVM). Even if you do not have a need for a heap in excess of one gigabyte there is a need for 64-bit JVMs to alleviate native storage issues. Native Code A 64-bit JVM requires that all native code that accessed through the JNI™ must also be 64-bit. This means that all 31-bit code must be converted to 64-bit or that code must be written that allows thunking between 31-bit mode and 64-bit mode. This exercise is further complicated by the restriction that the Language Environment® does not allow 31-bit and 64-bit language environment-enabled code to run in the same process. There are products that do not have 64-bit native support and thus prevent the use of 64-bit JVMs. For example, only recently did MQ supply a 64-bit non-RRS-enabled local connector for Java. 54 z/OS: WebSphere Business Process Management V6.2 Production Topologies Note: The 64-bit capable native runtime is provided in library SCLBDDL2 and that library, as well as SCLBDDL, should be in the system linklist. If SCLBDDL2 is not in LINKLST, or in the server's STEPLIB, it cannot run in 64-bit mode. The largest problem is the overhead of going to 64-bit mode. Switching to a 64-bit JVM while keeping the heap size constant has resulted in 7–10% increase in CPU consumption, This is predominantly due to 64-bit pointer versus 32-bit pointers, and increased GC rates. To mediate the increase in GC activity, heap sizes were increased, but you still have the overhead due to more CPU cycles being consumed and a greater appetite for storage. (This is not just a z/OS statement. It is true across all platforms.) Weigh the advantages of larger heap sizes and available memory versus the additional CPU required to run in 64-bit addressability mode. In many cases, more memory will allow the applications under test to actually run faster (thereby reducing CPU overhead,) due to better memory allocation and management. When in doubt, test the configuration and note the pros and cons of each environment. Disk storage In addition to the DASD volumes used to hold the product code and file system, you will need additional DASD storage to hold configuration data for application serving environments. The amount of storage depends on the number of environments and the size and complexity of the applications being deployed. A cell running WebSphere Process Server for z/OS and WebSphere Business Services Fabric for z/OS will require a large configuration dataset for each node. After configuring WebSphere Business Services Fabric for z/OS in a WebSphere Process Server for z/OS network deployment configuration, the deployment manager’s configuration occupied 1400 CYLs, while each node occupied over 1500 CYLs. These values were added to the spreadsheet used to plan the cell. When you have large datasets like these, dumping them can be tricky because a dump of all the configuration datasets on disk can easily exceed 3500 CYL. We suggest that you speak to your disk space administrator about setting up a pool of 3390-9 disks to hold the configuration datasets and dumps of the WebSphere Process Server for z/OS cell. You should also request that DFSMS maintains sufficient free space on these volumes that will allow the node configuration datasets to extend. Running out of space on a volume when configuring WebSphere Process Server for z/OS and Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 55 WebSphere Business Services Fabric for z/OS is the cause of many strange problems and will probably result in having to restore the configuration to the point before the failure. You will need to plan the dataset high-level qualifier used by the configuration datasets because your DFSMS ACS™ routines will need to be updated to map that high-level qualifier to the new storage pool of 3390-9s. 2.3.2 Planning for software requirements The Web page instructs you to use the latest service level of IBM WebSphere Application Server for z/OS 6.1. It is important to note that WebSphere Process Server for z/OS V6.2 requires at least WebSphere Application Server for z/OS service level 6.1.0.21. We ran our testing with level 6.1.0.22. If you intend to use more than one servant or a network deployment configuration of any kind, plan to use one of the supported releases of DB2 for z/OS. DB2 for z/OS V9.1 has improved the performance of LOB management and relaxed some security rules on setting the current schema. Both these changes benefit WebSphere Process Server for z/OS. For that reason DB2 for z/OS V9.1 is recommended. WebSphere Process Server for z/OS 6.2 requires the Web Services Feature Pack, which must be installed in the WebSphere Application Server for z/OS using the zProfile Management Tool (zPMT). The zPMT is part of the WebSphere Configuration Tool (WCT) and the Application Server Toolkit (AST). In summary the software levels we used were as follows: z/OS 1.9 DB2 for z/OS 9.1 WebSphere Application Server for z/OS 6.1.0.22 Feature Pack for Web Services for z/OS 6.1.0.21 WebSphere Process Server for z/OS 6.2 WebSphere Business Services Fabric for z/OS 6.2 WebSphere Business Monitor V6.2 WebSphere Configuration Tool (WCT)/zPMT 7.0.0.1 Tivoli Directory Server for z/OS 6.2 z/OS v1.8 or higher Any supported release of z/OS operating system can be used with WebSphere Process Server for z/OS, which covers Versions 1.8, 1.9, and 1.10. We recommend using the latest release (1.10) because it contains useful enhancements for your environment. 56 z/OS: WebSphere Business Process Management V6.2 Production Topologies Preparing the base operating system WebSphere Application Server for z/OS and WebSphere Process Server for z/OS make extensive use of the z/OS operating system components and sysplex implementation. See the document “Preparing the base operating system” and the subordinate articles (listed below the URL) in the Information Center, at the following Web page: http://publib.boulder.ibm.com/infocenter/dmndhelp/v6r2mx/index.jsp Checklist: Preparing the base operating system. Prepare z/OS operating system settings. Prepare z/OS sysplex settings. Prepare the z/OS job entry subsystem (JES2 or JES3). Identify TCP/IP resources you want to use and prepare your network. Set up Resource Recovery Services (RRS). Set up your SAF-compliant security package. If you will use another SAF-compliant security product, consult the product's manufacturer for assistance. Set up component trace (CTRACE). WebSphere Application Server for z/OS The minimum level of WebSphere Application Server for z/OS must be at version 6.1.0.21. In general, we recommend using the latest release level of WebSphere Application Server for z/OS version 6.1. (WebSphere Process Server for z/OS Version 6.2 does not support, and will not run on WebSphere Application Server for z/OS version 7.) What is the level of my WebSphere products? There are several ways to determine the level of WebSphere products in your runtime: For the configured nodes, run the versionInfo.sh script in the <profile_root>/bin directory. The service level appears in the Integrated Solutions Console Welcome window, and the System Administration → Nodes window in the Version column. There is also a message in the JOBLOG of the started servers: BBOM0007I CURRENT CB SERVICE LEVEL IS build level 6.1.0.22 (cf220846.13) release WAS61.ZNATV date 11/20/08 23:35:52. For a product’s SMP/E-maintained ZFS™, you can look in the <smpe_root>/properties/version/WAS.product file (in ASCII). Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 57 For WebSphere Application Server for z/OS, you can browse the BBO*.BBOLOAD(BBOUBINF) module and issue the find build command. Feature Pack for Web Services for z/OS WebSphere Process Server for z/OS Version 6.2 requires the Feature Pack for Web Services for z/OS. To implement the Feature Pack for Web Services functionality in an application server node, it is necessary to include the FPWS at the time the node is initially built. The minimum level of Feature Pack for Web Services for z/OS must be at version 6.1.0.21. PTF UK26168 and UK26278 are what load the FPWS code into this separate HFS. Note: You cannot update an existing application server node with the Feature Pack for Web Services function after the fact. zProfile Management Tool This tool goes on your workstation. We used the zProfile Management Tool (zPMT) component of the WebSphere Configuration Tool (WCT) V7.0.0.1 to configure our WebSphere cell. The WCT contains two components: the zPMT and the MMT (Migration Management Tool) for WebSphere Application Server for z/OS v7. It also supports the configuration of V6.1 servers. (The WCT/ zPMT is a smaller download than the Application Server Toolkit, ASTK/ zPMT and “lighter” in weight and performance.) Note: At the time of this writing (March, 2009) the zPMT does not support response files for WebSphere Process Server for z/OS augmentation. Download the WCT from the following IBM Support Web page: http://www.ibm.com/support/docview.wss?rs=180&uid=swg24020368 Note: On March 27, 2009, the WCT was updated to 7.0.0.3. The page offers to download a file called download.7000.wct.windows.ia32.zip, which you might think is 7.0.0.0, but it really is 7.0.0.3. When WCT is started, you will see a Software Updates option under Help. The build level for 7.0.0.3 is cf010908.27. WebSphere Integration Developer The WebSphere Integration Developer Version 6.2 is required for WebSphere Process Server for z/OS. 58 z/OS: WebSphere Business Process Management V6.2 Production Topologies Note: Use a level of WebSphere Integration Developer that is compatible with the level of WebSphere Process Server for z/OS you are running. 2.3.3 Using SMP/E to install WebSphere Process Server for z/OS In this Redbooks publication we do not explain how to install WebSphere Process Server for z/OS using SMP/E. That is described the document, Program Directory for WebSphere Process Server for z/OS V6.2, GI11-2880. A printed copy is shipped with the product, or you can download the program directory from the IBM Publications Center at the following Web page: ftp://ftp.software.ibm.com/software/websphere/integration/wps/library/p dfz620/program_directory_620.pdf The program directory can also be downloaded from the following Web page: http://www.ibm.com/shop/publications/order Search on the document number, GI11-4299. WebSphere Application Server for z/OS is included with WebSphere Process Server for z/OS, as shown in Figure 2-10 on page 60. The Feature Pack for Web Services must be included with the WebSphere Application Server for z/OS nodes before WebSphere Process Server for z/OS can be configured. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 59 WebSphere Process Server for z/OS (includes WESB) hlq.SBPZHFS FMID: HWPS620 /usr/lpp/zWPS/V6R2 WebSphere Application Server for z/OS 2 WebSphere Application Server for z/OS Feature Pack for Web Services 1 hlq.SBBOxxxx FMID: H28W610 hlq.SIWOHFS FMID: JIWO610 hlq.SBBOHFS FMID: H28W610 /usr/lpp/zWebSphere/V6R1 (WAS 6.1.0.21+ required for WPS V6.2) /usr/lpp/zWebSphere_OM/V6R1 (WAS FPWS 6.1.0.21+ required for WPS V6.2) Figure 2-10 Packaging of WebSphere Process Server for z/OS The starting point for this section is after you have successfully unloaded the products from the installation media, installed them using SMP/E, and mounted the product HFS data sets into your UNIX hierarchical file system (HFS). If you ordered WebSphere Process Server for z/OS using CBPDO, the installation using SMP/E involves the following tasks: Obtain the latest copy of the Program Directory document. Obtain the latest Preventative Service Planning (PSP) document for WebSphere Application Server and WebSphere Process Server. The PSP has the list of the latest important PTFs to install. The name of this document and instructions on obtaining it are in the Program Directory. If you ordered the products using IBM SystemPac® or ServerPac, follow the instructions in ServerPac: Installing your Order that ships with your SystemPac or ServerPac that came with the product. Also, the product information centers contain information about the installation using SMP/E. 60 z/OS: WebSphere Business Process Management V6.2 Production Topologies Note: WebSphere Process Server for z/OS includes WebSphere Enterprise Service Bus for z/OS and there is only the one HFS data set that contains both products. Therefore, after you have installed WebSphere Process Server for z/OS with SMP/E you will have hlq.SBBO* target libraries belonging to WebSphere Application Server for z/OS and hlq.SBPZ* libraries for WebSphere Process Server for z/OS, but you will not find hlq.SBSB* data sets for WebSphere Enterprise Service Bus for z/OS. 2.3.4 Planning z/OS dataset names and aliases The following sections will help you organize the z/OS libraries for WebSphere Application Server for z/OS with indirect names so you can migrate to new levels of the software on a node-by-node basis. LPA, Linklist, and STEPLIBs You can choose to load the contents of these libraries into the Link Pack Area (LPA) so all WebSphere address spaces share the same copy of the programs. Using LPA will reduce virtual storage requirements and improve program load performance, but it means that all the cells that load the WebSphere Application Server for z/OS modules from LPA will need to be upgraded at the same time a new service level is loaded into the LPA. In other words, a service upgrade will affect multiple cells, possibly all the cells in an LPAR. To have the flexibility to run nodes at different service levels, you must configure each node so it references the SBBO* libraries through STEPLIB DD statements. When using STEPLIB, we recommend that you create a set of ICFCAT aliases for each node that refers to the real dataset names of the SBBO* libraries. By doing this, you will avoid the need to make any manual changes to the STEPLIB in all the started task JCL, and to the export STEPLIB statement in the setupCmdLine.sh, when it is time to upgrade a node to a new service level. We will discuss the use of STEPLIBs in more detail in 2.5.7, “Customizing the new server” on page 112 Dataset names and ICFCAT aliases WebSphere Application Server for z/OS also has some z/OS load libraries and we recommend that you indirectly reference these libraries through the use of ICFCAT aliases rather than coding the real dataset names in JCL. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 61 WebSphere Application Server for z/OS has the following z/OS load libraries: SBBOLPA SBBGLOAD SBBOLOAD SBBOLOD2 You need only to delete and redefine the ICFCAT alias to switch a node from the set of SBBO* libraries at the old service level to the set of SBBO* libraries at the new service level. Create a set of ICFCAT aliases for each node in your WebSphere Process Server for z/OS cell. Here, we created sample job MDNAALIA (Example 2-1) to create a set of ICFCAT aliases for mdnodea that map to a set of physical SBBO* datasets called BBO6142.SBBO*. Notice that the second qualifier in the alias name matches the node name. Create a set of aliases for each node. Example 2-1 Sample JCL to define a set of ICFCAT aliases for mdnodea //MDNAALIA JOB (0),'MDADMIN',CLASS=A,MSGCLASS=H,NOTIFY=&SYSUID /*JOBPARM SYSAFF=SC42 //* //* Define a set of aliases for the application server //* //S1 EXEC PGM=IDCAMS //SYSPRINT DD SYSOUT=* //SYSIN DD * DEF ALIAS (NAME('BBO6142.MDNODEA.SBBOEXEC') + RELATE('BBO6142.V6R2.SBBOEXEC')) DEF ALIAS (NAME('BBO6142.MDNODEA.SBBGLOAD') + RELATE('BBO6142.V6R2.SBBGLOAD')) DEF ALIAS (NAME('BBO6142.MDNODEA.SBBOLOAD') + RELATE('BBO6142.V6R2.SBBOLOAD')) DEF ALIAS (NAME('BBO6142.MDNODEA.SBBOLD2') + RELATE('BBO6142.V6R2.SBBOLD2')) DEF ALIAS (NAME('BBO6142.MDNODEA.SBBOLPA') + RELATE('BBO6142.V6R2.SBBOLPA')) DEF ALIAS (NAME('BBO6142.MDNODEA.SBBOMSG') + RELATE('BBO6142.V6R2.SBBOMSG')) /* When configuring the WebSphere Application Server for z/OS nodes, always use the ICFCAT aliases names rather than the real dataset names. When using the spreadsheet to plan the cell, if you choose to use ICFCAT aliases on the Variables worksheet, the alias names are used in the response files that are generated by the spreadsheet. 62 z/OS: WebSphere Business Process Management V6.2 Production Topologies 2.3.5 Planning UNIX (UNIX System Services) file systems Here are some recommendations for setting up UNIX System Services filesystems in z/OS. HFS or zFS? We recommend that you use ZFS rather than HFS when allocating the configuration datasets for the nodes. ZFS provides improved performance over HFS and additional function, such as cloning for making copies and logging for increased availability and recovery. (IBM announced the stabilization of HFS filesystems in February, 2004.) Single shared file system or separate? We recommend you create separate configuration HFS/ ZFS filesystems per node. This allows for more granular back-up and restore at the HFS data set level. It also provides a way to avoid shared HFS between MVS images. The performance of write operations to HFS not owned by the MVS operating system is much slower. It is better to have each node’s HFS owned by the operating system on which the node’s servers run. Mount points The terms root and home are often used to describe where in the UNIX system services file system an HFS or zFS filesystem is mounted. Abbreviations used in this book to describe root directories in the HFS: was_smpe_root WAS SMP/E installation root directory – usually /shared/zWebSphere/V6R1 wps_smpe_root WPS SMP/E installation root directory – usually /shared/zWPS/V6R2 configuration_root Node’s mount point for the configuration file system app_server_root Application server or deployment manager’s home = {configuration_root}/AppServer profile_root Server’s profile directory = {app_server_root}/profiles/default/ on z/OS We mounted our configuration ZFS filesystems at a mount-point of /wasv61config/<cell_name>/<node_name>. For example, /wasv61config/mdnode/mddmnode/ and /wasv61config/mdnode/mdnodea/. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 63 The root (/) filesystem should be mounted Read-Only. Under the root should be a small filesystem for /wasv61config, where the various cells could add their cell root and should be mounted Read/Write. Otherwise, you would have to change the root to R/W, create the new mount point, and change it back to R/O. Controlling use of UID=0 We recommend that the user IDs used to install and configure WebSphere Process Server for z/OS not have SuperUser authority (UID=0.) We would advise tight control of users with access to BPX.SUPERUSER. Using surrogate profiles Instead of passing around the password for administrator’s user ID, you can use the RACF SURROGAT class as you can see in Example 2-2. Example 2-2 Setting up a surrogate profile for the WebSphere administrator RDEFINE SURROGAT BPX.SRV.MDADMIN UACC(NONE) PERMIT BPX.SRV.MDADMIN CLASS(SURROGAT) ID(FRED) ACCESS(READ) SETROPTS RACLIST(SURROGAT) REFRESH Then, authorized users (like FRED in the example above), could switch to the administrator’s user ID with the following form of the su command, without knowing the password for the mdadmin ID: su -s mdadmin Using filesystem privileges instead of superuser Instead of using a super-user ID, use the various UNIXPRIV profiles and RACF PERMISSIONS to the SUPERUSER.FILESYS** profiles in the UNIXPRIV class. See the job in WASCFG.MDBASEA.CNTL(@NOUID0) and Example 2-3. Example 2-3 Sample RACF commands to create profiles to avoid use of UID 0 on installer ID /* Define the profiles */ RDEF UNIXPRIV SUPERUSER.FILESYS RDEF UNIXPRIV SUPERUSER.FILESYS.MOUNT RDEF UNIXPRIV SUPERUSER.FILESYS.CHOWN RDEF UNIXPRIV SUPERUSER.FILESYS.CHANGEPERMS RDEF UNIXPRIV SUPERUSER.FILESYS.PFSCTL /* PE PE 64 UACC(NONE) UACC(NONE) UACC(NONE) UACC(NONE) UACC(NONE) Permit the configration group access and sysprogs SUPERUSER.FILESYS CLASS(UNIXPRIV) + ID(MDCFG,SYS1) ACCESS(CONTROL) SUPERUSER.FILESYS.MOUNT CLASS(UNIXPRIV) + ID(MDCFG,SYS1) ACCESS(UPDATE) z/OS: WebSphere Business Process Management V6.2 Production Topologies */ */ */ */ */ */ PE SUPERUSER.FILESYS.CHOWN ID(MDCFG,SYS1) PE SUPERUSER.FILESYS.CHANGEPERMS ID(MDCFG,SYS1) PE SUPERUSER.FILESYS.PFSCTL ID(MDCFG,SYS1) CONNECT HUTCHJM GROUP(MDCFG) CONNECT GATES GROUP(MDCFG) /*activate */ SETR RACLIST(UNIXPRIV) REFRESH CLASS(UNIXPRIV) + ACCESS(READ) CLASS(UNIXPRIV) + ACCESS(READ) CLASS(UNIXPRIV) + ACCESS(READ) 2.3.6 Using intermediate symbolic links Symbolic links are files that contains pointers to other files or directories, usually in a different filesystem. They can be valuable by providing flexibility not to bind your configuration file system to a specific copy of a SMP/E-maintained copy of a product HFS. When you configure a node, you need to specify the paths to the product mount points. We suggest that you create intermediate symbolic links that indirectly reference the physical mount points of the products rather that referring directly to the mount points. WebSphere products for z/OS delivered as separate HFS files WebSphere Application Server for z/OS ships a product HFS filesystem that contains files and code needed to support a WebSphere Application Server for z/OS configuration. This product HFS is shipped as hlq.SBBOHFS, where hlq is the high level qualifier you give it. For example, WAS610.WAS.SBBOHFS. This is mounted at a UNIX Systems Services mount point (such as /usr/lpp/zWebSphere/V6R1). The Feature Pack for Web Services that is required by WebSphere Process Server for z/OS resides in a separate product HFS. It is installed into the Optional Materials (FMID JIWO610) product HFS. This is mounted at a mount point such as /usr/lpp/zWebSphere_OM/V6R1 separately from the WebSphere product HFS. WebSphere Process Server for z/OS also resides in a separate product HFS. It is usually mounted at /usr/lpp/zWPS/V6R2. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 65 When you configure a node, you need to specify the paths to the WebSphere Application Server for z/OS and WebSphere Process Server for z/OS product mount points. We recommend that you create intermediate symbolic links that indirectly reference the physical mount points of the products rather that referring directly to the mount points. Why use intermediate symbolic links? Intermediate symbolic links allow you easily to switch a node from one service level to another by deleting and redefining the intermediate symbolic links. Note: For a full discussion on this best practice, refer to White Paper WP100396, Test, Production and Maintenance: The essential concepts needed to know how to configure isolation between environments, available from the following Web page: http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100396 We recommend that you use intermediate symbolic links to have the flexibility to upgrade one node at a time. By using intermediate symbolic links, the upgrade granularity is at the level of a node. If you do not use intermediate symbolic links, an upgrade to a new service level of WebSphere Process Server for z/OS will affect all the nodes that refer to the product’s mount point. You will have to stop all the nodes on the LPAR referring to the product ZFS, unmount it, mount the new service level, and restart all the nodes on the LPAR. In other words, when you do not use an intermediate symbolic link the upgrade granularity is that of the LPAR. Note: The zPMT Version 7, which creates the BBOWHFSA, BBOMHFSA, and BBODHFSA jobs, automatically creates the intermediate symbolic links for the WebSphere Application Server for z/OS, and Feature Pack for Web Services Feature Pack for WebSphere Application Server for z/OS SMP/E HFS’s. The symbolic link for WebSphere Process Server for z/OS is not created by these jobs. You must do it manually. This defect should be fixed in the next release. Use sample job @DMSYML in the WPS.CNTL file of the additional materials to create the symbolic links for WebSphere Process Server for z/OS. See Appendix A, “Additional material” on page 615. You can also define these manually, as shown in Example 2-4 on page 67, but they must be done before the job that populates the configuration HFS (BBOWWPFA). 66 z/OS: WebSphere Business Process Management V6.2 Production Topologies Example 2-4 Creating a symbolic link for WebSphere Process Server for z/OS cd /wasv6config/mdcell/mdnodea ln -s /usr/lpp/zWPSMP/V6R2 wps_smpe 2.3.7 Planning security There are many decisions you must make about security. Running without security in a production environment is not an option. Your careful approach and good planning can help prevent problems down the road. We recommend that you enable security when you first configure the WebSphere Application Servers. This helps you create the necessary user IDs, profiles, keyrings, and certificates at the outset, and is much easier that trying to retro-fit security later on. When you enable security in a WebSphere Process Server for z/OS cell, you have to choose a user registry. You can use a LocalOS, SAF-based user registry like RACF, an LDAP user registry, a custom registry, or a federated registry. In preparing this Redbooks publication we used a RACF user registry. Therefore, there are several steps that help you configure WebSphere Process Server for z/OS and WebSphere Business Services Fabric for z/OS to use RACF. The WPS_RACF worksheet on the spreadsheet described in 2.2.2, “Using a spreadsheet to plan names for configuration objects” on page 45 can be used to help plan all the user IDs, groups and EJBROLE profiles that would be needed in RACF. We do not describe in detail how to configure an LDAP user registry, but the spreadsheet includes a worksheet called WPS_LDAP, which contains entries in ldif format for loading into Tivoli Directory Server for z/OS. Details about the LDAP server are set on the Variables worksheet. The users and groups are derived from the other worksheets. Most of them come from the WPS_RACF worksheet where the majority of WebSphere Process Server for z/OS and WebSphere Business Services Fabric for z/OS user IDs and groups are planned. It is important to have a clear security plan before you start. The zProfile Management Tool (zPMT) will generate JCL and RACF commands that configure RACF to support the needs of WebSphere Application Server for z/OS, but you must create your own commands to support the needs of WebSphere Process Server for z/OS and WebSphere Business Services Fabric for z/OS. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 67 We suggest that you use the spreadsheet together with the MDRACF and MDEJBROL jobs that are provided in the additional material that accompanies this Redbooks publication and are described in Appendix A, “Additional material” on page 615. For most of the roles we defined a RACF group that would uniquely provide access to that role. This would allow us to give a user ID that role by connecting it to that group. Some of the EJBROLE profiles were set to UACC(READ), which is the same as making the role available to all authenticated users. For some roles we defined a new user ID that was connected to the role group. For example, user ID MDBPEADM was defined in group MDBPEADG, which was on the access list for the EJBROLE MD.BPESystemAdministrator. You can use your existing user IDs and simply connect them into the correct role groups. However, when you configure BPC you are prompted for both a user ID and a group that is associated with the roles of BPESystemAdministrator and HumanTaskSystemAdministrator. We thought it was better to create a specific task user ID for this, rather than to use an existing, personal, user ID that might not be persistent in the system. The fundamental choice you have to make is what user registry to use. Will you use a SAF user registry such as RACF, or an LDAP user registry such as Tivoli Directory Server for z/OS? The choice is influenced by many factors, but it is certainly important to know which one you will use before you attempt to configure WebSphere Process Server for z/OS. Even if you decide to use RACF as the user registry for WebSphere Process Server for z/OS, you will need to define an LDAP server for the users that access WebSphere Business Services Fabric for z/OS. Why use Tivoli Directory Server for z/OS? LDAP servers are available on many platforms, but because this Redbooks publication is about WebSphere Process Server for z/OS this section describes some of the reasons to consider running LDAP on z/OS. There are two different LDAP Servers available on z/OS since z/OS 1.8. z/OS Integrated Security Services (ISS) LDAP IBM Tivoli Directory Server (TDS) for z/OS V6.0 LDAP IBM Tivoli Directory Server (TDS) for z/OS is preferred because the ISS LDAP is functionally stabilized and its schema is less compatible with other LDAP servers. 68 z/OS: WebSphere Business Process Management V6.2 Production Topologies Note: The TDS z/OS schema is compatible with TDS on other platforms. Products like Tivoli Access Manager, for example, can use TDS on z/OS as their user registry. A non-z/OS WebSphere Application Server can also use the TDS z/OS LDAP as its user registry. Tivoli Directory Server for z/OS offers several advantages compared to a non-z/OS LDAP server that must be taken into account by z/OS customers when selecting an LDAP server. These advantages are discussed below. TDS for z/OS is shipped with the base z/OS operating system and is therefore available at no charge to z/OS customers. TDS for z/OS has the LDAP Native Authentication (LNA) feature. Native authentication is where a user’s password is held in RACF rather than on the userpassword attribute in the LDAP database. Using LNA can increase the security of powerful system administration user IDs such as those with access to the Integrated Solutions Console. Such users can be defined as LNA user IDs where their passwords are managed with RACF, while other normal users are held in the LDAP database. Passtickets can be used with LNA user IDs, also. TDS for z/OS has its database in DB2 z/OS, which can be shared across a sysplex: – You have free high availability (HA) because you start an LDAP server on each LPAR in the sysplex, and the load is balanced across the LDAP Servers by sysplex distributor. Tune the LDAP with properties that control a number of concurrent connections, and so forth. You just need the processor/memory to support the load you need. – Disaster recovery for TDS z/OS is free because DB2 z/OS data is nearly always part of an existing, funded, tested, disaster recovery solution. Therefore, you avoid the cost of providing and testing disaster recovery of an LDAP server on some other platform. You can also configure the TDS z/OS in failover mode, rather than load sharing mode, just as on other platforms. You can even replicate between sysplex nodes. TDS z/OS uses the z/OS UNIX file system, which is almost certainly more secure that any other UNIX file system because the powerful privileges are controlled by RACF. This means that the TDS z/OS LDAP is more secure than one running on non-z/OS platforms. The passwords of users held in the LDAP database can be encrypted with the private key stored in the z/OS hardware. This means that there are no keystore files protected by passwords that can be hacked. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 69 The LDAP configuration file, which needs to be protected against unauthorized read/write, can be held in a z/OS partitioned dataset, which makes it even harder for someone to gain access to the LDAP configuration. TDS LDAP z/OS also provides an SDBM schema that maps directly on to the RACF database. This allows a security administrator to use a graphical LDAP client to manage the RACF database. This might be useful for some customers. TDS on z/OS can use the hardware crypto cards on the System z machine to manage crypto processing related to SSL. This gives excellent scalability of logins. For scalability in general, the DB2 for z/OS subsystem that provides the database for TDS is a scalable, available, database. Typically, the majority of frequently used LDAP pages would end up in memory in the coupling facility (that is, in DB2 global buffer pools). So access to LDAP data is at memory speeds. Also, you can run an LDAP server on each LDAP of your sysplex. That gives great availability and scalability. The TDS for z/OS LDAP server also has a new internal cache management implementation, with intercache consistency in a sysplex. 2.3.8 Planning RACF user IDs, groups and EJBROLE profiles When configuring a WebSphere Application Server for z/OS V6.1 cell using the zProfile Management Tool (zPMT), jobs are generated that configure the RACF user IDs, groups, and profiles that are required by the WebSphere Application Server for z/OS cell. We discuss the jobstreams in 2.5.4, “Running the configuration jobs to create the standalone server” on page 108 The installation and configuration of WebSphere Application Server for z/OS creates many RACF profiles specific to a particular application server or deployment manager. This makes it difficult when you expand your configuration at a later time. Techdoc WP101427 describes an alternative method for defining profiles in the CBIND, SERVER, STARTED, and other RACF classes that are more generic and allow expansion of your configuration. Additional RACF user IDs, groups, and profiles are required for WebSphere Process Server for z/OS. In 2.2.2, “Using a spreadsheet to plan names for configuration objects” on page 45, we created a sample job called MDRACF that defined all the user IDs and groups we had planned for our cell. Note: The sample jobs that setup the WPS user IDs, groups, and roles cannot be run until job BBOSBRAK has been run to create the WebSphere Application Server for z/OS administrative groups and user IDs. 70 z/OS: WebSphere Business Process Management V6.2 Production Topologies Using EJBROLE profiles We also created a job called MDEJBROL that defined RACF EJBROLE profiles to support the roles used by WebSphere Process Server for z/OS, including CEI, and Business Space. This job creates one profile per role. If you want to group roles together in fewer profiles, see “Using GEJBROLE profiles” on page 73. It is a good idea to create all the required user IDs, groups, and EJBROLE profiles before you configure WebSphere Process Server for z/OS. The user IDs, groups, and EJBROLE profiles we created are summarized in Table 2-1. Roles and groups in Table 2-1 and Table 2-2: The roles and groups in these tables reflect a resource-oriented approach to defining RACF profiles, where there is a profile for each role and a RACF group provides access to that role. Some roles have been defined as UACC(READ), meaning that anyone that has authenticated will have that role. You may not want to use UACC(READ) for the roles shown in the tables. We have done this for the API roles to show you that you can use UACC(READ). An unauthenticated (guest) user ID cannot access roles that are UACC(READ). We discuss an alternative approach called organization-oriented in“Using GEJBROLE profiles” on page 73. You may wish to adopt an organization-oriented approach to simply the RACF setup for EJB™ roles Table 2-1 WebSphere Process Server for z/OS user IDs, groups and roles Group or User Description RACF EJBROLEs Group ID, User ID or ‘UACC(READ)’ BPE API User Group MD.BPEAPIUser UACC(READ) BPE Administrator’s Group MD.BPESystemAdministrator MDBPEADG BPE Admin User MDBPEADM BPE System Monitor’s Group MD.BPESystemMonitor BPE SysMon User MDBPESMG MDBPESM Human Task API User Group MD.TaskAPIUser UACC(READ) Human Task Administrator’s Group MD.TaskSystemAdministrator MDHTADMG HT Admin User MDHTADM Human Task System Monitor’s Group MD.TaskSystemMonitor MDHTSMG Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 71 Group or User Description RACF EJBROLEs HT SysMon User BPC Web Client Role Group MDHTSM MD.WebClientUser BPC Web Client User BPC IVT User Role Group Group ID, User ID or ‘UACC(READ)’ MDWCUG MDWCU MD.BPCIVTUser MDB RunAs Role User Group MDWCUG MDMDBG MDB JMSAPIUser Role MD.JMSAPIUser MDJMSAPI MDB Escalation User MD.EscalationUser MDESCAL CEI Event Administrator Group MD.eventAdministrator MDEVTADG CEI Event Admin User MDEVTADM CEI Event Consumer Group MD.eventConsumer MDEVTCO CEI Event Creator Group MD.eventCreator MDEVTCRG CEI Event Updater Group MD.eventUpdater MDEVTUPG CEI Catalog Administrator Group MD.catalogAdministrator MDCATADG CEI Catalog Admin User MDCATADM CEI Catalog Reader Group MD.catalogReader MDCATRG Failed Event Manager User Group MD.WBIOperator MDCFG, MDBPEADG, MDEVTADG Business Space Administrator Group MD.Administrator MDBSPADG Cleanup User MD.CleanupUser MDCLEANU Cleanup User Group MDCLEANG There are a number of roles associated with WebSphere Application Server for z/OS, WebSphere Process Server for z/OS, and WebSphere Business Services Fabric for z/OS. 72 z/OS: WebSphere Business Process Management V6.2 Production Topologies When using RACF to perform EJB role authorization checking there are two main ways to approach the management of the RACF profiles: Resource-oriented approach This approach starts by creating profiles to protect all the resources (in this case EJBROLE profiles), then considering the access list for each profile. Organization-oriented approach This approach starts by considering how many different types of users there are, then defining the minimum number of profiles required to provide the correct access. We took the granular resource-oriented approach by defining an EJBROLE profile for each role and creating a number of different RACF groups that we permitted access to each role. A drawback to this approach is that it leads to a large number of EJBROLE profiles. The organization-oriented approach results in fewer RACF profiles and is easier to maintain, and helps performance. Adding users, groups and EJBROLEs for Fabric See 5.2.2, “Configure security” on page 209 Using GEJBROLE profiles When following the organization-oriented approach, a useful technique is to define RACF GEJBROLE profiles rather than EJBROLE profiles. When you define a profile in class EJBROLE, it refers to only one role. When you define a profile in class GEJBROLE, it can contain more than role. Roles are added to a GEJBROLE profile using the ADDMEM(<role>) option. For example, suppose that your organization consists of the two following organizational groups: People responsible for administering WebSphere Process Server for z/OS Ordinary users that may need certain roles to use applications You can create a GEJBROLE profile called WPSAdmins and add to it all the roles that provide administrator privileges. You can create a second GEJBROLE profile called WPSUsers and add to it all the roles that an ordinary user needs to run. Having only two GEJBROLE profiles is at the opposite extreme to having one EJBROLE profile for each role. You may find that the correct solution for your organization lies between these two extremes. It is not possible to define only two GEJBROLE profiles for administrators and users, for example, because you will need to define at least two more profiles to support the JMSAPIUser and EscalationUser runAs roles. We recommend the organization-oriented approach, however, because it will reduce 20 profiles to around five. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 73 A possible solution using GEJBROLEs is shown in Table 2-2. Notice that administrators are in RACF group WPSADM, and that group is on the access list of all profiles. Roles and groups in Table 2-1 and Table 2-2: The roles and groups in these tables reflect a resource-oriented approach to defining RACF profiles, where there is a profile for each role and a RACF group provides access to that role. Some roles have been defined as UACC(READ), meaning that anyone that has authenticated will have that role. You may not want to use UACC(READ) for the roles shown in the tables. We have done this for the API roles to show you that you can use UACC(READ). An unauthenticated (guest) user ID cannot access roles that are UACC(READ). We discuss an alternative approach called organization-oriented in“Using GEJBROLE profiles” on page 73. You may wish to adopt an organization-oriented approach to simply the RACF setup for EJB roles Table 2-2 Possible solution using GEJBROLEs 74 Organizational role GEJBROLE profile name Roles added to the GEJBROLE profile Groups on Access List Administrators WPSAdmins BPESystemAdministrator, BPESystemMonitor, TaskSystemAdministrator, TaskSystemMonitor, BPCIVTUser, eventAdministrator, catalogAdministrator, WBIOperator, ObserverUser, Administrator WPSADM API Users WPSAPIUsers BPEAPIUser, TaskAPIUser, BusinessRuleUser WPSADM, WPSAPIU Normal Users WPSUsers eventConsumer, eventCreator, eventUpdater, catalogReader, WebClientUser WPSADM, WPSUSR JMSAPIUser runAs role JMSAPIUser JMSAPIUser none z/OS: WebSphere Business Process Management V6.2 Production Topologies Organizational role GEJBROLE profile name Roles added to the GEJBROLE profile Groups on Access List Escalation runAs role EscalationUser EscalationUser none Cleanup service runAs role CleanupUser CleanupUser none The JMSAPIUser role is a runAs role, whose purpose is to map the role to a user ID (which for our MD cell was MDJMSAPI). The MD.JMSAPIUser GEJBROLE profile needs no access list, and because the runAs user ID is assigned to the role on the APPLDATA option of the GEJBROLE profile, you cannot group it together with other roles in the same GEJBROLE profile. Note: Once you have decided to use GEJBROLE profiles, define all EJB role profiles in that class. Do not mix EJBROLE and GEJBROLE profiles. Although a GEJBROLE profile containing only one role is the same as a single EJBROLE profile, it is best to define the GEJBROLE profile with one member. RACF considers profiles in both classes and merges the access lists of any roles that appear in more than one profile. This can cause confusion. We created sample JOB GEJBROLE to implement the role assignments in the table above for our MD cell. However, you cannot run this job before the BBOSBRAK job that is created and run as part of the server installation in 2.5.4, “Running the configuration jobs to create the standalone server” on page 108. We suggest you run these jobs to implement role assignments in 2.5.5, “Defining user IDs, group IDs, and EJBROLE profiles” on page 110. 2.4 Preparing DB2 It is important to include your database administrator (DBA) or an experienced DB2 expert when you plan for, and configure WebSphere Process Server for z/OS. This section describes the planning, preparation, and execution of the DB2 database, tables and other required DB2 artifacts. 2.4.1 DB2 terminology There are many DB2 constructs you will need to set up your databases for WebSphere Process Server. Sample names in this paper should help clarify these terms. Work with your DBA to assign the names and perform the tasks. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 75 For a good paper on DB2 concepts, see DB2 Basics: Table Spaces and Buffer Pools, available at the following Web page: http://www.ibm.com/developerworks/data/library/techarticle/0212wieser/0 212wieser.html VCAT High-level dataset name qualifier (or alias) for the VSAM backed datasets for WebSphere Process Server databases. Volumes Contain VSAM backing datasets for these databases, or an SMS-managed storage class. DB2 Storage groups (STOGROUPs) for databases that help them to be managed and refreshed as a group. Database Collection of tables referencing the above storage group. Table spaces VSAM datasets that contain tables with records in them. Tables Data objects consisting of rows and specified columns. Indexes VSAM-backed files containing pointers to rows in a table. Schema name Qualifier (typically CREATOR) to uniquely define SQL definitions (tables, views, procs, indices) and avoid name collisions across databases and between tables for multiple SIBs in a server. Note: Table Names must be unique within a subsystem (ssid) on zOS where DB2 subsystems can access multiple Databases. User IDs To connect to the databases from the WPS runtime (i.e., authorization IDs and SQL IDs). These are usually IDs with DBADM authority. DB2 stored procedure builder must be enabled in your DB2 subsystems (i.e., built with the DSNTPSMP utility) if you are using Relationships. (The Stored Procedure is used as multiple Relationships will exist for each Key. When the relationship is updated there are typically several rows that are being updated/deleted. The storedProcedure is used to manage these updates locally to the database.) 76 z/OS: WebSphere Business Process Management V6.2 Production Topologies STEPLIBs If you do not have the DB2 libraries in your linklist concatenation, you will have to add them to your Procs, and shell scripts. Buffer Pools See “Planning DB2 buffer pools” on page 80. SSID DB2 subsystem identifier. Java Common Connectivity (JCC) This DB2 feature supports the IBM DB2 Driver for JDBC™ and SQLJ. Additional DB2 tailoring may be required to support this feature. See (DSNTIJSG). 2.4.2 Planning DB2 names WebSphere Process Server for z/OS involves many components requiring many DB2 tables. It is important to organize and document the DB2 objects to ensure a smooth and successful implementation. For example, you should decide on a naming convention for the following: Databases Storage groups VCATs Tables Schema prefixes Bufferpools Authentication alias IDs Plan these names with your DB2 system administrator. The spreadsheet we describe in 2.2.2, “Using a spreadsheet to plan names for configuration objects” on page 45 includes a worksheet that can help you choose names for the databases if you decide to have separate databases for each component. A key question you must answer is “Do I put all the WebSphere Process Server for z/OS components in one database or do I create a separate database for each component?” We suggest that you plan to create a single databases for all the WPS components, at least for your first implementation. The planning spreadsheet can assign a single database for all components, or separate database names to each component. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 77 Using a single database The advantages of a single data base are as follows: Easier to create all the necessary tables by running the createDB.sh script just once for all the DB2 tables. The createDB.sh script is a sample script that wraps the DButility.sh and is easier to use than DButility.sh. It is described in 2.7.3, “Executing createDB.sh to generate the DDL” on page 126. Simpler to specify and manage GRANTs (DBADM) on just one database. The createDB.sh shell script creates objects in one database. You can simply run createDB.sh once to generate (and execute) the DDL for all WPS components in the server. Easier to manage the environment as a Single database contains all the definitions for a given WPS Serve. DBA's can more easily clone the environment as well as backup its contents. If you have one database the majority of tables and indexes will end up in the same buffer pools. If this is what you want, you might choose to use only one database. If you have multiple clusters, creating multiple additional databases for each cluster can become quite cumbersome. Using a single database for each cluster would simplify this. A good practice would be to use your cluster name as your database name, and use the cluster name as the owner of the DB2 resources. Using multiple databases The reasons for using multiple databases are listed here: More control and easier to tailor the DDL for specific bufferpools, space requirements, and schema names. Easier to validate that you have all the tables and re-run the CREATE TABLESPACE and TABLE DDL files. The WebSphere Process Server for z/OS Information Center recommends using separate databases for performance reasons. That may be good advice for non-z/OS platforms. However, when using DB2 for z/OS, choosing to define multiple databases only has an indirect effect on performance, because you have the opportunity to specify the BUFFERPOOL and INDEXBP options on the CREATE DATABASE statements. Much of the DDL provided with WebSphere Process Server for z/OS does not specify bufferpools, so they are assigned the default that applies to the database, table, or index in which it is created. 78 z/OS: WebSphere Business Process Management V6.2 Production Topologies If you want to place tables and indexes in separate buffer pools, we suggest that you create separate databases and assign different BUFFERPOOL and INDEXBP values to each component, at least initially. Although the createDB.sh shell script that is described in 2.7.3, “Executing createDB.sh to generate the DDL” on page 126 creates objects in one database, you can simply run createDB.sh once for each WebSphere Process Server for z/OS component, assigning a different database name on each invocation. Using multiple databases makes it easier to manage when the tables and indexes for each component are isolated in their own database. It is possible that you will need to delegate some DB2 administration privileges for specific component databases to individuals in your organization. This is much easier to do if you GRANT DBADM to such people. The planning spreadsheet assigns different database names to each component by default. Managing DB2 objects Listed here are some observations about specifying, creating, and deleting various DB2 objects, which you may need to perform when initially setting up the DB2 databases for WebSphere Process Server for z/OS components. Messaging engine tables The typical tables that need to be rebuilt are the message engines. You can do this using createDB.sh with the *DBSIB option. The fix for APAR PK35226 in WebSphere Application Server for z/OS 6.1.0.19 added a new table, SIBOWNERO, to improve the locking mechanism for the messaging engines, so there are now twelve tables in each messaging engine database. Production tables You should not be recreating the WPS and HTM tables on the fly; They contain references to process instances and templates that would require you to uninstall applications. In addition, you may not be able to uninstall the application if you do this. Deleting the WPSDB common tables may cause the migration utilities to run on server startup, which does not work well on zOS. Functions and Sequences These objects exist outside the database. They will not be dropped by DROP DATABASE commands; you need explicit DROPs. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 79 If you drop a data base and re-create the tables with DDL containing CREATE FUNCTION and CREATE SEQUENCE statements, you will get SQLCODE = -601 errors and the remaining DDL statements may be flushed (unless you allow exceptions to occur). See Example 2-5. Example 2-5 DROPing Functions and Statements—Sample DDL DROP FUNCTION M1WPS.INTERVALIN (INTERVAL INT, TS_START TIMESTAMP, TS_END TIMESTAMP); SET CURRENT SQLID = 'M1DBU'; DROP SEQUENCE SEQ_W_OBJ_LIT_ANY_ID DROP SEQUENCE SEQ_W_OBJ_LIT_DATE_ID ; ; Planning DB2 buffer pools DB2 data requests for data and indexes may be filled from memory rather than disk to help performance. Buffer pools use fixed page sizes, which must be the same as the page size of any table spaces they serve. A default buffer pool can be associated with a database and can be used by more than one database and table space. It is better to separate catalog, data, index and workfiles into separate bufferpools. Generally, we should have fewer but larger bufferpools and allow DB2 to manage the resources. If we identify performance issues based on the workloads we may choose manually to isolate table spaces into their own bufferpool. You should not use bufferpools BP0, BP8K0, or BP16K0, because these are used by the DB2 catalog and directory, and reserved in most installations for the DB2 subsystem. Unfortunately, some generated DDL defaults to these values, so you should review all DDL before executing it and set the correct bufferpools. Although it may be easier to assign bufferpools at the database level, they can also be assigned on the TABLESPACE definitions in the generated DDL before you execute the DDL. After you have some experience with WebSphere Process Server for z/OS, you can alter the buffer pool assignments using the DB2 ALTER TABLESPACE or ALTER INDEX commands. However, it will not physically change them unless you REORG the table space or REBUILD the indexes, which may take some time. 80 z/OS: WebSphere Business Process Management V6.2 Production Topologies Although it is easier to drop them and recreate them, you cannot do that with production tables. Therefore, the only means is to use a REORG and REBUILD so you do not need to drop them and re-create. Work with your DB2 database administrator to plan the buffer pools you will use. We used the spreadsheet described in 2.2.2, “Using a spreadsheet to plan names for configuration objects” on page 45 to plan all the names and buffer pools we used in DB2. The table on the Variables worksheet that plans database names and bufferpools does not allow you to choose BP0, BP8K0, or BP16K0. For each bufferpool we issued -ALTER BUFFERPOOL(BPxxx) VPSIZE(yyyy) to define a buffer pool BPxx with yyyy pages. Because we were configuring a cluster, which in turn required DB2 in data-sharing mode, any new buffer pools we added also required cache structures to be defined in the Coupling Facility for Global Buffer pools. For instruction on how to do this, refer to the DB2 for z/OS Information Center article Defining coupling facility structures available at the following Web page: http://publib.boulder.ibm.com/infocenter/dzichelp/v2r2/topic/com.ibm.db 29.doc.dshare/db2z_definingcfstructures.htm Naming the databases, storage groups, schemas and VCATs If you have one database the names of these DB2 components is relatively simple. If you have separate databases and storage groups for each component, their names should be well-organized to avoid confusion during the configuration phase, managing the runtime, and in problem determination. We are using the same components for all tables whether we are using the same database, or multiple databases: VCAT (WASCFG) Storage Group (MDWPSSG) Using a single database, we used MDWPSDB. See Table 2-3 on page 82 for a list of the database names for each component. The schema qualifier may vary, but we chose MDWPS for all our tables except for the messaging engines, and the Business Monitor. The table names are the same for all messaging engines, so they have to be schema-qualified with different prefixes to make them unique. The Business Monitor database contains some tables with duplicate names in the common database, such as SCHEMA_VERSION, and D2D_ITEM. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 81 Table 2-3 Layout of our DB2 database and schema names for the MD cell Component DB Name Schema Common WPS & ESB MDWPSDB MDWPS BPE/HTM MDWPSDB MDWPS CEI MDWPSDB MDWPS Business Space MDWPSDB MDWPS Fabric MDWPSDB MDWPS Monitor MDWPSDB MDMON SCA SYS Bus MDWPSDB MD01S SCA APP Bus MDWPSDB MD01A BPC Bus MDWPSDB MD01B CEI Bus MDWPSDB MD01C Fabric Bus MDWPSDB MD01F Monitor Bus MDWPSDB MD01M If you have separate databases and storage groups for each component, their names should be well-organized to avoid confusion during the configuration phase, managing the runtime, and in problem determination, as shown in Table 2-4. Table 2-4 Alternative layout using separate databases for the MD cell 82 Component DB Name Schema Bufferpools Common WPS & ESB MDWPSDB MDWPS BP1, BP2 BPE/HTM MDBPCDB MDWPS BP5, BP6 CEI MDCEIDB MDWPS BP77, BP8 Business Space MDBSPDB MDWPS BP11, BP12 Fabric MDFABDB MDWPS BP13, BP14 Monitor MDMONDB MDMON BP15, BP16 SCA SYS Bus MDSIBDB MD01S BP3, BP4 SCA APP Bus MDSIBDB MD01A BP3, BP4 BPC Bus MDSIBDB MD01B BP3, BP4 z/OS: WebSphere Business Process Management V6.2 Production Topologies Component DB Name Schema Bufferpools CEI Bus MDSIBDB MD01C BP3, BP4 Fabric Bus MDSIBDB MD01F BP3, BP4 Monitor Bus MDSIBDB MD01M BP3, BP4 2.4.3 Choosing currentSchema or currentSQLID An understanding of schema and how it relates to sqlid is essential to configure WebSphere Process Server for z/OS successfully. This section contains background information about schemas in DB2 for z/OS. An option is to define all the WebSphere Process Server for z/OS tables with a schema equal to the user ID associated with the WebSphere servant job. If you choose this option you do not need to code either currentSchema or currentSQLID. You do not need to specify any Authentication Alias on your type2 datasources either. See 2.4.5, “Using the started task ID to connect to the database” on page 91. What is the schema? The currentSchema and currentSQLID properties can be set on the custom properties of a data source. Both properties affect the qualifier used when an application issues unqualified SQL. What is qualifier and what is unqualified SQL? When an application issues an SQL statement like SELECT * FROM MYTABLE, this is unqualified SQL because the table name is not qualified like <qualifier>.MYTABLE. There can be multiple instances of a table called MYTABLE in DB2 for z/OS. A qualifier is used to distinguish one instance of MYTABLE from another. The default qualifier is the authorization ID (usually the user ID) executing the unqualified SQL statement. So, if the user ID executing the SQL statement is FRED, then SELECT * FROM MYTABLE is equivalent to SELECT * FROM FRED.MYTABLE. The term qualifier is more commonly known as the schema, so in this Redbooks publication we will talk about schema rather than qualifier. WebSphere Process Server for z/OS implications When an SQL request is made by an application running in WebSphere Process Server for z/OS, there are several ways of specifying the user ID under which the request runs. It is possible to influence the schema (qualifier) by setting the Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 83 appropriate user ID when making the request. Alternatively, it is possible to execute the SQL request under any user ID, but specify the schema you want to use when making unqualified requests. Another possibility is that the application discovers the schema it needs to use at runtime (perhaps from an xml file) and issues fully-qualified SQL requests rather than unqualified SQL requests. When an application issues fully-qualified SQL there is no issue to resolve. The question of whether you need to use currentSchema or currentSQLID arises only when applications make unqualified SQL requests. Unqualified SQL from WebSphere Process Server for z/OS Currently, several components of WebSphere Process Server for z/OS make unqualified SQL requests. This means that if you do not code either currentSchema or currentSQLID on the data source custom properties, the SQL request will inherit a schema equal to the data source’s J2C authentication alias user ID. For example, if you have configured a data source with a J2C authentication alias for user ID MDDBU, a statement like SELECT * FROM MYTABLE will execute as SELECT * FROM MDDBU.MYTABLE in DB2. Using the J2C authentication alias as the schema One option is to define all the WebSphere Process Server for z/OS tables with a schema equal to the J2C authentication alias user ID. If you choose this option you do not need to code either currentSchema or currentSQLID. If this is the easy option, why not do it? The reason not to use the J2C authentication alias as the schema is that DB2 for z/OS gives certain implicit privileges to the schema and these privileges give more power to the WebSphere application than a DBA will normally allow. The implicit privileges include DROP, for example. Normally, DROP is not something that an application is allowed to do. For this reason, the DB2 for z/OS DBA may refuse to allow tables to be created with a schema equal to the user ID that accesses those tables. Many DB2 for z/OS DBAs have a rule that no table can be created with a qualifier equal to any user ID. They use a security policy that says that all schemas must be set to RACF groups, rather than user IDs. 84 z/OS: WebSphere Business Process Management V6.2 Production Topologies For these reasons you will probably not want the user ID executing the SQL request to be equal to the schema. This means that tables have to be created with some other schema name, and there needs to be a way to specify the schema that is used when an application makes an unqualified SQL request. Using the Started Task ID to connect to the database One option is to define all the WebSphere Process Server for z/OS tables with a schema equal to the user ID associated with the WebSphere servant job. If you choose this option you do not need to code either currentSchema or currentSQLID. You do not need to specify any Authentication Alias on your type2 datasources either. See 2.4.5, “Using the started task ID to connect to the database” on page 91. Specifying the schema with currentSchema In the custom properties of a data source, there is a property called currentSchema. If the application makes an unqualified SQL request, and you specify a value in currentSchema, the SQL request is qualified by the value of currentSchema. For example, if all the WebSphere Process Server for z/OS tables have been created with a schema of MDWPS, specifying currentSchema = MDWPS will allow all unqualified SQL statements from WebSphere Process Server for z/OS to operate on the set of tables that are qualified with MDWPS. The authorization ID in DB2 used for unqualified SQL requests is the J2C authentication alias user ID. Therefore, an implication of using currentSchema is that the J2C authentication alias user ID must be granted the privileges it needs to work with the tables, indexes, sequences, and functions in DB2. For example, if the J2C authentication alias is MDDBU and the schema is MDWPS, for MDDBU to issue SELECT * FROM MYTABLE it is necessary to issue the following command: GRANT SELECT ON MDWPS.MYTABLE TO MDDBU Specifying the schema with currentSQLID In the custom properties of a datasource, there is a property called currentSQLID. This property causes the user ID under which an SQL statement executes to switch to the user ID named in the currentSQLID property. For example, on a data source that has a J2C authentication alias for user ID MDDBU, SQL requests will execute under user ID MDDBU. However, coding currentSQLID = MDWPS is the same as issuing the statement SET CURRENT SQLID = ’MDWPS’ before executing the application’s SQL statement. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 85 Therefore, if currentSQLID = MDWPS, an application issuing the SQL command SELECT * FROM MYTABLE will switch their current SQLID special register to MDWPS and execute SELECT * FROM MYTABLE. Because the schema will default to the SQLID, this has the same affect as issuing the following command: SELECT * FROM MDWPS.MYTABLE Why not use currentSQLID then? You can read in the Information Center that currentSQLID can be used to specify the schema. You will notice that WebSphere Process Server for z/OS adds the schema to the currentSQLID property on its datasource called WBI_Datasource. So what is wrong with using currentSQLID? The use of currentSQLID is something you might do in a non-z/OS environment, which is why it appears in documentation. In a non-z/OS environment, the DB2 database is almost always remote with respect to the WebSphere Process Server, and is accessed across the network. In that situation you may want to use a different currentSQLID to connect to different WebSphere Process Server databases. The problem with using currentSQLID with DB2 for z/OS is that switching from one authorization ID to another is something that has to be authorized. Using currentSQLID is a bit like issuing an su (switch user) in UNIX. On a z/OS system, switching from one user ID to another is not allowed unless you have been granted the privilege to do so. If you set the currentSQLID on a data source, the J2C authentication alias user ID will need to be granted the ability to switch to the schema you want to set on currentSQLID. The Information Management Software for z/OS Solutions Information Center lists the authorization required to issue SET CURRENT SQLID. See the following Web page for this information: http://publib.boulder.ibm.com/infocenter/dzichelp/v2r2/index.jsp Here is an quote from the Web page: If any of the authorization IDs of the process has SYSADM authority, CURRENT SQLID can be set to any value. Otherwise, the specified value must be equal to one of the authorization IDs of the application process. This rule always applies, even when SET CURRENT SQLID is a static statement. Therefore, a user ID with SYSADM authority can accomplish this, but you do not want to grant DB2 SYSADM authority to the J2C authentication alias user ID. 86 z/OS: WebSphere Business Process Management V6.2 Production Topologies The phrase “equal to one of the authorization IDs of the application process” in the quote above means that the user ID can switch to an SQLID that is equal to one of the RACF groups to which it is connected. For example, if the J2C authentication alias is connected to a RACF group whose name is equal to the schema, it can switch SQLID to the schema. For secondary authorization groups to be considered, DB2 for z/OS must be using RACF to control authorization to the objects it manages, as opposed to using DB2’s own security controls based on GRANTs. The corresponding groups and user IDs must be defined, and DB2 EXIT routines DSN3SATH and DSN3SSGN supplied by DB2 in SDSNSAMP need to be installed. For example, if all the following factors are true, the SQLID can be switched to the schema MDWPS: DB2 takes into account connections to RACF groups (secondary authorization IDs) to manage authorization checking of DB2 resources. DB2 EXIT routines DSN3SATH and DSN3SSGN must be installed. The WebSphere Process Server for z/OS tables have been created with a schema of MDWPS. A RACF group called MDWPS has been created. The J2C authentication alias is connected to RACF group MDWPS. The data source property currentSQLID is set to MDWPS. One thing to notice about doing this, however, is that connecting the J2C authentication alias user ID to the schema group is the same as giving it full ownership privileges over the tables under the schema. This is what we want to avoid. When the objective is to arrange things so that no user ID in the WebSphere cell has more than the minimum level of access to DB2 resources, it cannot be done by using currentSQLID. What if DB2 does not use RACF for authorization checking? If DB2 is not using RACF to perform authorization checking, but is using its own internal security management system, then DB2 does not take into consideration the RACF groups to which a user is connected when it makes authorization decisions. A user ID can only be authorized to switch the SQLID by being granted DB2 SYSADM authority. Schema recommendation Using the currentSQLID property is indirectly allowing WebSphere Process Server for z/OS to use the correct schema and brings additional security issues that are unlikely to be acceptable to a security-conscious DB2 system Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 87 administrator. When using the currentSQLID property, the J2C authentication alias user ID ends up with more privileges than it needs, including DROP. We suggest that you use currentSchema rather than currentSQLID. Changes in handling currentSchema from V8.1 to V9.1: The way DB2 for z/OS handles currentSchema changed from DB2 for z/OS V8.1 to V9.1. In DB2 for z/OS V8.1, it was not possible to set the currentSchema to a value different from the current SQLID without granting the current SQLID schema privileges. In other words, the SQLID (which would be the database J2C authentication alias user ID) would have to be connected to the schema group. In DB2 for z/OS V9.1, this restriction no longer applies. It is possible to use currentSchema without granting extra privileges to either the servant or J2C authentication alias user ID. Note: Although we suggest not using currentSQLID property on a data source, there are times when you will need to use the SET CURRENT SQLID command when executing the DDL to define all the WebSphere Process Server for z/OS tables in DB2 for z/OS. Issue the DB2 command SET CURRENT SQLID=’<schema>’; before executing the CREATE TABLE statements because some of the DDL is unqualified. This is explained in “Unqualified DDL considerations” on page 88. Your choice of currentSQLID or currrentSchema also affects the GRANT commands you will need to use. That is discussed in 2.4.4, “Using GRANT statements in DB2” on page 89. Unqualified DDL considerations Some of the DDL statements generated by WebSphere Process Server for z/OS that create all the tables, indexes and views in DB2 are unqualified. For example, there might be a statement like CREATE TABLE MYTABLE IN ... When a CREATE statement is unqualified, it is assigned a schema (qualifier) equal to the authorization ID executing the command. When you run the DDL to define all the WebSphere Process Server for z/OS tables it is important to execute the CREATE statements under an authorization ID equal to the correct schema. For example, when the DDL is unqualified, to create table MYTABLE with a schema of MDWPS it is necessary to issue SET CURRENT SQLID=’MDWPS’; before executing the CREATE TABLE MYTABLE statement. 88 z/OS: WebSphere Business Process Management V6.2 Production Topologies As discussed in “Specifying the schema with currentSQLID” on page 85, it is not possible to switch SQLID unless you are authorized. Someone with DB2 SYSADM authority can execute the DDL because a SYSADM can issue SET CURRENT SQLID. Or, if the user ID executing the DDL is connected into the schema’s RACF group, it is possible to switch the SQLID to that of the schema. We suggest that you ask your DB2 system administrator to execute the DDL. Ensure that you have inserted the correct SET CURRENT SQLID statements before the CREATE statements. Otherwise, all the WebSphere Process Server for z/OS tables are created with a schema equal to the authorization ID of the DB2 SYSADM. To draw attention to this, we structured our JCL that executed DDL so that the SYSIN contained SET CURRENT SQLID as the first statement. The DDL statements to be executed were concatenated on SYSIN below this. An extract of the job that executed our BPC DDL is shown in Example 2-6 to illustrate this concept. Example 2-6 The use of SET CURRENT SQLID in JCL that executes DDL . . //SYSTSIN DD * DSN SYSTEM(D9FG) RUN PROGRAM(DSNTEP2) PLAN(DSNTEP91) LIB('DB9FU.RUNLIB.LOAD') //SYSIN DD * SET CURRENT SQLID = 'MDWPS' ; // DD DISP=SHR,DSN=WASCFG.MDCELL.WPS.SQL(MDBPCTB) 2.4.4 Using GRANT statements in DB2 The GRANT statements that you will need to issue differ depending on whether you use currentSQLID or currentSchema. We did not have time to explore all the possibilities, and your particular requirements may differ, but this section will help you plan what you need to do. When using currentSQLID When using currentSQLID = <schema> the user ID executing the SQL is the same person as the schema, and so has all implicit privileges over that schema. There is no need to issue additional grants when using currentSQLID set to the schema. Nevertheless we issued the GRANTs in Example 2-7 on page 90. The sample job MDCREDB can be used to create the databases and issue the GRANTs, or if you want to use a single database, see member MDCR1DB. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 89 Example 2-7 GRANT statements issued when using currentSQLID GRANT SELECT ON SYSIBM.SYSTABLES TO MDDBU; GRANT GRANT GRANT GRANT GRANT GRANT GRANT GRANT GRANT GRANT GRANT GRANT GRANT GRANT GRANT GRANT GRANT GRANT < not DBADM ON DATABASE MDWPSDB TO MDWPS; DBADM ON DATABASE MDBPCDB TO MDWPS; DBADM ON DATABASE MDESBDB TO MDWPS; DBADM ON DATABASE MDEVTDB TO MDWPS; CREATEIN, DROPIN ON SCHEMA MDWPS TO MDWPS; DBADM ON DATABASE MD01ADB TO MD01A; DBADM ON DATABASE MD01BDB TO MD01B; DBADM ON DATABASE MD01CDB TO MD01C; DBADM ON DATABASE MD01SDB TO MD01S; CREATEIN, DROPIN ON SCHEMA MD01A TO MD01A; CREATEIN, DROPIN ON SCHEMA MD01B TO MD01B; CREATEIN, DROPIN ON SCHEMA MD01C TO MD01C; CREATEIN, DROPIN ON SCHEMA MD01S TO MD01S; USE OF STOGROUP MDWPSSG TO MDWPS; USE OF STOGROUP MDWPSSG TO MD01A; USE OF STOGROUP MDWPSSG TO MD01B; USE OF STOGROUP MDWPSSG TO MD01C; USE OF STOGROUP MDWPSSG TO MD01S; shown are GRANT USE OF BUFFERPOOL statements > Our DB2 for z/OS system was using RACF to perform authorization checking. We decided to grant DBADM to the schema MDWPS. A RACF group called MDWPS was created, which allowed us to delegate schema (owner) privileges by connecting user IDs into the schema group. We did not connect the J2C authentication alias MDDBU to that group. There was no need when using currentSchema. Notice in Example 2-7 that each bus has a different schema with a name like MD01A. The example does not show the many GRANT USE OF BUFFERPOOL statements. Because some of the DDL that creates tables is also unqualified, it is important to issue SET CURRENT SQLID = ’<schema>’ before executing the DDL. It is necessary to issue GRANT USE OF BUFFERPOOL to the schema groups. When using currentSchema When using currentSchema, SQL statements execute under the identity of the J2C authentication alias user ID. That user ID has no implicit privileges over the tables under the schema, making it necessary to issue GRANT statements to allow it to SELECT, UPDATE, INSERT, and DELETE rows in the tables. The sibDDLGenerator.sh creates GRANT statements when it generates DDL for the SIBs, but it is necessary to construct GRANT commands based on the CREATE statements that were issued for the other WebSphere Process Server 90 z/OS: WebSphere Business Process Management V6.2 Production Topologies for z/OS components. We did this by copying the generated DDL into a series of GRANTxxx members in our WASCFG.MDCELL.WPS.SQL library and editing it to convert the CREATE statements into GRANT commands. Then we created sample JCL MDGRANT to execute the GRANT statements. See Appendix A, “Additional material” on page 615 for details on how to obtain these GRANT statements and the MDGRANT job. Because we did not know exactly what kinds of accesses were being performed by WebSphere Process Server for z/OS we issued GRANT ALL PRIVILEGES ON commands for each table. 2.4.5 Using the started task ID to connect to the database One option is to define all the WebSphere Process Server for z/OS tables with a schema equal to the user ID associated with the WebSphere servant Job. If you choose this option you do not need to code either currentSchema or currentSQLID. You do not need to specify any Authentication Alias on your type2 datasources either. Unlike specifying an Authentication Alias, this user ID has no password and the ID cannot be used by real users. It is not subject to the same security rules (for example, password expiration). Started task users may be allowed additional privileges and may be granted DBADM authority. This may be a a good reason for creating a database with this ID as well and granting the started task's ID DBADM authority for all additional schemas existing within the database. As these database tables are actually part of the WPS product, the WPS server is a logical owner of these tables. Database Scenario using no AuthenticationAlias Here is a sample configuration where we can use the user ID's associated with the WebSphere Process Server-started tasks as the owner of the schema table definitions. In this scenario, the deployment manager will be the owner of the tables defined on the cell scope datasources. These datasources will have no Authentication Alias definitions specified. These definitions are in a separate database as they are common to all cluster(s) and server(s) within this cell. The database name would be MDCELL as will be the owner of the tables. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 91 The table owner will be determined by the servant user ID. In this example, we are using the cell name MDCELL. Cell Name= MDCELL<= DMGR Servant UserID (zServantUserid=MDCELL) A second database will be defined for the cluster. This database MDSR1 will contain all the definitions that are shared across the cluster members as part of the DB2 data sharing group. This database will contain 6 separate schemas (MDSR1,MDC1S, MDC1A, MDC1B, MDC1C, MDC1F). A schema owned by the cluster and one for each of the message engines defined within the cluster. Again, these datasource will have no Authentication Alias and the default owner will the user ID for the servant tasks (MDSR1). Cluster1= MDSR1<= Servant and CRA userID(zServantUserid=MDSR1) In this configuration, there are tables owned by the cell (MDCELL) that need to be accessed by the cluster owner (MDSR1). These cell tables require ALIAS definitions be created for the owner. CREATE ALIAS MDSR1.CUSTPROPERTIES FOR MDCELL.CUSTPROPERTIES; To create these definitions you can run the SQL command in Example 2-8. It will generate the ALIAS definitions for each of the tables currently defined within the cell database. This not necessary for the SIB tables, because they are fully qualified by the messaging engine data store. Example 2-8 DB2 SELECT command to generate the CREATE ALIAS sql command SELECT 'CREATE ALIAS MDSR1.'||STRIP(CHAR(NAME))||' FOR MDCELL.'||STRIP(CHAR(NAME))||' ;' FROM SYSIBM.SYSTABLES WHERE DBNAME='MDCELL'; Note: Using this approach changes the user IDs assigned to the started tasks when the server is initially created. Currently all servants and control region adjuncts run with the MDASRU identity, including for the deployment manager. Reviewing the user ID assignment for DB2 schemas Figure 2-11 on page 93 is a diagram of the eventual network deployment cell with a single cluster depicting the different schema names. 92 z/OS: WebSphere Business Process Management V6.2 Production Topologies Single Cluster (MDCELL) VIPA Address SC42 SC53 Daemon Daemon MDDEMN MDDEMN MDNODEA MDNODEB NodeAgent MDAGNTA Schemas NodeAgent MDAGNTB AppServer DB: MDCELL MDCELL AppServer ME MDSR1A MDSR1AS Cluster MDSR1 MDSR1B MDSR1BS ME DB: MDSR1 Data Sharing Group: D9FG Location: DB9F DB2:D9FA MDSR1 MDC1S MDC1A MDC19 MDC1C MDC1F DB2:D9FB Figure 2-11 Eventual network deployment MDCELL single cluster showing schema names Figure 2-12 illustrates how we modified the spreadsheet to specify the different database names in Figure 2-11: Figure 2-12 Spreadsheet showing the two databases and separate schemas associated with the STC ID 2.4.6 Deciding how you will create and execute the DDL Defining a connection between WebSphere Process Server for z/OS and DB2 for z/OS involves definitions in both WebSphere Process Server for z/OS and in DB2. Because various names must match in these two places, WebSphere Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 93 Process Server for z/OS provides shell scripts and Integrated Solutions Console wizards that take various names as input and ensure that the definitions in WebSphere Process Server for z/OS match those in DB2 for z/OS. Although the shell scripts and wizards hide the details of all the definitions that are required, an implication is that a user ID is required that has sufficient privileges both to perform administration functions in WebSphere Process Server for z/OS and database administration functions in DB2 for z/OS. If such a user ID does not exist in your organization, it is necessary to avoid creating tables in DB2 for z/OS at the same time as defining the DB2 JDBC provider and data sources in WebSphere Process Server for z/OS. To avoid granting both WebSphere administration authority and DB2 SYSADM authority to the same user ID, you can ask your DB2 system administrator to create the databases and stogroups and issue GRANT statements to delegate authority over the databases, stogroups, and buffer pools to the user ID that is executing the DDL. There are two issues, to consider: Generation of the DDL Execution of the DDL WebSphere Process Server offers several alternative methods for generating and executing the DDL statements that create the necessary tables and indexes in DB2. One of the early decisions you must make is the method you will use to generate the DDL, and the method you will use to execute it. Generating the DDL The options for generating the required DDL are as follows: When you configure WebSphere Process Server for z/OS into a stand-alone server, all the required DDL is generated when you run the zWPSConfig.sh shell to augment the stand-alone server. When you configure WebSphere Process Server for z/OS into a network deployment cell, only the DDL for the common database is generated when you run the zWPSConfig.sh to augment the deployment manager. You must generate DDL for other components of WebSphere Process Server for z/OS using one of the other methods. DDL is generated for components like BPC and CEI when you run the wizards that configure these components. The sibDDLGenerator.sh generates DDL for a Service Integration Bus There is a createDB.sh that can both generate all the DDL and execute it. 94 z/OS: WebSphere Business Process Management V6.2 Production Topologies If you start by configuring WebSphere Process Server for z/OS into a stand-alone server, all the components of WebSphere Process Server for z/OS are resident in the same server and will use one database in DB2 by default. You can specify separate databases if you edit the response file before running zWPSConfig.sh If you want to configure BPC in a different cluster from the support and messaging components, you can configure a network deployment cell comprising a deployment manager and two empty nodes. You can generate the DDL using createDB.sh in one or multiple databases. Executing the DDL When you have generated the required DDL and edited it to match your requirements there are several ways to execute it: Create tables when executing the various wizards in the Integrated Solutions Console by checking the box Create Tables. We do not recommend creating tables when you run the wizards because you will likely want to inspect the DDL before it is executed. There are also likely to be security issues executing DDL in this way. Use DButility.sh See explanation for “Use createDB.sh.” Use createDB.sh The createDB.sh wraps the DButility.sh and is easier to use than DButility.sh. Therefore if you wish to execute the DDL from the UNIX environment rather than using DSNTEP2, we suggest that you use createDB.sh. Use SPUFI We do not recommend using SPUFI because of the risk of problems if the code page used by your terminal is not correct. Use DSNTEP2 See explanation for “2.7.7, “Executing the DDL with DSNTEP2 batch job” on page 132.” Use the DB2 command line processor (CLP) You can use the DB2 command line processor to issue sql statements from the UNIX System Services environment. See 2.7.8, “Using the DB2 command line processor” on page 133 for details. Use DB2 Connect™ Because the DDL is generated on z/OS, the use of DB2 Connect would require the DDL to be downloaded to a workstation first. For a DB2 for z/OS system it is easier to use DSNTEP2 or createDB.sh. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 95 The two most attractive options for executing DDL are to use createDB.sh and to use DSNTEP2. Note: You do not have to use createDB.sh to execute the DDL if you use it to generate the DDL. You may use createDB.sh to generate the DDL and use DSNTEP2 to execute it. At what stage in the configuration can you execute the DDL? The sample DDL templates are available after you run the zSMPInstall.sh script for the server configuration, so you can run createDB.sh just after that point. We found that it was much less effort to use createDB.sh to generate all the DDL in one instance. Also, when all the DDL already exists before you run the configuration wizards, there are a lot less errors in the logs to worry about. We strongly suggest using createDB.sh to generate the DDL, and execute it using either createDB.sh or DSNTEP2 before you start to configure SCA, CEI, and BPC. In this Redbooks publication we describe the use of createDB.sh to generate all the DDL just after augmenting the server. We also describe executing the DDL at that time. If you prefer not to use createDB.sh, there is a description on how to use the DDL generated by the various wizards that configure components such as SCA, CEI, or BPC in Chapter 2 of z/OS: WebSphere Business Process Management V6.1.2 Production Topologies, SG24-7703. 2.4.7 Creating a DB2JccConfiguration.properties file A DB2JccConfiguration.properties file may be required when using a type 2 data source to connect to DB2 for z/OS. If the default DB2 subsystem ID is not specified in the DSNHDECP load module on your current STEPLIB, then this file must contain at least the property db2.jcc.ssid, which specifies the subsystem ID (SSID) or group sharing name of the DB2 system to which your WebSphere Process Server for z/OS cell will connect. For example, we specified db2.jcc.ssid=D9FG, where D9FG is the group-sharing name of our DB2 for z/OS system. To understand how the DB2JccConfiguration.properties file is used and why we use a file with that long name, the following section presents background information about JDBC driver properties. 96 z/OS: WebSphere Business Process Management V6.2 Production Topologies DB2 properties file background When accessing DB2 for z/OS through the DB2 JDBC Universal driver there are several ways to specify the DB2 subsystem you wish to connect to. See DB2 Universal Database for z/OS Application Programming Guide and Reference for Java, SC18-7414, for a full discussion of the options. The options are as follows: Pass system property -Ddb2.jcc.ssid when invoking Java. For Java-stored procedures, such properties can be coded in file referred to by a //JVMPROPS DD card. Specify the location of a properties file using the JVM system property db2.jcc.propertiesFile. This is how WebSphere Application Server for z/OS does it. Add properties to a file called DB2JccConfiguration.properties and make that file available to the classpath. It is possible to include the directory that holds the DB2JccConfiguration.properties file on the CLASSPATH, or include the file in a jar file that is included on the CLASSPATH. Specify the DB2 subsystem name in the customized DSNHDECP module. Ensure that the customized SDSNEXIT that holds that module is above the SDSNLOAD in any JOBLIB or STEPLIB concatenation of any address space accessing DB2. When using the second or third options, specify the name of the DB2 subsystem to which you want to connect on the property db2.jcc.ssid, which you code in the properties file. The options listed above take precedence in the order listed. For example, if db2.jcc.propertiesFile is not set, DB2 searches the classpath for a file called DB2JccConfiguration.properties. If DB2 also fails to find a file of that name, DB2 tries to contact the DB2 whose SSID is coded in the DSNHEDCP module located in the customized SDSNEXIT library. Using -D JVM system properties It is not common to pass the JDBC properties individually as JVM system properties. Typically, only the -Ddb2.jcc.propertiesFile is passed to specify the location of a file that contains the JDBC properties. Using db2.jcc.propertiesFile In WebSphere Application Server for z/OS the db2.jcc.propertiesFile system property is normally the method used to specify the location of the DB2 Universal JDBC driver properties file. The zWPSConfig.sh will set this property when you augment nodes to support WebSphere Process Server. When you use the db2.jcc.propertiesFile to set the location of the JCC properties file, you can call the properties file anything you like. We decided to call this file DB2JccConfiguration.properties. so it can also be located through a CLASSPATH search. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 97 Using DB2JccConfiguration.properties When the db2.jcc.propertiesFile has not been set, DB2 will search CLASSPATH for a DB2JccConfiguration.properties file. When configuring WebSphere Process Server, batch jobs and shell scripts will normally obtain their JCC properties using this method. The group-sharing name of our DB2 was D9FG, so we created a directory called /etc/d9fg, and created a DB2JccConfiguration.properties file in that directory. We coded db2.jcc.ssid=D9FG in the DB2JccConfiguration.properties file. User IDs require read access to the file: Both the user ID running the configuration jobs and the user ID of the WebSphere Application Server for z/OS Control Region require read access to the configuration file specified above. Because /etc is not a shared mount point we created an /etc/d9fg directory on each LPAR that hosts a node in our cluster. Alternatively, you can create the DB2JccConfiguration.properties in a directory under a shared mount point so there is only one copy for the whole sysplex. We put it in the deployment manager’s mount point at /wasv61config/mdcell/mddmnode, so all servers in the cell could access it. 2.4.8 Customizing .profile and .setup files for users The default UNIX environment that is specified in /etc/profile might be using old versions of Java and DB2 for z/OS. To ensure that the UNIX command-line environment is customized for the correct version of Java and DB2 Universal JDBC Driver, it is necessary to customize a .profile for each user ID that will work with WebSphere Application Server and DB2 z/OS from the command line or from UNIX batch jobs. For WebSphere Application Server for z/OS, the administrator user ID and the controller user ID share a home directory. For example, the home directory of the administrator user ID of MDCELL was /var/WebSphere/home/MDCFG. We created a .profile in /var/WebSphere/home/MDCFG that set PATH, LIBPATH, STEPLIB, and CLASSPATH so the UNIX environment was correctly customized for Java and JDBC. (See Example 2-9 on page 99.) This allowed any shell script we invoked to make JDBC calls and connect to the correct DB2. JAVA_HOME is set to point to the same Java used by the deployment manager so that any scripts launched under user MDADMIN will run with the same level of Java as that used by WebSphere Application Server itself. 98 z/OS: WebSphere Business Process Management V6.2 Production Topologies Rather than concatenate the paths to their existing settings, we decided to make sure the CLASSPATH was how we wanted it by using unset CLASSPATH to clear the current CLASSPATH before setting it. Example 2-9 Part of a .profile showing JAVA_HOME and paths necessary for DB2 # Java setup # export JAVA_HOME=/wasv61config/mdcell/mddmnode/DeploymentManager/java PATH=$JAVA_HOME/bin/classic:$JAVA_HOME/bin:$PATH # # DB2 setup # export DB2_PATH=/usr/lpp/db2/d9fg/db2910_jdbc STEPLIB=DB9FU.SDSNLOAD STEPLIB=DB9FU.SDSNLOD2:$STEPLIB STEPLIB=DB9FU.SDSNEXIT:$STEPLIB PATH=$DB2_PATH/bin:$DB2_PATH/bin:$PATH LIBPATH=$DB2_PATH/lib:$DB2_PATH/lib:$LIBPATH unset CLASSPATH CLASSPATH=$DB2_PATH/classes/sqlj.zip CLASSPATH=$DB2_PATH/classes/db2jcc_javax.jar:$CLASSPATH CLASSPATH=$DB2_PATH/classes/db2jcc_license_cisuz.jar:$CLASSPATH CLASSPATH=$DB2_PATH/classes/db2jcc.jar:$CLASSPATH CLASSPATH=/etc/d9fg/DB2JccConfiguration.properties:$CLASSPATH export PATH STEPLIB LIBPATH CLASSPATH # echo 'JAVA_HOME='$JAVA_HOME echo 'DB2_PATH='$DB2_PATH echo 'PATH='$PATH echo 'STEPLIB='$STEPLIB echo 'CLASSPATH='$CLASSPATH echo 'LIBPATH='$LIBPATH In addition to setting a .profile for the WebSphere administrator user ID, it is a good idea for the WebSphere systems programmer to use the same .profile for their personal TSO user ID. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 99 Customizing .profile to use the DB2 command line processor Here are some additional lines you can add to your .profile if you want to use the DB2 command line processor: Example 2-10 Additional statements in .profile to use the DB2 command line processor: # Set up to use the DB2 command line processor: export JAVA5=/usr/lpp/java/J5.0/bin/java export JAVA5_HOME=/usr/lpp/java/J5.0 export JAVA5_64=/usr/lpp/java/J5.0_64/bin/java export JAVA5_64_HOME=/usr/lpp/java/J5.0_64 # # Setup to use the Java level I like. # export JAVA_HOME=$JAVA5_HOME export PATH=$JAVA_HOME/bin:$PATH export CLPHOME=/usr/lpp/db2/d9fg/db2910_base export CLASSPATH=$CLASSPATH:$CLPHOME/lib/clp.jar export CLPPROPERTIESFILE=$HOME/clp.properties . ./jcc/jcc3_env.sh Creating a .setup file for users To help with typing in telnet sessions, we also created some alias names in the .setup files in the same home directories as the .profile files. Example 2-11 shows a sample setup file; you may want to tailor it according to your preferences: Example 2-11 Sample setup file alias ll="ls -lat" alias dirs="ls -alLD" set -o vi alias lookup="find . -name " alias db2="java com.ibm.db2.clp.db2" "ll" allows you to list the files details, with the newest ones at the top. "set -o vi" helps you retrieve commands from the access history. Press the escape key once (esc) and release it and press k. The last command that you typed will appear on the window. Keys ‘j’ and ‘k’ help you scroll back and forth in the history buffer. “lookup” allows you to issue the find command more easily. 100 z/OS: WebSphere Business Process Management V6.2 Production Topologies “db2” allows you to issue DB2 command-line processor commands. See 2.7.8, “Using the DB2 command line processor” on page 133. 2.4.9 Verify Java and the DB2 Universal JDBC Driver After modifying the .profile, we ran a couple of simple tests to check that the command line environment for Java and the DB2 Universal JDBC driver were correct. These tests are not a full verification of JDBC function. Checking that the Java environment is correct To check that the Java environment is correct, start an OMVS session, and issue the commands in Example 2-12. Example 2-12 Testing the Java environment > su mdadmin FSUM5019 Enter the password for mdadmin: <enter the password> > java -fullversion java full version "J2RE 1.5.0 IBM z/OS build pmz31devifx-20080502 (SR7 + IZ15683 + IZ21286)" Verifying the DB2 JDBC Universal driver check To verify the DB2 JDBC Universal driver check, use the db2jcc -version command, as shown in Example 2-13. Example 2-13 Testing that db2jcc can be invoked > db2jcc -version IBM DB2 JDBC Universal Driver Architecture 3.6.67 Using the db2jcc command is not a comprehensive test of the JDBC driver. It only confirms that the UNIX command line environment is good enough to launch db2jcc. If you wish to verify that the DB2 Universal Driver has been configured correctly, refer to the Infocenter article Verify the installation of the IBM Data Server Driver for JDBC and SQLJ available at the following Web page: http://publib.boulder.ibm.com/infocenter/dzichelp/v2r2/topic/com.ibm.db 29.doc.java/db2z_jccverifyinstall.htm Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 101 2.5 Creating a standalone WebSphere application server Configuring a WebSphere Application Server for z/OS standalone server is described in IBM White Paper WP100999 - Using zPMT and Spreadsheet to Build a Quick Standalone Server. This also refers to Tecdoc PRS1331 which contains spreadsheets to help you customize the installation jobstreams. These papers mention the importance of using intermediate symbolic links and ICFCAT aliases. You should have already prepared the dataset alias names as described in “Dataset names and ICFCAT aliases” on page 61. Note: WebSphere Process Server for z/OS Version 6.2 requires the WebSphere Application Server for z/OS Version 6.1 Feature Pack for Web Services. This must be included at the time the server is initially configured. In 2.5.1, “Task overview” on page 102, we summarize the main tasks involved in configuring the standalone application server. The starting point of a WebSphere Process Server for z/OS configuration is when you have successfully started the standalone server, but not yet federated it into an ND cell. Important: Do not to federate any node into the cell until after the node has been augmented to support WebSphere Process Server for z/OS. 2.5.1 Task overview After performing the preparation steps (2.3, “Preparing the z/OS system components” on page 53) you can configure the standalone server. In summary, the steps to configure the WebSphere Application Server for z/OS are as follows: 1. Use the spreadsheet provided in the appendix of this Redbooks publication to create the response file as described in White Paper WP100999 in the Techdocs library. 2. Use the WCT/zPMT to create a standalone server profile with the Feature Pack for Web Services (FPWS) installed in it. 3. The zPMT tool will upload (FTP) the JCL and properties files to the target .CNTL and .DATA datasets used to configure the application server. 102 z/OS: WebSphere Business Process Management V6.2 Production Topologies 4. Follow the instructions generated by the zPMT and run the BBO* jobs to configure the security domain and the application server. 5. Start the application server to verify that it is correctly built before you augment it for WebSphere Process Server for z/OS. 6. Stop all STCs and back up the configuration ZFS. Note: It is highly recommended to post this information in a prominent place for all to reference. This will help minimize mistakes and confusion as we configure the production topology in the following tasks. As a reference, Figure 2-13 is a diagram with the naming conventions we will be using for our production topology cell. Daemon Daemon MDDEMN MDDEMN Deployment Manager MDDMGR DMgr Servant MDDMGRS Node Agent Node Agent MDAGNTA MDAGNTB Server Server MDSR1A MDSR1B Controller Controller Servant Servant MDSR1AS MDSR1BS Adjunct Adjunct MDSR1AA MDSR1BA MDNODEA MDNODEB Cluster MDSR1 MDCELL SYSA SYSB Figure 2-13 The names for all our STC address spaces in the MDCELL Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 103 Use this diagram as a reference for your implementation tasks. Post it in a prominent location. Figure 2-14 shows the port assignments. Figure 2-14 TCP/IP Port Assignments for the MDCELL Post this diagram in a prominent place for reference. 2.5.2 Using the spreadsheet to plan the server configuration We described the spreadsheet in 2.2.2, “Using a spreadsheet to plan names for configuration objects” on page 45. You can find more details in Techdoc WP10999 for building a standalone server. Locate the response file definitions for the standalone server on the bottom of the spreadsheet. Click the Stand-Alone Server tab, then click the column A heading, and copy it to your clipboard. Paste it into a text editor (such as Notepad) and save it in the directory where you keep your response files. 104 z/OS: WebSphere Business Process Management V6.2 Production Topologies 2.5.3 Using the zPMT to create the installation jobs zPMT Notes: We used the WCT version 7 of the zProfile Management Tool (zPMT) to configure our WebSphere cell. The Workstation Customization Tool (WCT) is a new tool that includes zPMT and is used to configure WebSphere Application Server for z/OS V7 cells. It can be used to configure WebSphere Application Server V6.1 cells too. You can download the WCT from the following Web page: http://www.ibm.com/support/docview.wss?rs=180&uid=swg24020368 Before using the WCT for the first time, we suggest you obtain the presentation in Techdoc PRS3357 WebSphere Application Server for z/OS V7.0: Introducing the WCT for z/OS, which can be downloaded from the following Web page: http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/PRS3357 If you plan to use the Version 6 Application Server Toolkit version of the zPMT, see IBM White Paper WP100871, Introducing the zPMT Configuration Tool for WebSphere z/OS, which is available for download at the following Web page: http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100871 At a high level: The creation of the customized jobs that build the node must be done using the zPMT (either from the ASTK or the WCT). The ISPF customization dialogs will not be able to incorporate the FPWS function. It involves one simple zPMT window over and above what is normally required to configure a standalone node or a deployment manager node. The same customized jobs are created by the zPMT as would be done if FPWS was not included. (However, some of the jobs do slightly more things.) The planning spreadsheet offered at ibm.com/support/techdocs under number PRS1331 can be used to plan nearly all of the variable input. At this time it still requires you to fill in the one FPWS window manually. See Techdoc WP101084 for details about these steps. 1. Start the WebSphere Customization Tool, and launch the zPMT. 2. Select the MDCELL in the top of the window, and click Create in the bottom of the window, as shown in Figure 2-15 on page 106. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 105 Note: Make sure you select WebSphere Application Server Version 6.1 when you create the profiles. Figure 2-15 WebSphere Customization Tool - Profile Management Tool - Profile Create 106 z/OS: WebSphere Business Process Management V6.2 Production Topologies You will be presented with the “Environment Selection” window shown in Figure 2-16. Figure 2-16 Profile Management Tool V7, Environment Selection window showing Application Server with Feature Pack for Web Services Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 107 3. Click Next and Browse for the location of your response file that you created from the spreadsheet in 2.5.2, “Using the spreadsheet to plan the server configuration” on page 104. We provide copies of our response files in the zPMT_ResponseFiles directory. See Appendix A, “Additional material” on page 615. You should generate your own. 4. Follow the instructions in Techdoc WP101084, in the section on “Creating a Deployment Manager node with FPWS included”. 5. As you step through the various windows, verify that the fields are accurately filled in as you specified them in the spreadsheet. Pay particular attention to the following fields: – Target dataset Prefix – Administrator’s user ID (defaults to WSADMIN. Change it to your administrator’s ID) – WebSphere Application Server for z/OS Product file system directory (defaults to the Intermediate symbolic link) – Feature Pack for Web Services Product file system directory, and Intermediate symbolic link. – JOB card template 6. Click Create to save the customized jobs and property files. Upload the customized jobs by clicking Process. You can also print the instructions from the zPMT before you sign out. 2.5.4 Running the configuration jobs to create the standalone server Once the customized jobs and property files are uploaded to the z/OS partitioned dataset, you can go to your TSO session and open the target .CNTL PDS, and find the instructions in the BBOSSINS member. These instructions will guide you through the tasks and jobstreams to submit which create the standalone server. Follow the instructions, with these alterations: BBOSBRAJ and BBOSBRAK These can be run as described by the instructions. An alternative is to use the RACF commands in Techdoc WP101427. BBOSBRAM Pay attention to where the home directories get created. On our systems this was in the /var/WebSphere/home/MDCFG/ directory. This is where you can put the .profile and .setup files. 108 z/OS: WebSphere Business Process Management V6.2 Production Topologies Note: The /var/ directory is really /SC42/var/ because it is mounted in a system-specific root. BBOCBRAJ and BBOCBRAK If you plan to grow this cell with multiple nodes and clusters, a good alternative to this is to use the scripts in Techdoc WP101427 “Using Generic RACF Profiles with WebSphere on z/OS V7“. BBOMSGC Skip this job unless you want messages translated to Japanese. BBOWCPY This copies the procedures to Proclib. If you are running these jobs to re-build a server, and have made changes to the procedures, you may want to skip this. BBOWCFS This creates the ZFS or HFS filesystem and mounts it. If you have to rebuild the server’s configuration, you can skip this job and start with the next one, but delete all the files and subdirectories in the node’s mount point before proceeding. BBOWHFSA If you are using alias names for the WebSphere BBO* load libraries, you must first run the job to create the aliases. See “Dataset names and ICFCAT aliases” on page 61. BBOWWPFA We had a problem described in APAR PK50623. The circumvention was to regenerate the customization jobs in the zPMT without defining a HTTP Server BBOWHFSB This job corrects all the file ownership and permission settings of directories and file in the configuration HFS. (You can re-run this job later to clean up rogue changes to the ownership and permission settings.) BBOWIVT We usually skip this job, but it is a way to test your new application server by running a J2EE application. It is a convenient way to drive an application without a browser connection because of firewall restrictions. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 109 2.5.5 Defining user IDs, group IDs, and EJBROLE profiles We discussed the EJBROLE profiles in 2.3.8, “Planning RACF user IDs, groups and EJBROLE profiles” on page 70, but you cannot run these jobs before the BBOSBRAK job that we ran to create the RACF profiles as part of the server installation above. Depending on your approach to use discreate EJBROLE profiles, or GEJBROLE profiles, you can run these jobs for which we have provided samples in the “Additional materials” accompanying this Redbooks publication. Remember, these are just samples, which you should tailor for your environment. MDRACF to define all the user IDs and groups we have planned for our cell, and... MDRACFG to define the RACF Group IDs for accessing the GEJBROLE profiles. MDEJBROL to implement the role assignments for EJBROLE profiles, and, GEJBROLE if you want to implement GEJBROLE profiles. We suggest you run these jobs before configuring WebSphere Process Server for z/OS which sill need these RACF roles, groups, and user IDs defined. 2.5.6 Starting the application server and verifying the configuration When you are in the process of configuring new servers, it is a good idea to look in the server’s log files for any error messages, and look in the Integrated Solutions Console for the settings of various components. This is quite easy if you know what you are looking for. Examine the server logs on JES spool Using SDSF Display Active window, select your application server’s started tasks with the PREfix or Select MD* commands and look in the control region (MDSR1A) and servant regions (MDSR1S). Select each with the S action character, and look for the following information: “Severe” error messages with the f severe all command. “Warning” messages are sometimes not significant. Use the f warning all command. For Java errors, look for the string exception: 110 z/OS: WebSphere Business Process Management V6.2 Production Topologies Hopefully, there will be none. If there are, note the number displayed by the find all command, and step through each one to start your problem determination process. See “Here are some detailed problems we encountered along with the solutions we used.” on page 594 for a list of common errors and their solutions. Explore with the Integrated Solutions Console Sign on to the Administration Console with a URL of this form: http://<host>:<port_number>/ibm/console For our standalone server, this was: http://wtsc42.itso.ibm.com:20046/ibm/console Because global security was enabled, we were re-directed to this URL: https://wtsc42.itso.ibm.com:20047/ibm/console Examine the following information: Servers: Just this standalone server. Check out the version information on the server’s page. Port numbers should all be in the correct range: 20000–20049 for our server as shown in Figure 2-17 on page 112. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 111 Figure 2-17 Port number allocations JDBC providers: Derby embedded non-XA JDBC Provider and the XA DB2 Universal JDBC Driver-compliant Provider. Data sources (only one: the DefaultDatasource using the Derby JDBC Provider) Applications: only the DefaultApplication and ivtApp. There are no SI Buses, nor Integration Applications. 2.5.7 Customizing the new server There are a few settings and post-configuration tasks that have to be made after the new server is created: Give the administrator’s user ID a password. Add DB2 STEPLIBS to the server procedures in PROCLIB. Back up the server’s configuration ZFS filesystem. 112 z/OS: WebSphere Business Process Management V6.2 Production Topologies Note: We do not recommend putting the DB2 libraries in the STEPLIB concatenation in the setupCmdLine.sh script. Instead, export them in the user’s .profile or export them manually before running any DB2 scripts. Give the administrator’s user ID a password The user ID for the administrator was created with no initial password. You can do that now with the following RACF command if you have the proper RACF authorization: ALTUSER xxADMIN PASSWORD(xxxxxx) NOEXPIRED If you do not have the proper RACF authorization, ask your security administrator to issue this command. Adding DB2 STEPLIBS to the server procedures in PROCLIB The STEPLIB dataset names for the WebSphere Application Server for z/OS libraries should have already been added in the servant region procedure Z-members if these datasets ware not in Linklist or LPA. If your DB2 libraries are not in Link List or LPA, add them to the same Z-members of Proclib for the servant started tasks as shown in Example 2-14. Example 2-14 Servant Procedure Z-parm members (a partial view) SYS1.PROCLIB(MDASRAZ) //*Steplib Setup //* //STEPLIB DD DISP=SHR,DSN=BBO6142.MDNODEA.SBBOLD2 // DD DISP=SHR,DSN=BBO6142.MDNODEA.SBBOLOAD // DD DISP=SHR,DSN=BBO6142.MDNODEA.SBBGLOAD // DD DISP=SHR,DSN=DB9F9.SDSNEXIT // DD DISP=SHR,DSN=DB9F9.SDSNLOAD // DD DISP=SHR,DSN=DB9F9.SDSNLOD2 2.5.8 Backing up the configuration ZFS You should make a practice of backing up the configuration HFS (or zFS) file systems after any milestone, and before you are about to do anything that might corrupt it, such as augmenting it for an additional product. Before running zSMPInstall.sh, back up the node’s configuration ZFS data set. We used the JCL shown in Example 2-15 on page 114 to perform the back up. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 113 Example 2-15 JCL to back up the standalone server’s node //MDDUMP JOB NOTIFY=&SYSUID,CLASS=A,MSGCLASS=A, // REGION=0M,TIME=1440 //* //* DUMP CONFIG HFS FOR NODE A //* //DUMP EXEC PGM=ADRDSSU //DASD DD DSN=WASCFG.MDCELL.DUMP.NODEA.PREINS, // DISP=(NEW,CATLG),UNIT=3390,SPACE=(CYL,(500,100),RLSE) //SYSPRINT DD SYSOUT=* //SYSIN DD * DUMP DATASET( INCLUDE( WASCFG.MDCELL.MDNODEA.CONFIG.ZFS ) ) OUTDD(DASD) TOL(ENQF) OPTIMIZE(4) COMPRESS /* Note: Remember that all the JCL examples in this chapter are provided in Appendix A, “Additional material” on page 615. 2.6 Adding WebSphere Process Server for z/OS When you have created a standalone server, you can install support for WebSphere Process Server for z/OS. Adding WebSphere Process Server for z/OS to the server’s node consists of the following steps: 1. Run the zSMPInstall,sh script See 2.6.1, “Running the zSMPInstall script” on page 114 2. Copy the standAloneProfileDB2.rsp file and tailor it for your configuration. 2.6.2, “Preparing the standalone profile response file” on page 117 3. Run the zWPSConfig.sh script 4. Create the DB2 databases and tables. (This is covered in 2.7, “Creating the DB2 databases and tables” on page 122). 2.6.1 Running the zSMPInstall script When invoked with the -install option, the zSMPInstall.sh script performs the following tasks: 114 z/OS: WebSphere Business Process Management V6.2 Production Topologies Validates the prerequisites. Creates the symbolic links from the WebSphere Process Server for z/OS installation file system to the server’s /lib and /bin directories in the configuration file system. Updates the code base permissions. Updates the Integrated Solutions Console with the WebSphere Process Server for z/OS product definitions, including definitions for the following components: – Business Process Choreographer (BPC) – WebSphere Process Server for z/OS core resources – Common Event Infrastructure (CEI) Check to ensure that the application server is stopped before running the zSMPInstall.sh script. You can invoke zSMPInstall.sh from a telnet command line, as shown in Example 2-16, or you can run it from a batch job, as shown in Example 2-17 on page 115 Example 2-16 Running zSMPInstall.sh from a telnet command line su b6admin (pwd = 'xxxxxxx') cd /wasv61config/mdcell/mdnodea/wps_smpe/zos.config/bin/ ./zSMPInstall.sh -smproot /wasv61config/mdcell/mdnodea/wps_smpe \ -runtime <server_root> -install Note: The back-slash (\) in Example 2-16 indicates that the command is continued on the next line. Do not type the back-slash if you are entering the command on one line We ran the script from a batch job, as shown in Example 2-17 Example 2-17 JCL to run the zSMPInstall.sh script //MD##INS JOB (0),CLASS=A,MSGCLASS=A,REGION=0M, // USER=MDADMIN,PASSWORD=MDADMIN //*JOBPARM SYSAFF=SC42 //***********************************************************/ //* Install WPS profiles into MDNODEA */ //***********************************************************/ //***********************************************************/ //* STEP TO CREATE LINKS TO WPS LIBRARIES */ //***********************************************************/ Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 115 //INSTO EXEC PGM=IKJEFT01,REGION=0M,TIME=1440 //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BPXBATCH SH + cd /wasv61config/mdcell/mdnodea/AppServer/bin; + export PATH=.:$PATH; + /wasv61config/mdcell/mdnodea/wps_smpe/zos.config/bin/+ zSMPInstall.sh + '-smproot /wasv61config/mdcell/mdnodea/wps_smpe' + '-runtime /wasv61config/mdcell/mdnodea/AppServer' + '-install' + 1> /tmp/mdNodeAinstall_0511.out + 2> /tmp/mdNodeAinstall_0511.err //***********************************************************/ //* STEPS TO COPY THE OUTPUT THE JOB LOG */ //***********************************************************/ //CPOUT EXEC PGM=IKJEFT01,REGION=0M //SYSEXEC DD DISP=SHR,DSN=BBO6142.MDNODEA.SBBOEXEC //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BBOHFSWR '/tmp/mdNodeAinstall_0511.out' BBOHFSWR '/tmp/mdNodeAinstall_0511.err' /* // Output from this script will look like the messages in Example 2-18 Example 2-18 zSMPInstall.sh output BBOHFSWR CWPIZ0253I: CWPIZ0254I: CWPIZ0255I: CWPIZ0256I: CWPIZ0257I: CWPIZ0259I: CWPIZ0260I: CWPIZ0262I: CWPIZ0263I: CWPIZ0264I: '/tmp/installonly_1032.out' parsing command arguments... parsing arguments complete setting up configuration... set up configuration complete creating the symbolic links... creation of symbolic links complete doing post install file updates... post install updates complete running Configuration Manager update... Configuration Manager update complete If any errors occur while running the script, start by looking in the <server_root>/logs/wbi/install/installconfig.log. When you find the error, if you issue FIND ‘buildfile’ PREVIOUS, it will tell you the ant file that failed. This will direct you to one of the following logs to troubleshoot the problem. It is helpful to 116 z/OS: WebSphere Business Process Management V6.2 Production Topologies sort the logs by date to look at the most recent log file first by using the ls -alt UNIX command. In our configuration, <server_root> was as shown in Example 2-19. Example 2-19 Log files in the <server_root>/logs directory of our configuration filesystem /wasv61config/mdcell/mdnodea/AppServer /<server_root>/logs/wbi/c2n.log /<server_root>/logs/wbi/zSMPInstall.log /<server_root>/logs/wbi/zSMPInstall.trace /<server_root>/logs/wbi/100SCleanOSGICache.ant.log /<server_root>/logs/wbi/15SCopyBSpaceIfNewer.ant.log /<server_root>/logs/wbi/80SCopyInstallValidatorLog.ant.log /<server_root>/logs/wbi/85SConfigNoProfileFirstStepsWBI.ant.log /<server_root>/logs/wbi/90SCleanDeployTool.ant.log /<server_root>/logs/wbi/90SConfigureWSProfileForWBI.ant.log /<server_root>/logs/wbi/90SConfigWBIMigrationScript.ant.log /<server_root>/logs/wbi/91SConfigNoProfileFirstStepsCharset.ant.log /<server_root>/logs/wbi/93SDeployBPCAdminConsolePlugins.ant.log /<server_root>/logs/wbi/93SDeployBusinessSpaceAdminConsoleFeature.log /<server_root>/logs/wbi/93SDeployServerAdminConsolePlugins.ant.log /<server_root>/logs/wbi/93SDeployWBICommonAdminConsolePlugins.ant.log /<server_root>/logs/wbi/94SDeployCoreAdminConsolePlugins.ant.log Back up the standalone server ZFS Back up the server’s configuration ZFS data set again before running zWPSConfig.sh. We used the JCL in Example 2-15 on page 114 but changed the //DASD DD card to //DASD DD DSN=WASCFG.MDCELL.DUMP.NODEA.PRECFG. 2.6.2 Preparing the standalone profile response file Parameters and their values are passed to the zWPSConfig.sh in a response file. A sample response file called standAloneProfileDB2.rsp is provided with WebSphere Process Server for z/OS for augmenting a standalone server when you will use DB2 for z/OS as a data store. 1. Copy the standAloneProfileDB2.rsp response file from /<wps_install_root>/zos.config/ to a work directory and rename it mdbaseaStandaloneDB2.rsp. In our case, <wps_install_root>was /usr/lpp/zWPSMD/V6R2. The work directory was /var/WebSphere/home/MDCFG/wpswork. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 117 Note: We were configuring more than one WebSphere Process Server for z/OS cell and so decided to change the name of the response file from standAloneProfileDB2.rsp to mdbaseaStandaloneDB2.rsp to avoid confusion. 2. Customize the properties in the response file, as shown in Example 2-20. Example 2-20 Response file mdbaseaStandaloneDB2.rsp DBPRODUCT=DB2UDBOS390_V8_1 ### (See notes above.) DBLOCATION=DB9F DBJDBCCLASSPATH=/usr/lpp/db2/d9fg/db2910_jdbc/classes DBUSER=MDDBU DBPASS=MDDBU DBHOSTNAME=wtsc42.itso.ibm.com DBSERVERPORT=37893 SQLID=MDWPS SQLDB=MDCELLDB SQLSTO=MDWPSSG augment profileName=default profilePath=/wasv61config/mdcell/mdnodea/AppServer/profiles/default templatePath=/wasv61config/mdcell/mdnodea/AppServer/profileTemplates/de fault.wbiserver cellName=mdbasea nodeName=mdnodea serverName=mdsr01a_WPS dbCreateNew=false dbDelayConfig=true enableAdminSecurity=true adminUserName=mdadmin ### (See note above.) adminPassword=mdadmin ### (See note above.) ceiDbName=$SQLDB ceiBufferPool4K=BP9 ceiBufferPool8K=BP8K2 ceiBufferPool16K=BP16K2 ceiEventCatalogDBName=$SQLDB configureBPC=true bpcDbName=BPEDB bpcDbStorageGroup=$SQLSTO 118 z/OS: WebSphere Business Process Management V6.2 Production Topologies bpcDbSqlId=$SQLID bpcDbProduct=$DBPRODUCT bpcDbConnectionLocation=$DBLOCATION bpcDbJDBCClasspath=$DBJDBCCLASSPATH bpcDbUser=$DBUSER bpcDbPassword=$DBPASS dbName=$SQLDB dbStorageGroup=$SQLSTO dbType=$DBPRODUCT dbConnectionLocation=$DBLOCATION dbJDBCClasspath=$DBJDBCCLASSPATH dbUserId=$DBUSER dbPassword=$DBPASS dbHostName=$DBHOSTNAME dbServerPort=$DBSERVERPORT dbSchemaName=$SQLID dbDriverType=2 dbCommonForME=true configureBSpace=true Make note of the following parameters: – DBPRODUCT=DB2UDBOS390_V8_1 even though we are using DB2 V9 There is no difference in the schema generated for the two versions of DB2, and some components do not support V9 yet. Therefore, we specified V8 in the WPS response files. – DBLOCATION, DBHOSTNAME, and DBSERVERPORT values can be displayed with the MVS DB2 DISPLAY,DDF command shown in Example 2-21: Example 2-21 Command to display the DDF information in DB2 subsystem D9F1 SDSF OPERLOG DATE 03/23/2009 6 WTORS COMMAND ISSUED COMMAND INPUT ===> /-D9F1 DIS DDF RESPONSE=SC42 DSNL080I -D9F1 DSNLTDDF DISPLAY DDF REPORT FOLLOWS: DSNL081I STATUS=STARTD DSNL082I LOCATION LUNAME GENERICLU DSNL083I DB9F USIBMSC.SCPD9F1 -NONE DSNL084I TCPPORT=37893 SECPORT=0 RESPORT=37894 IPNAME DSNL085I IPADDR=::9.12.4.52 DSNL086I SQL DOMAIN=wtsc42.itso.ibm.com DSNL086I RESYNC DOMAIN=wtsc42.itso.ibm.com Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 119 DSNL099I DSNLTDDF DISPLAY DDF REPORT COMPLETE – JMSUSER is no longer a parameter. The administrator’s user ID is used for the authentication aliases and users in the bus connector’s role. See “Correcting other known problems in V6.2” on page 140. This causes the administrator’s user password in clear in a file in the UNIX file system, which is a security exposure, so we altered it to use a different ID that did not have the authorization of the administrator. (We could change the adminUserName and adminUserPassword to a user ID like MDJMSU which would avoid this problem.) The spreadsheet described in 2.2.2, “Using a spreadsheet to plan names for configuration objects” on page 45 produces response files for augmenting nodes in the WPS_rsp tab. You can select the one on that sheet for your standalone server, and compare with the one you tailored from the zos.config directory (as a cross-check.) A copy of this and other response files that we used, can be found in the zos_wpswork directory. See Appendix A, “Additional material” on page 615. 2.6.3 Running the zWPSConfig.sh script After preparing the response file, we ran this script using a batch job. A copy of the MDBACFG job is in the additional materials, and is shown in Example 2-22. Example 2-22 JCL to run the zWPSConfig.sh script //MDBACFG JOB (0),CLASS=A,MSGCLASS=A,REGION=0M, // USER=MDADMIN,PASSWORD=MDADMIN /*JOBPARM SYSAFF=SC42 //***********************************************************/ //* Configure WPS in standalone server mdsr1 */ //***********************************************************/ //* Run zWPSConfig.sh */ //***********************************************************/ //INSTO EXEC PGM=IKJEFT01,REGION=0M,TIME=1440 //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BPXBATCH SH + export LIBPATH=/usr/lpp/db2/d9fg/db2910_jdbc/lib:$LIBPATH; + cd /wasv61config/mdcell/mdnodea/AppServer/bin; + ./zWPSConfig.sh + -response /var/WebSphere/home/MDCFG/wpswork/+ mdbaseaStandaloneDB2.rsp + 120 z/OS: WebSphere Business Process Management V6.2 Production Topologies -augment + 1> /tmp/mdnodeazWPSConfig_0B11.out + 2> /tmp/mdnodeazWPSConfig_0B11.err; /* //***********************************************************/ //* STEP TO COPY THE OUTPUT THE JOB LOG */ //***********************************************************/ //CPOUT EXEC PGM=IKJEFT01,REGION=0M //SYSEXEC DD DISP=SHR,DSN=BBO6142.MDNODEA.SBBOEXEC //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BBOHFSWR '/tmp/mdnodeazWPSConfig_0B11.out' BBOHFSWR '/tmp/mdnodeazWPSConfig_0B11.err' /* // You can also run this from a telnet session with the commands shown in Example 2-23: Example 2-23 Running the zWPSConfig.,sh script from the command line in a telnet client su mdamin password: xxxx export LIBPATH=/usr/lpp/db2/d9fg/db2910_jdbc/lib:$LIBPATH cd /wasv61config/mdcell/mdnodea/AppServer/bin ./zWPSConfig.sh -augment \ -response /var/WebSphere/home/MDCFG/wpswork/mdbaseaStandaloneDB2.rsp Back up the standalone server ZFS Back up the server’s configuration ZFS data set again after running zWPSConfig.sh. We used the JCL shown in Example 2-24 on page 124, but changed the //DASD DD card to //DASD DD DSN=WASCFG.MDCELL.DUMP.NODEA.POSTCFG. Note: Do not start the server until after the DB2 tables are created and verified. Doing so will create problems in the node’s configuration filesystem. 2.6.4 Configuring the SCA, CEI and BPC components When augmenting a standalone server for WebSphere Process Server for z/OS, these components were all configured by the zWPSConfig shell script. The BPC Explorer application was also installed. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 121 For empty nodes, these would be configured using the wizards described in z/OS: WebSphere Business Process Management V6.1.2 Production Topologies, SG24-7703. 2.6.5 Business Space powered by WebSphere This was partially configured by the zWPSConfig shell script. For a thorough description of this new component, see Chapter 9, “Creating an integrated user experience using Business Space powered by WebSphere” on page 547 2.7 Creating the DB2 databases and tables As soon as the standalone server has been augmented, we suggest you use the createDB.sh to generate the DDL. The createDB.sh script generates more complete, and better-formed DDL, and there are fewer files to manage. It is the quickest and best way to generate DDL. Note: You must run createDB.sh from a telnet client, because it uses the vi editor to make global changes to DDL files. You cannot use an OMVS session, or a batch job, because these do not support vi. Perform the following steps to configure DB2 z/OS using createDB.sh: 1. Copy createDB.sh to a work directory, and customize the sample createDB.sh script for your configuration. 2. Executing createDB.sh to generate the DDL. 3. If you specify "-All +DBSIB or *DBSIB" parameters on the createDB.sh script, you will only get the DDL for the SCA system and application messaging engines. The only way to get the DDL generated for the other buses (CEI, BPC, WSRR, Fabric) is to specify those databases as well. For example, if you specify "-All +DBSIB +DBCEI" you will get DDL for the two SCA buses and for the CEI bus. However, you will also get the DDL generated for the CEI database in the ceidb.sql file.. 4. We created the database and storage group for our cell using the batch job in Example 2-31.. 5. Execute the DDL using createDB.sh or execute the DDL with the DSNTEP2 batch job. 122 z/OS: WebSphere Business Process Management V6.2 Production Topologies 2.7.1 Using the sample createDB.sh script The createDB.sh is a sample script that you can use to create your WebSphere Process Server for z/OS database objects in a single database. The createDB.sh script uses sql template files provided with the product. 2.7.2 Customizing the sample createDB.sh script Set up a work directory to hold a copy of createDB.sh that you will customize. 1. Open a telnet session and log in as the WebSphere administrative user. 2. Create a wpswork directory under the home directory of the WebSphere administrative user ID. For example, for our MD cell we issued the following command: mkdir /var/WebSphere/home/MDCFG/wpswork 3. Copy the sample script createDB.sh from /<wps_install_root>/zos.config/samples to the work directory. In our case, <wps_install_root> is /usr/lpp/zWPS/V6M2. The work directory is /var/WebSphere/home/MDCFG/wpswork. 4. From the same directory, also copy the sibDropxx.sql file and the sibSchemaxx.sql file into the work directory. The next step is to modify the createDB.sh. 1. Although the permissions should be correct, we assigned the appropriate permissions (755) to the new version of createDB.sh. 2. Update the copy of createDB.sh script with the values appropriate for your environment as follows: a. The top section of the script contains a list of fields used during the execution of the script. Modify the default values of the fields to match your environment. Note: DBDelayConfig = true prevents the createDB.sh from executing the DDL. If you set DBDelayConfig = false, the properties DBCREATE, DBWPSCreate, ... DBSIBCreate control which DDL is executed. If you pass the +All option when invoking createDB.sh, it sets the DBxxCreate options to true. If your DB2 for z/OS system’s SDSNLOAD and SDSNLOD2 libraries are not in LINKLST, you will need to export STEPLIB as well as LIBPATH as shown in Example 2-24 on page 124. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 123 Example 2-24 Modified createDB.sh for our MD cell LIBPATH=/usr/lpp/db2/d9fg/db2910_jdbc/jcc/lib:$LIBPATH STEPLIB=DB9F9.SDSNEXIT:DB9F9.SDSNLOAD:DB9F9.SDSNLOD2:$STEPLIB export LIBPATH STEPLIB SCRIPTHOME=`pwd` export SCRIPTHOME SKIP=0 INVALIDoption=0; DBCREATE=true DBUSESQLID=true DBWPSCreate=true DBBPCCreate=true DBBSCreate=true DBCEICreate=true DBSIBCreate=true DBBSFCreate=true DBWSRRCreate=false DBGenerate=true DBPREFIX=MD DBSCOPE=01 DBJDBCClasspath=/usr/lpp/db2/d9fg/db2910_jdbc/classes DBUSER=MDDBU DBPASSWORD=MDDBU DBJDBCProperties=/etc/d9fg DBConnectionLocation=DB9F DBDelayConfig=true DBHostName=wtsc42.itso.ibm.com DBServerPort=37893 DBVOLUMES="'*'" DBVCAT=WASCFG CEIBP4K=BP9 CEIBP8K=BP8K2 CEIBP16K=BP16K2 CEISIZE=10 SCRIPTTEMP=$SCRIPTHOME/tmp OUTPUT=$SCRIPTTEMP/output.out ERROR=$SCRIPTTEMP/error.out 124 z/OS: WebSphere Business Process Management V6.2 Production Topologies b. An additional modification we made was to change the storage group, schema, and database name that were being generated and used by the script so they matched the names we wanted to use. The variables to change are further down the createDB.sh (around line number 72) and are shown in Example 2-25. Example 2-25 Setting storage group, schema and database name in createDB.sh 73 ### Modified Schema Prefix to MDWPS: 74 DBSCHEMA=${DBPREFIX}WPS 75 ### Modified StoGroup name to MDWPSSG: 76 DBSTO=${DBPREFIX}WPSSG 77 ### Modified database name to MDWPSDB: 78 DBNAME=${DBPREFIX}WPSDB further down, @ line 221, we modified the script to use our value for DBSTO: ### Following 5 lines added to use value for DBSTO ### specified on line 72 above.### 221 if [ "$DBSCHEMA" = "" ] 222 then 223 DBSCHEMA=${DBPREFIX}DBSG 224 fi 225 echo " DBSCHEMA $DBSCHEMA echo " Note: There is only one variable DBNAME in createDB.sh. This means that if you run createDB.sh with the +All option, all the tables are created in one database. If you want to define the different components of WebSphere Process Server for z/OS in different databases, you can run createDB.sh multiple times, specifying a different DBNAME variable for each component of WebSphere Process Server for z/OS. Alternatively, you can use the +All option and manually edit the DDL later as described in “Generating the DDL for the SIB messaging engines” on page 127. We found it was easier to execute createDB.sh with the +All option and edit the database names in the DDL afterwards. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 125 c. Even though we are using DB2 version 9, the schema for V8 and V9 should be identical. We did not change the following two hard coded variables in the createDB.sh script. • • DB2zOSV8 DB2UDBOS390_V8_1 2.7.3 Executing createDB.sh to generate the DDL After setting up the work directory and modifying the script, you can run the createDB.sh script to generate the DDL. 1. Set up the correct UNIX environment by running the setupCmdLine.sh from the <profile_root>/bin/ directory as shown in Example 2-26. Example 2-26 Running the setupCmdLine.sh script cd /wasv61config/mdcell/mdnodea/AppServer/profiles/default/bin . ./setupCmdLine.sh Notice that the command . ./setupCmdLine.sh is preceded by a period, a space, and a period, and a slash (“. ./“). Running the command in this fashion retains the export of WAS_HOME that is performed by setupCmdLine.sh and makes WAS_HOME available to the createDB.sh script. It also does not depend on your $PATH settings. Verify the variables that were set by running the setupCmdLine.sh script (and the user ID that you are running with as in Example 2-27: Example 2-27 Verifying the variables set by setupCmdLine.sh >id uid=209(MDADMIN) gid=16(MDCFG) >echo $WAS_HOME /wasv61config/mdcell/mdnodea/AppServer >echo $STEPLIB none:BBO6142.MDNODEA.SBBOLOAD:BBO6142.MDNODEA.SBBOLD2: BBO6142.MDNODEA.SBBOLPA: 126 z/OS: WebSphere Business Process Management V6.2 Production Topologies 2. Run the modified createDB.sh script to generate the DDL. See Example 2-28. Notice that we pipe the STDOUT and STDERR to files in our filesystem for possible debugging. Example 2-28 Executing createDB.sh cd /var/WebSphere/home/MDCFG/wpswork ./createDB.sh +All > x.crDBout 2> x.crDBerr When invoking createDB.sh to generate DDL for one component, you need to pass the -All option (minus -All instead of plus +All) to suppress creation of DDL for everything. Specify an option like +DBBPC to generate DDL for just the BPC component as shown in Example 2-29. Example 2-29 Executing createDB.sh for one component cd /var/WebSphere/home/MDCFG/wpswork ./createDB.sh -All +DBBPC 3. Confirm that the DDL was successfully created. When we ran createDB.sh to generate DDL for DB2 for z/OS V9, the files shown in Example 2-30 were generated in the following directory: <server_root>/profiles/default/dbscripts/CommonDB/DB2zOSV8/MDWPSDB. Example 2-30 DDL files generated by createDB.sh bpc.sql ceidb.sql common.sql createDatabaseX.sql sibAPP.sql sibBPC.sql sibCEI.sql sibSCA.sql Generating the DDL for the SIB messaging engines If you specify "-All +DBSIB or *DBSIB" parameters on the createDB.sh script, you will only get the DDL for the SCA system and application messaging engines. The only way to get the DDL generated for the other buses (CEI, BPC, WSRR, Fabric) is to specify those databases as well. For example, if you specify "-All +DBSIB +DBCEI" you will get DDL for the two SCA buses and for the CEI bus. However, you will also get the DDL generated for the CEI database in the ceidb.sql file. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 127 2.7.4 Modifying the generated DDL After the DDL is generated using the createDB.sh script, some manual changes are still needed. Note: The DDL generated by createDB.sh is in ASCII. If you intend to execute the DDL using createDB.sh, it must remain in ASCII. If you have z/OS V1.9, you can use ISPF option 3.17 to edit an ASCII file. Alternatively, use an FTP client to download the DDL in binary to a workstation where it can be edited and uploaded again in binary. If you intend to execute the DDL using DSNTEP2, it is probably better to convert the DDL to EBCDIC, copy it into z/OS sequential files, and edit it using the normal ISPF editor. This is described in 2.7.7, “Executing the DDL with DSNTEP2 batch job” on page 132. Be sure to pre-allocate the z/OS sequential files with sufficient space. When using oget to copy files from UNIX to z/OS you are not warned if the files runs out of space. Some records may not be copied. The only changes we made to the generated DDL statements were to the bufferpools for the bufferpools in the common.sql file and sibxxx.sql files. We also changed the PRIQTY and SECQTY values in the sibxxx.sql files. Note: Setting PRIQTY and SECQTY to -1 allows DB2 for z/OS to manage space allocation of table spaces and indexes depending on the volume of data they contain. This means you do not have to worry about space allocations at the time you create table spaces and indexes. Separate databases If you want to use four separate databases, here is a summary of the changes you can make to the files listed in Example 2-30 on page 127: bpc.sql If you generate the DDL using createDB.sh with the +All option, make a global change of the database name from the common database name to the database name you want to use for BPC. For our MD cell, for example, we changed MDWPSDB to MDBPCDB. (The ISPF editor exclude command (x) is useful for making selective changes like this.) 128 z/OS: WebSphere Business Process Management V6.2 Production Topologies ceidb.sql The buffer pools for ceidb.sql were properly passed from createDB.sh, so the only change to be made is the database name from MDWPSDB to MDEVTDB. common.sql Only the buffer pools need to be changed for common.sql. A global find and replace was done to change BP0 to BP1 and BP16K0 to BP16K1. createDatabaseX.sql We made no changes to this file because we did not use it. We created some JCL called MDCREDB, which created the databases and issued appropriate GRANTs to delegate authority over the databases. For all 4 sibxxx.sql files, sibAPP.sql, sibBPC.sql, sibCEI.sql, and sibSYS.sql, we changed all the PRIQTY and SECQTY entries to be -1. This must be done carefully because in some of the DDL the PRIQTY or SECQTY property are not on the same line as their values. For all 4 sibxxx.sql files, sibAPP.sql, sibBPC.sql, sibCEI.sql and sibSYS.sql, we set the correct buffer pools and change the database name from MDWPSDB to MDSIBDB. 2.7.5 Creating the database and storage group Instead of running createDB.sh to create one database using the DDL in file createDatabaseX.sql, we created a sample job MDCR1DB that created the single database and granted the needed permissions. See Appendix A, “Additional material” on page 615 for information about how to obtain this sample JCL. We recommend that you pay particular attention to the MDCR1DB job because it consolidates a lot of our experiences related to properly setting up security. The GRANT statements in MDCR1DB are discussed in 2.4.4, “Using GRANT statements in DB2” on page 89. Ask your DB2 database administrator to execute MDCR1DB because SYSADM authority is needed. We created the database and storage group for our cell using the batch job in Example 2-31 on page 130. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 129 Example 2-31 MDCR1DB job to define the MD cell’s database and storage group. //MDCR1DB JOB (0),'MDADMIN',CLASS=A,REGION=0M,NOTIFY=&SYSUID /*JOBPARM S=SC42 //***********************************************************/ //* Run this job using a userid with DB2 SYSADM authority */ //* or grant DBADM to the <sqlid> on SET CURRENT SQLID. */ //* Set the SYSAFF to the LPAR for this node. */ //* Change MD to your two character cell prefix. */ //* The DB2 system administrator should review all the */ //* names and bufferpools used. */ //* Change all the dataset names and plan name to match */ //* those in use with your DB2 system. */ //* Set the SYSIN to the dataset holding the DDL. */ //***********************************************************/ //JOBLIB DD DISP=SHR,DSN=DB9F9.SDSNEXIT // DD DISP=SHR,DSN=DB9F9.SDSNLOAD //* //* Create WPS databases and stogroups. //* //CREATE EXEC PGM=IKJEFT01,DYNAMNBR=20 //SYSTSPRT DD SYSOUT=* //SYSUDUMP DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSTSIN DD * DSN SYSTEM(D9FG) RUN PROGRAM(DSNTEP2) PLAN(DSNTEP91) LIB('DB9FU.RUNLIB.LOAD') //SYSIN DD * CREATE STOGROUP MDWPSSG VOLUMES ('TSTO53') VCAT WASCFG; CREATE DATABASE MDWPSDB STOGROUP MDWPSSG BUFFERPOOL BP1 INDEXBP BP2; /* 2.7.6 Executing the DDL with the createDB script This is the recommended tool for executing the DDL statements that create the table spaces and tables for the various components in WebSphere Process Server for z/OS. 130 z/OS: WebSphere Business Process Management V6.2 Production Topologies Note: If you prefer to execute the DDL using DSNTEP2 see 2.7.7, “Executing the DDL with DSNTEP2 batch job” on page 132. After customizing the DDL and creating the database, we ran the createDB.sh once with the +All option as shown in Example 2-32. Example 2-32 createDB.sh script ./createDB.sh +All -gen -DBDelayConfig false Configuring WPS Definitions [sql] 88 of 88 SQL statements executed successfully Configuring BPC Definitions [sql] 2033 of 2033 SQL statements executed successfully Configuring CEI Tables [sql] 375 of 375 SQL statements executed successfully Configuring SIB Definitions SCA Bus Schema MD01S [sql] 40 of 40 SQL statements executed successfully APP Bus Schema MD01A [sql] 40 of 40 SQL statements executed successfully BPC Bus Schema MD01B [sql] 40 of 40 SQL statements executed successfully CEI Bus Schema MD01C [sql] 40 of 40 SQL statements executed successfully Note: Remember to change to the <server_root>/bin directory and execute the command . setupCmdLine.sh before running createDB.sh. Separate databases If you want to create tables in four separate databases, you can run the createDB.sh four times (as shown in Example 2-33 on page 132), once for each major component, so we can more easily isolate any errors. In between each execution, we modified the database name at line number 79 in the createDB.sh script. The majority of the output messages and any error messages are written to files output.out and error.out in the /tmp directory that is created under the /wpswork directory from which you executed createDB.sh. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 131 Example 2-33 createDB.sh commands with results summary ./createDB.sh -All +DBWPS -gen -DBDelayConfig false Configuring WPS Definitions [sql] 88 of 88 SQL statements executed successfully <edit line 70 of createDB.sh set DBNAME= the BPC database name> ./createDB.sh -All +DBBPC -gen -DBDelayConfig false Configuring BPC Definitions [sql] 2033 of 2033 SQL statements executed successfully <edit line 70 of createDB.sh set DBNAME= the CEI database name> ./createDB.sh -All +DBCEI -gen -DBDelayConfig false Configuring CEI Tables [sql] 375 of 375 SQL statements executed successfully <edit line 70 of createDB.sh set DBNAME= the SIB database name> ./createDB.sh -All +DBSIB -gen -DBDelayConfig false Configuring SIB Definitions SCA Bus Schema MD01S [sql] 40 of 40 SQL statements executed successfully APP Bus Schema MD01A [sql] 40 of 40 SQL statements executed successfully BPC Bus Schema MD01B [sql] 40 of 40 SQL statements executed successfully CEI Bus Schema MD01C [sql] 40 of 40 SQL statements executed successfully 2.7.7 Executing the DDL with DSNTEP2 batch job Note: Skip this step if you executed the DDL in 2.7.6, “Executing the DDL with the createDB script” on page 130. If you prefer to execute the DDL using the DB2 DSNTEP2 program in a batch job, you must first copy it from the UNIX file system into z/OS sequential files or as members of a partitioned dataset. Because the DDL is in ASCII, it must also be converted to EBCDIC before it can be executed by DSNTEP2. We created sample job MDCPYDDL to do this. The job converts the DDL from ASCII to EBCDIC and copies it into z/OS sequential files where you can edit it with the ISPF editor. Pre-allocate these with SPACE(TRK(1,2)) to avoid the risk that some records are not copied. 132 z/OS: WebSphere Business Process Management V6.2 Production Topologies The sequential files are allocated as VB. After editing the DDL, copy it into a partitioned dataset with an LRECL of 80 so it can be executed using DSNTEP2. We used a FB LRECL 80 PDS called WASCFG.MDCELL.WPS.SQL. We created sample job MDDDL to execute the DDL using DSNTEP2. Both the MDCPYDDL and MDDDL jobs are provided in the additional material in Appendix A, “Additional material” on page 615. 2.7.8 Using the DB2 command line processor The command line processor can be used to run sql statements in the UNIX Systems Services environment from the command line prompt. For more information see the following topic and sub-topics in the DB2 V.9 Information Center at the following Web page: http://publib.boulder.ibm.com/infocenter/dzichelp/v2r2/topic/com.ibm.db 29.doc.comref/db2z_commanddescriptions.htm Setting up to use the command line processor Several environmental variables and setting need to be setup to use the command line processor (CLP): Export the following variables. This can be done in the user’s .profile as described in 2.4.8, “Customizing .profile and .setup files for users” on page 98. – – – – – JAVA_HOME PATH CLASSPATH CLPHOME CLPPROPERTIESFILE Create an alias for db2 in the .setup file to invoke the CLP. Create a clp.properties file to specify the semi-colon (;) as the termination character for sql statements. Example 2-34 is a sample: Example 2-34 Sample clp.properties file. #Specify the value as ON/OFF or leave them blank DisplaySQLCA=ON AutoCommit=ON InputFilename= OutputFilename= DisplayOutput= StopOnError= TerminationChar=; Echo=ON Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 133 StripHeaders= MaxLinesFromSelect= MaxColumnWidth=20 IsolationLevel= #Create your own alias name for DB2 servers. Here we connect to DBP0 #SERVER1=<URL>,<username>,<password> DBP0=wsc1.washington.ibm.com:446/WSCDBP0,SYSADM1,SYSADM1 Create a shell script to set environment variables so that the operating system can locate the IBM Data Server Driver for JDBC and SQLJ. a. Set the location of the sdk home, jcc home, DB2 OS data set highlevel qualifier, the current directory. b. Add the jcc related jar files to the CLASSPATH. c. Add jars from JCC_HOME/classes for jcc. d. Add the current directory to pick up classes anchored here. e. Add the jcc related directories to the PATH and LIBPATH. f. Add the Java directory to the PATH if needed. Example 2-35 is a sample shell script to set these variables: Example 2-35 Sample jcc3_env.sh script. #!/bin/sh ## set the location of the sdk home, jcc home, DB2 OS data set # highlevel qualifier, and the current directory # JAVA5_HOME=/usr/lpp/java/J5.0 JCC_HOME=/usr/lpp/db2910_jdbc DB2_HLQ=DSN910 BASE_DIR=`pwd` # # Add the jcc related jar files to the CLASSPATH JCC_CLASSPATH= # # Add jars from JCC_HOME/classes for jcc JCC_CLASSPATH=$JCC_HOME/classes/db2jcc.jar JCC_CLASSPATH=$JCC_HOME/classes/sqlj.zip:$JCC_CLASSPATH JCC_CLASSPATH=$JCC_HOME/classes/db2jcc_javax.jar:$JCC_CLASSPATH JCC_CLASSPATH=$JCC_HOME/classes/db2jcc_license_cisuz.jar:$JCC_CLASSP ATH # # Add the current directory to pick up classes anchored here. export CLASSPATH="$BASE_DIR":"$JCC_CLASSPATH":"$CLASSPATH" 134 z/OS: WebSphere Business Process Management V6.2 Production Topologies # # Add the jcc related directories to the PATH and LIBPATH export PATH="${JCC_HOME}"/bin:$PATH export LIBPATH="${JCC_HOME}"/lib:$LIBPATH # # Add the java directory to the PATH if needed whence java | grep -q -e "J5.0" -e "J6.0" rc=$? if [ rc -eq 1 ]; then export PATH="${JAVA5_HOME}"/bin:$PATH fi # # If SDSNLOD2, SDSNLOAD and SDSNEXIT are not in the linklist, # fixup the following statements and uncomment the export statement. SL= SL=${DB2_HLQ}.JUDY.SDSNEXIT SL=$SL:${DB2_HLQ}.SDSNLOD2 SL=$SL:${DB2_HLQ}.SDSNLOAD export STEPLIB=$SL:$STEPLIB Run this jcc3_env.sh shell script with the . (dot) command to set the environment variables for this session. You must add the following statement to the front of your sql files to connect to the correct DB2 subsystem (or group-attach name): CONNECT TO DBP0; Running the CLP Once you update your .profile, .setup, create the properties file and shell script, you can now run the DB2 CLP pointing to the updated sql file as shown here: db2 -f <profile_root>/dbscripts/CommonDB/DB2zOSV8/xxx.sql 2.7.9 Granting table privileges to the J2C auth alias user ID As described in 2.4.4, “Using GRANT statements in DB2” on page 89, if you decided to use currentSQLID on the data sources, and granted sufficient authority for the servant region to switch to that user ID, the J2C authentication alias user ID will have implicit privileges over the tables. You do not need to issue GRANTS for table privileges. If you are using currentSchema, however, issue GRANT commands that give the J2C authentication alias user ID privileges over the tables. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 135 We created a series of members called GRANTxxx in our WPS.SQL dataset by copying the supplied DDL and editing all the CREATE TABLE statements so they became GRANT ALL ON commands. We then created sample job MDGRANT to execute all the GRANT statements using DSNTEP2. You can obtain the GRANT statements we used and the sample JCL MDGRANT from Appendix A, “Additional material” on page 615. 2.7.10 Verifying the DB2 table creation If you encounter any problems with the DB configuration, here are some helpful resources: DB2 Information Center WebSphere Process Server for z/OS Information Center 10.3.7, “DB2 tools” on page 583 10.4.6, “DB2 problems” on page 603. Note: It is important to make certain that the DB2 tables are all created and validated before you start up the server... Use ADBL - the DB2 administrator’s tool (or SPUFI) to verify that all the tables are created. From the primary window, select (1) for DB2 Catalog, then (D) for databases. Specify a filter of MD* to see your database, as shown in Example 2-36. Example 2-36 D9F1 Databases DB2 Admin ------------------- D9F1 Databases ---------- Row 20 from 52 Command ===> Scroll ===> CSR Commands: GRANT MIG DIS STA STO UTIL Line commands: T - Tables S - Table spaces X - Indexes G - Storage group ICS - status DIS - Display database STA - Start database STO - Stop database A - Auth ? - Show all line commands Storage Buffer Created Index Select Name Owner Group Pool DBID By T E BPool MD* * * * * * * * * ------ -------- -------- -------- -------- ------ -------- - - ------MDWPSDB SENIOKJ M1WPSSG BP1 275 SENIOKJ E BP2 ******************************* END OF DB2 DATA ********************* 136 z/OS: WebSphere Business Process Management V6.2 Production Topologies Display the tables in each of the databases by typing T in the Select column and pressing the enter key (Example 2-37). Example 2-37 DB2 Table display DB2 Admin ------ D9F1 Tables, Views, ---- Row 1 to 25 of 412 Command ===> Commands: GRANT MIG Line commands: C - Columns A - Auth L - List X - Indexes S - Table spaces V - Views T - Tables P - Plans Y - Synonyms SEL - Select ? - Show all line commands Sel Name Schema * * ----- -----------------> -------STAFF_QUERY_INSTAN MDWPS STAFF_LOCK_T MDWPS PROCESS_TEMPLATE_B MDWPS ACTIVITY_TEMPLATE_ MDWPS TASK_TEMPLATE_T MDWPS SCHEMA_VERSION MDWPS SCOPE_TEMPLATE_B_T MDWPS SERVICE_TEMPLATE_B MDWPS TASK_MESSAGE_INSTA MDWPS T DB Name * * - -------T MDWPSDB T MDWPSDB T MDWPSDB T MDWPSDB T MDWPSDB T MDWPSDB T MDWPSDB T MDWPSDB T MDWPSDB TS Name Cols * * -------- -----STAF801 16 STAFF02 3 TEMP802 40 TEMP808 49 TEMP836 45 TEMPL01 2 TEMPL03 12 TEMPL04 9 INSTA44 If you use one database, you should have 403 tables in your database. Separate databases If you use separate databases for WPS, BPC, CEI, and the SIBs as shown in Table 2-4 on page 82, you should see something like this for the number of tables which we created for our M1CELL: (This will vary depending on the level of WebSphere Process Server for z/OS and WebSphere Business Services Fabric for z/OS that you are using.) M1WPSDB: 53 Tables M1BPCDB: 238 Tables, 235 table spaces M1CEIDB: 38 Tables, 36 table spaces (table names all start with cei) M1SIBDB: 48 Tables, 48 table spaces for 4 MEs (table names all start with SIB) M1BSPDB: 26 Tables, 26 table spaces (table space names all start with BS) Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 137 2.8 Starting the standalone application server Using the MVS console, or SDSF TSO windows, issue the start command for the node agent. On our system, the command was as follows: START MDACRA,JOBNAME=MDSR1A,ENV=MDBASEA.MDNODEA.MDSR1A Make sure the four address spaces start up. Example 2-38 shows what they look like in the SDSF DA window for our standalone application server MDSR1A: Example 2-38 SDSF DA window for a standalone server SDSF DA SC42 (ALL) COMMAND INPUT ===> PREFIX=MD* DEST=(ALL) NP JOBNAME JobID MDDEMNA STC25300 MDSR1A STC25296 MDSR1AA STC25304 MDSR1AS STC25301 PAG 0 CPU/L/Z 4/ 4/ 0 LINE 1-4 SC OWNER=* SORT=SysName/A JOBNAME/A SYSNA SysN CPU% Real StepName ProcStep Owner SC42 0.00 8801 MDDEMNA BBODAEMN MDACRU SC42 0.03 65T MDSR1A BBOCTL MDACRU SC42 0.12 105T MDSR1AA BBOSR MDASRU SC42 0.12 146T MDSR1AS BBOSR MDASRU 2.8.1 Review any error messages and repair When the server finishes initialization, look in the server’s log files for any error messages, and examine the Integrated Solutions Console for the settings of various components. This is quite easy if you know what you are looking for. Examine the server logs on JES spool Using SDSF Display Active window, select your application server’s started tasks with the PREfix or Select MD* commands and look in the control region (MDSR1A), control region adjunct (MDSR1AA), and servant regions (MDSR1S). Select each with the S action character, and look for any sever error messages with the FIND SEVERE ALL command. Hopefully, there will be none. If there are, note the number displayed by SDSF, and step through each one and start your problem determination process. Note: There are one or two severe messages relating to TCP/IP channels at startup. These are usually safe to ignore once all the address spaces have fully initialized. CHFW0030E: Error starting chain _InboundTCPProxyBridgeService 138 z/OS: WebSphere Business Process Management V6.2 Production Topologies Explore with the Integrated Solutions Console Sign on to the Administration Console and note the new content that has been added by the zWPSConfig augment scripts (applications, buses, JDBC dataSources and drivers) http://<host>:<port_number>/ibm/console Are the applications successfully installed? Look under Applications → Enterprise Applications: AppScheduler BPCECollector_mdnodea_mdsr01a_WPS BPCExplorer_mdnodea_mdsr01a_WPS BPEContainer_mdnodea_mdsr01a_WPS DefaultApplication ESBSamplesGallery HTM_PredefinedTaskMsg_V620_mdnodea_mdsr01a_WPS HTM_PredefinedTasks_V620_mdnodea_mdsr01a_WPS TaskContainer_mdnodea_mdsr01a_WPS WBISamplesGallery ivtApp persistentLkMgr query sca.sib.mediation wpsFEMgr_6.2.0 Have new buses been defined? Look under Service integration → Buses: MDSR1.000-BPC.mdbasea.Bus MDSR1.000-CommonEventInfrastructure_Bus MDSR1.000-SCA.APPLICATION.mdbasea.Bus MDSR1.000-SCA.SYSTEM.mdbasea.Bus Are the messaging engines started? Click Application Servers → server_name → Messaging Engines under Server Messaging. Do all the messaging engines show a green arrow under the status column? Verify the ME data store settings Look under each Messaging engine → Message store. Schema names in the data stores should be MD01A, MD01B, MD01C, and MD01S. The Create Tables box should not be checked. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 139 Other errors If you encounter any errors, you may be able to fix them using the Integrated Solutions Console, depending on the particular error message. Here are some helpful resources: WebSphere Process Server for z/OS Information Center 10.4.6, “DB2 problems” on page 603 in this Redbooks publiction Google Correcting other known problems in V6.2 There are a couple of post-configuration tasks we are aware of that should be corrected at this point. Messaging engines connector roles The WebSphere Process Server V6.2 response files no longer have variables for the JMSUSER alias. It uses the administrator’s user ID for the bus connector role and some of the authentication aliases, as shown in Figure 2-18. Figure 2-18 Administrator’s user ID as the bus connector role This puts the administrator’s user password in clear text in a file in the UNIX file system, which is a security exposure, so we altered it to use a different ID that did not have the authorization of the administrator, such as MDJMSU or to a group like MDJMSG. We could have altered the response files earlier in 2.6.2, “Preparing the standalone profile response file” on page 117 to set a different user ID, instead of changing it afterward. 140 z/OS: WebSphere Business Process Management V6.2 Production Topologies Authentication aliases The administrator’s user ID ends up being assigned to some of the ME auth aliases as you can see in Figure 2-19. Figure 2-19 Administrator ID used JMS authentication aliases The WSADMIN user ID is a normal user ID that is subjected to a password expiry. When the WSADMIN user ID is used on these aliases, the cell is going to stop working when the password expires. These aliases should be assigned to a different user ID like MDJMSU, which is set up with a non-expiring password. The bus connector role should be assigned to MDJMSU or to a group like MDJMSG. Note: This could have been avoided if we used MDJMSU as the administrator’s ID in the response file for the zWPSConfig.sh script in 2.6.2, “Preparing the standalone profile response file” on page 117. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 141 Fix for APAR PK74286 If you are using DB2 version 9, add a sib.properties file to avoid the CWSIS1530E message. See “Adjunct errors - see APAR PK74286” on page 609 Here is a synopsis: CWSIS1530E: The data type, 12, was found instead of the expected type, -1, for column, URI, in table, M101A.SIBCLASSMAP. To circumvent this, add the following line to a sib.properties file in the properties subdirectory of all profiles where servers that host messaging engines run: sib.msgstore.jdbcPerformColumnChecks=false This disables column checking, allowing the messaging engine to start. Remove the symbolic link in the following file and replace it with a real file with the above line in it: /wasv61config/m1cell/m1nodeb/AppServer/properties/sib.properties 2.9 Testing sample applications If everything looks right in the Integrated Solutions Console, try installing a sample application. There are two listed here that are easy to test. These applications do not have a security constraint and will not prompt for login. You can permit the MDGUEST user ID READ access to the required roles. 2.9.1 Installing a simple BPC application (WPSEcho.ear) There are two sample BPEL applications that are available in the additional materials for this Redbooks publication (and in Techdoc PRS2520.) Extract the WPSEcho.ear file to a working directory on your workstation. The WPSEcho application has common event logging enabled. This was done in WebSphere Integration Developer. This application does not have a security constraint and therefore will not prompt for login. Permit the MDGUEST user ID READ access to the CEI eventCreator role (MD.eventCreator). If you wish to test a copy of this without CEI logging, select the wpsecho-nocei-20080909.ear file. 142 z/OS: WebSphere Business Process Management V6.2 Production Topologies 1. Using the Integrated Solutions Console, go to Applications → Install New Application, as shown in Figure 2-20. Figure 2-20 Preparing for the application installation 2. When you get to the window that shows steps 1 through 8: a. Click STEP 8 – Summary and click Finish. b. When you see the message “Application WPSEchoApp installed successfully”, click the Save to Master Configuration link. 3. Start the WPSEchoApp application in the Integrated Solutions Console. 4. Click Applications → Enterprise Applications. 5. Select WPSEchoApp. 6. Click Start. 7. Test it by with the following URL: http://<host>:<port>/WPSEchoWeb/WPSEchoInput.html 8. The reply from the Echo Application should be echoed back to your window. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 143 2.9.2 Installing a long-running BPEL application Perform the following steps ton install a long-running BPEL application 1. Install the WscHtTest.ear (Human Task Application), which can be found in the SampleWpsHtmApps directory. See Appendix A, “Additional material” on page 615. (Take all the defaults.) 2. Start the WscHtTest application in the administrative console. 3. Test it with the following URL: http://<host>:<port>/WscHtTestWeb/WscHtTestInput.html 4. Enter a character string and click Submit. 5. The application will respond with a reply Request sent for processing. 6. Go to the BPC Explorer worklist at: http://<host>:<port>/bpc 7. Select My Tasks, select the work item, and click Work on. This is shown in Figure 2-21. Figure 2-21 BPC Explorer - One HtTask in “My Tasks” 144 z/OS: WebSphere Business Process Management V6.2 Production Topologies 8. Click Complete, as shown in Figure 2-22 Figure 2-22 BPC Explorer - Mark Task Complete 9. Look in the SYSPRINT output of the servant region and see messages starting with “WscHtTest”. These messages show that the Human Task was invoked, a message was received, and the process completed (Example 2-39). Example 2-39 SYSPRINT output WscHtTest: WscHtTest: WscHtTest: WscHtTest: WscHtTest: WscHtTest: Request received. Input data = Input to HumTask Invoking Human Task. Reply recieved from Human Task. Human Task reply = true Process complete. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 145 2.10 Testing the vehicle loan process application This Redbooks publication provides a sample vehicle loan process created for the fictitious company ITSOBank. You can use this vehicle loan process to test the topology you have built in this chapter. The sample application described in this section is supplied in Appendix A, “Additional material” on page 615. For more information about the vehicle loan process, refer to WebSphere Business Process Management V6.2 Production Topologies, SG24-7732. 2.10.1 Installing the sample application When you have obtained the additional materials, navigate to the Scenarios\WPS\EAR\v6.2_Plain directory. From here, you need to copy the ITSOApp.ear and ITSO_implApp.ear files to a working directory on your workstation. Installation follows the normal process and is described here. 1. Navigate to Applications → Install New Application. 2. Choose Local file system, click Browse, and navigate to the location of the uploaded EAR files. Select the ITSO_implApp.ear radio button, and click OK. Select the Prompt me only when additional information is required radio button, as shown in Figure 2-23. Click Next. Figure 2-23 Preparing for the application installation window 146 z/OS: WebSphere Business Process Management V6.2 Production Topologies 3. On the “Step 1: Select installation options” window, click Next. 4. On the “Step 2: Map modules to servers” window, select WebSphere:cell=mdcell,cluster=mdsr1. Select the ITSO_implWeb check box, click Apply, and click Next. 5. On the “Step 3: Summary” window click Finish. 6. When you see the message “Application ITSO_implApp installed successfully”, click Save, and OK. 7. Repeat the process for the ITSOApp.ear file. 2.10.2 Updating the SCA import URLs In the sample application, all the Web service SCA Imports are bound to URLs that point to itsodmgr. There is no port explicitly defined in the URLs, so the default HTTP port of 80 is used. You will likely need to change these to match your hostname:port. To do so, perform the following steps: Note: Ensure that at least one WebSphere Process Server for z/OS cluster member is started and that the three ITSO sample applications are started. 1. In the Integrated Solutions Console, navigate to Applications → SCA Modules and click ITSO. 2. Navigate to Imports → CreditCheckImport → Bindings, as shown in Figure 2-24. Figure 2-24 Expanding the CreditCheckExport bindings 3. Click CreditCheckExport_CreditCheckHttpService. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 147 4. Change the host name and port number in the Endpoint field (Figure 2-25). Click OK. When you have a cluster with Web servers running the WebSphere plug-in, you would normally specify the Distributed VIPA and port of the Web server. We specified the HTTP port of the WebSphere Process Server for z/OS servers because we had not finished testing the Web servers at the time we deployed the sample applications. Figure 2-25 Specify the appropriate host name and port 5. Repeat these steps for the remaining import bindings in the SCA module such as HighRiskLoadProvider through VerifyCustomerImport. 6. Save the configuration changes. 2.10.3 Testing the sample application Perform the following steps to test the sample application: 1. Navigate to Applications → Enterprise Applications. Select the ITSOApp and ITSO_implApp check boxes, and click Start. 2. Log in to the Business Process Choreographer console. In our environment we used the following URL: http://<server_host>:<http_port>/bpc 148 z/OS: WebSphere Business Process Management V6.2 Production Topologies 3. Click My Process Templates. You should see a window resembling Figure 2-26. Figure 2-26 Process Templates with ‘ew Loan Process’ 4. Select New Loan Process and click Start Instance. Provide some test input data and click Submit, as shown in Figure 2-27. Figure 2-27 Process input message This should launch the business process. Chapter 2. Configuring a WebSphere Process Server for z/OS standalone server 149 5. If the environment is working correctly, it returns a response in the Process Output Message window (Figure 2-28). Figure 2-28 Process output message Note: If you get the following error message in your browser instead of the process output message above, you may not have reset all the SCA service Endpoints: WSWS3713E: Connection to the remote host localhost failed.Received the following error: EDC8128I Connection refused. Uninstalling the BPEL applications To uninstall this, or any other enterprise application from the server, issue the commands shown in Example 2-40. Example 2-40 Uninstalling bpel applications cd <AppServer_root>/ProcessChoreographer/admin ../../bin/wsadmin.sh -lang jacl -f bpcTemplates.jacl -uninstall "<AppName>" -force 150 z/OS: WebSphere Business Process Management V6.2 Production Topologies 3 Chapter 3. Creating a WebSphere Process Server for z/OS network deployment cell This chapter describes how to configure a WebSphere Process Server for z/OS network deployment configuration, using as a basis, the standalone server that we built in Chapter 2, “Configuring a WebSphere Process Server for z/OS standalone server” on page 39. This will be a fully-functional WebSphere Process Server for z/OS configuration, but is only a stepping-stone to the production topology we described in Chapter 1. This chapter has the following sections: Overview Creating the deployment manager Adding WebSphere Process Server for z/OS to the deployment manager Federating the standalone server into this ND cell Testing the WebSphere Process Server-configured server Creating a cluster from the server Summary © Copyright IBM Corp. 2009. All rights reserved. 151 3.1 Overview After creating the standalone application and configuring it for WebSphere Process Server for z/OS, as discussed in Chapter 2, “Configuring a WebSphere Process Server for z/OS standalone server” on page 39, this chapter describes how to build a deployment manager, configure it for WebSphere Process Server for z/OS, and federate the standalone node into the deployment manager’s cell as in Figure 3-1: Daemon Daemon MDDEMN MDDEMNA Deployment Manager MDDMGR DMgr Servant MDDMGRS Node Agent MDAGNTA Federate Server MDSR1A Controller Servant MDSR1AS Adjunct MDSR1AA MDNODEA MDNODEA MDCELL Cell MDBASEA Figure 3-1 Federating the standalone node into the new deployment manager’s cell We then start the federated server and test the WebSphere Process Server for z/OS components in a network deployment configuration. Then we will create a cluster containing one server on this single node. and perform some administrative tasks to make it right. 152 z/OS: WebSphere Business Process Management V6.2 Production Topologies 3.1.1 Planning for the network deployment cell You should have already planned for the network deployment configuration in 2.2, “Planning the production network deployment topology” on page 44. If you have already created a standalone application server on z/OS, you should have already assembled the proper tools as described in 2.3, “Preparing the z/OS system components” on page 53. Note: If you are starting with the deployment manager before creating a standalone node, you need to plan your configuration and assemble the required tools at this point. Refer to and follow the following sections in Chapter 2, “Configuring a WebSphere Process Server for z/OS standalone server” on page 39. 2.2, “Planning the production network deployment topology” on page 44 2.3, “Preparing the z/OS system components” on page 53 , “We created sample JOB GEJBROLE to implement the role assignments in the table above for our MD cell. However, you cannot run this job before the BBOSBRAK job that is created and run as part of the server installation in 2.5.4, “Running the configuration jobs to create the standalone server” on page 108. We suggest you run these jobs to implement role assignments in 2.5.5, “Defining user IDs, group IDs, and EJBROLE profiles” on page 110.” on page 75 2.5, “Creating a standalone WebSphere application server” on page 102. Creating a deployment manager is similar to the steps for creating a standalone server, using the zPMT and the generated customization jobs. 2.6, “Adding WebSphere Process Server for z/OS” on page 114 2.7, “Creating the DB2 databases and tables” on page 122 3.2 Creating the deployment manager Configuring a WebSphere Application Server for z/OS deployment manager is described in the following IBM White Papers: WP101030,WebSphere z/OS V6.1 - A Top Down Configuration Approach This White Paper is available for download at the following Web page: http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101030 The follow White Paper is much more detailed, and may be used to study this task in more depth: Chapter 3. Creating a WebSphere Process Server for z/OS network deployment cell 153 WP100653, WebSphere for z/OS V6 Sample ND Configuration This White Paper is available for download at the following Web page: http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100653 In 3.2.1, “Task overview” on page 154, we summarize the main tasks involved in configuring the deployment manager. Note: WebSphere Process Server for z/OS Version 6.2 requires the WebSphere Application Server for z/OS Version 6.1 Feature Pack for Web Services. This may be included at the time the deployment manager is initially configured, or an existing deployment manager may be augmented with the Feature Pack for Web Services. We used the zProfile Management Tool (zPMT) in the WebSphere Customization Tool (WCT) to configure our WebSphere cell. See 2.5.3, “Using the zPMT to create the installation jobs” on page 105 The milestone in our WebSphere Process Server for z/OS configuration is when we have successfully started the deployment manager and have federated the standalone server into the cell. It is important not to federate any empty nodes into the cell until they have been augmented to support WebSphere Process Server for z/OS. 3.2.1 Task overview In summary, the steps to configure the WebSphere Application Server for z/OS deployment manager and federate the standalone server are as follows: 1. Use the same spreadsheet you used for the standalone server to select the response file variables from the DMGR tab. 2. Use zPMT to create a deployment manager profile with the Feature Pack for Web Services (FPWS) included. You can also use an existing deployment manager, and use the zPMT to augment it with the Feature Pack for Web Services. 3. Generate the customized jobstreams and upload to the target MDDMNODE.CNTL and MDDMNODE.DATA datasets. 4. Follow the instructions generated by zPMT and run the BBO* jobs to configure the deployment manager. 5. Start the deployment manager and verify that it is correctly built before you augment it for WebSphere Process Server for z/OS. 6. Back up the configuration ZFS. 154 z/OS: WebSphere Business Process Management V6.2 Production Topologies 3.2.2 Using the spreadsheet to plan the deployment manager You should already have done this as described in 2.5.2, “Using the spreadsheet to plan the server configuration” on page 104. 3.2.3 Using the zPMT to create the installation jobs Use the same spreadsheet you used for the standalone server, using the DMGR tab to create the response file with your customized variables. Use this response file as input to the WebSphere Application Server for z/OS zPMT to create a deployment manager profile with the Feature Pack for Web Services (FPWS) included. See Techdoc WP101084 for details about this activity. You can also use an existing deployment manager, and use the zPMT to create a batch job to augment it with the Feature Pack for Web Services. This is similar to the process you did for the standalone server as described in 2.5.3, “Using the zPMT to create the installation jobs” on page 105 3.2.4 Running configuration jobs to create the deployment manager Once the customized jobs and property files are uploaded to the z/OS partitioned dataset, you can go to your TSO session and open the target .CNTL PDS, and find the instructions in the BBOCCINS member. Follow the instructions. Follow the instructions, with these alterations: BBOSBRAJ and BBOSBRAK These can be run as described by the instructions. If you used the alternative RACF commands from Techdoc WP101427, these are unnecessary. BBOSBRAM Because we are using the same set of reduced group names and user IDs, this is identical to the job for the standalone server. This means you can skip this job, but it will not hurt to run it again. BBODBRAJ and BBODBRAK These are unnecessary if you used the generic set of commands from Techdoc WP101427. Chapter 3. Creating a WebSphere Process Server for z/OS network deployment cell 155 Note: Look in the output of the BBODBRAK job. If it gets a message similar to the following, you should re-run it with an earlier date. The certificate that you are creating has an incorrect date range. The certificate is added with NOTRUST status. See “BBODBRAK” on page 594 for details. BBOWCPY1 This copies the procedures for the deployment manager and daemon to Proclib. These are different from the similar procedures we created for the standalone server. BBODCFS This creates the ZFS or HFS filesystem for the deployment manager and mounts it. Note: If you are using alias names for the WebSphere BBO* load libraries, you must first run the job to create the aliases and symbolic links. See 2.3.6, “Using intermediate symbolic links” on page 65. The remaining jobs ran without problems. BBODHFSA This job populates the configuration ZFS filesystem. BBOWWPFD This job sets up the deployment manager runtime. BBODHFSB This job corrects the file ownership and permission settings. 3.2.5 Starting deployment manager and verifying the configuration Using the MVS console, or SDSF TSO panels, issue the start command for the node agent. On our system, the form of the command was as follows: START MDMGCR,JOBNAME=MDDMGR,ENV=MDCELL.MDDMNODE.MDDMGR Ensure the three address spaces start up. Example 3-1 shows what they look like in the SDSF DA window for our new deployment manager: 156 z/OS: WebSphere Business Process Management V6.2 Production Topologies Example 3-1 SDSF DA window showing the deployment manager started tasks SDSF DA SC42 (ALL) COMMAND INPUT ===> PREFIX=MD* DEST=(ALL) NP JOBNAME JobID MDDEMN STC24430 MDDMGR STC24428 MDDMGRS STC24440 PAG 0 CPU/L/Z 3/ 3/ 0 LINE 1-6 SC OWNER=* SORT=SysName/A JOBNAME/A SYSNA SysN CPU% Real StepName ProcStep Owner SC42 0.00 8870 MDDEMN BBODAEMN MDACRU SC42 0.07 69T MDDMGR BBOCTL MDACRU SC42 0.02 121T MDDMGRS BBOSR MDASRU 3.2.6 Reviewing and fixing any errors As we did with the new application server, it is good to look in the server’s log files on JES spool for any error messages, and look in the Integrated Solutions Console for the settings of various components. This is quite easy if you know what you are looking for. Examining the server logs on JES spool Using the SDSF Display Active window, select your deployment manager’s started tasks with the PREfix or Select MD* commands and look in the control region (MDDMGR), and servant regions (MDDMGRS). Select each with the S action character. Look for any severe error messages with the FIND SEVERE ALL command. Hopefully, there will be none. If there are, note the number of severe errors displayed by SDSF, and proceed through each one and start your problem determination process. Exploring with the Integrated Solutions Console Sign on to the Integrated Solutions Console with a URL of this form: http://<host>:<port_number>/ibm/console For our deployment manager, this was: http://wpsplex.itso.ibm.com:20018/ibm/console Because global security was enabled, we were re-directed to this URL: https://wpsplex.itso.ibm.com:20019/ibm/console Examine the following: JDBC providers: Derby embedded non-XA JDBC Provider and the XA DB2 Universal JDBC Driver-compliant Provider. Data sources (only 1 - the DefaultDatasource using the Derby JDBC Provider There are no servers, applications or buses to examine. Chapter 3. Creating a WebSphere Process Server for z/OS network deployment cell 157 3.2.7 Backing up the deployment manager’s ZFS data set Before running zSMPInstall.sh, back up the deployment manager’s configuration ZFS data set. We used the JCL shown in Example 3-2 to perform the back up. Example 3-2 JCL to back up the deployment manager’s node //MDDUMP JOB NOTIFY=&SYSUID,CLASS=A,MSGCLASS=A, // REGION=0M,TIME=1440 //* //* DUMP CONFIG HFS FOR DMGR AND NODE A //* //DUMP EXEC PGM=ADRDSSU //DASD DD DISP=(NEW,CATLG),DSN=WASCFG.MDCELL.DUMP.PREINS, // UNIT=3390,SPACE=(CYL,(500,100),RLSE) //SYSPRINT DD SYSOUT=* //SYSIN DD * DUMP DATASET( INCLUDE( WASCFG.MDCELL.MDDMNODE.CONFIG.ZFS ) ) OUTDD(DASD) TOL(ENQF) OPTIMIZE(4) COMPRESS /* 3.3 Adding WebSphere Process Server for z/OS to the deployment manager When you have defined a deployment manager you can install support for WebSphere Process Server for z/OS. Adding WebSphere Process Server for z/OS to the deployment manager’s node consists of the following steps: 1. Running the zSMPInstall script. See 3.3.1, “Running the zSMPInstall script” on page 159. 2. Preparing the DmgrDB2.rsp response file. See page 3.3.2, “Preparing the DmgrDB2.rsp response file” on page 161 3. Backing up the deployment manager’s ZFS. See page 3.3.3, “Backing up the deployment manager’s ZFS” on page 162 4. Running the zWPSConfig script. See page 3.3.4, “Running the zWPSConfig script” on page 163 158 z/OS: WebSphere Business Process Management V6.2 Production Topologies 5. Starting the deployment manager. See page 3.3.5, “Starting the deployment manager” on page 164. Note: Remember that all the JCL examples in this chapter are provided in Appendix A, “Additional material” on page 615. 3.3.1 Running the zSMPInstall script When invoked with the -install option the zSMPInstall.sh script performs the following tasks. (This is similar to the tasks performed when we ran it for the standalone server.) See 2.6.1, “Running the zSMPInstall script” on page 114 1. Check to ensure that the deployment manager is stopped before running the zSMPInstall.sh script. 2. You can invoke zSMPInstall.sh from a telnet command line, but we created sample job MDDMINS to do this, as shown in Example 3-3 on page 159. On the -smproot property we specified an intermediate symbolic link to the WebSphere Process Server for z/OS product, /wasv61config/mdcell/mddmnode/wps_smpe, instead of referring directly to the real mount point at/usr/lpp/zWPSMP/V6R2. Example 3-3 Job MDDMINS to invoke zSMPInstall.sh //MDDMINS JOB (0),CLASS=A,MSGCLASS=A,REGION=0M, // USER=MDADMIN,PASSWORD=MDADMIN //*JOBPARM SYSAFF=SC42 //***********************************************************/ //* Install WPS profiles into DMGR */ //***********************************************************/ //***********************************************************/ //* STEP TO CREATE LINKS TO WPS LIBRARIES */ //***********************************************************/ //INSTO EXEC PGM=IKJEFT01,REGION=0M,TIME=1440 //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BPXBATCH SH + cd /wasv61config/mdcell/mddmnode/AppServer/bin; + export PATH=.:$PATH; + /wasv61config/mdcell/mddmnode/wps_smpe/zos.config/bin/+ zSMPInstall.sh + '-smproot /wasv61config/mdcell/mddmnode/wps_smpe' + '-runtime /wasv61config/mdcell/mddmnode/DeploymentManager' + '-install' + Chapter 3. Creating a WebSphere Process Server for z/OS network deployment cell 159 1> /tmp/mddmnodeinstall_0511.out + 2> /tmp/mddmnodeinstall_0511.err //***********************************************************/ //* STEPS TO COPY THE OUTPUT THE JOB LOG */ //***********************************************************/ //CPOUT EXEC PGM=IKJEFT01,REGION=0M //SYSEXEC DD DISP=SHR,DSN=BBO6142.MDDMNODE.SBBOEXEC //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BBOHFSWR '/tmp/mddmnodeinstall_0511.out' BBOHFSWR '/tmp/mddmnodeinstall_0511.err' /* // Note: All jobs discussed in this chapter, including MDDMINS, are included in Appendix A, “Additional material” on page 615. The output from this job will look like Example 3-4. Example 3-4 MDDMINS job output BBOHFSWR CWPIZ0253I: CWPIZ0254I: CWPIZ0255I: CWPIZ0256I: CWPIZ0257I: CWPIZ0259I: CWPIZ0260I: CWPIZ0262I: CWPIZ0263I: CWPIZ0264I: '/tmp/installonly_1032.out' parsing command arguments... parsing arguments complete setting up configuration... set up configuration complete creating the symbolic links... creation of symbolic links complete doing post install file updates... post install updates complete running Configuration Manager update... Configuration Manager update complete If any errors occur in the job output, start by looking in the <dmgr-home>/logs/wbi/install/installconfig.log. When you find the error, if you issue FIND ‘buildfile’ PREVIOUS, it will tell you the ant file that failed. This will direct you to one of the following logs to troubleshoot the problem. It is helpful to sort the logs by date to look at the most recent log file first. In our configuration, <dmgr_home> was as shown in Example 3-5. Example 3-5 <dmgr_home> in our configuration /wasv61config/mdcell/mddmnode/DeploymentManager /<dmgr_home>/logs/wbi/c2n.log /<dmgr_home>/logs/wbi/zSMPInstall.log 160 z/OS: WebSphere Business Process Management V6.2 Production Topologies /<dmgr_home>/logs/wbi/zSMPInstall.trace /<dmgr_home>/logs/wbi/zWPSConfig.log /<dmgr_home>/logs/wbi/zWPSConfig.trace /<dmgr_home>/logs/wbi/100SCleanOSGICache.ant.log /<dmgr_home>/logs/wbi/15SCopyBSpaceIfNewer.ant.log /<dmgr_home>/logs/wbi/80SCopyInstallValidatorLog.ant.log /<dmgr_home>/logs/wbi/85SConfigNoProfileFirstStepsWBI.ant.log /<dmgr_home>/logs/wbi/90SCleanDeployTool.ant.log /<dmgr_home>/logs/wbi/90SConfigureWSProfileForWBI.ant.log /<dmgr_home>/logs/wbi/90SConfigWBIMigrationScript.ant.log /<dmgr_home>/logs/wbi/91SConfigNoProfileFirstStepsCharset.ant.log /<dmgr_home>/logs/wbi/93SDeployBPCAdminConsolePlugins.ant.log /<dmgr_home>/logs/wbi/93SDeployBusinessSpaceAdminConsoleFeature.log /<dmgr_home>/logs/wbi/93SDeployServerAdminConsolePlugins.ant.log /<dmgr_home>/logs/wbi/93SDeployWBICommonAdminConsolePlugins.ant.log /<dmgr_home>/logs/wbi/94SDeployCoreAdminConsolePlugins.ant.log 3.3.2 Preparing the DmgrDB2.rsp response file Parameters and their values are passed to the zWPSConfig.sh in a response file. A sample response file called DmgrDB2.rsp is provided with WebSphere Process Server for z/OS for augmenting a deployment manager when you will use DB2 for z/OS as a data store. 1. Copy the DmgrDB2.rsp response file from the /<wpsspme>/zos.config/ to a work directory and rename it dmDmgrDB2.rsp. In our case, <wpsspme> is a symbolic link that points to /usr/lpp/zWPSMP/V6R2 The work directory was /var/WebSphere/home/MDCFG/wpswork. Note: We were configuring more than one WebSphere Process Server for z/OS cell and so decided to change the name of the response file from DmgrDB2.rsp to mddmnodeDmgrDb2.rsp to avoid confusion. 2. Customize the properties in the response file as shown in Example 3-6. Example 3-6 mdDmgrDB2.rsp DBPRODUCT=DB2UDBOS390_V8_1 DBLOCATION=DB9F DBJDBCCLASSPATH=/usr/lpp/db2/d9fg/db2910_jdbc/classes DBUSER=MDDBU DBPASS=MDDBU Chapter 3. Creating a WebSphere Process Server for z/OS network deployment cell 161 DBHOSTNAME=wpsplex.itso.ibm.com DBSERVERPORT=37893 SQLID=MDWPS SQLDB=MDWPSDB SQLSTO=MDWPSSG augment profileName=default profilePath=/wasv61config/mdcell/mddmnode/DeploymentManager/profiles/de fault templatePath=/wasv61config/mdcell/mddmnode/DeploymentManager/profileTem plates/dmgr.wbiserver cellName=mdcell nodeName=mddmnode enableAdminSecurity=true adminUserName=mdadmin adminPassword=mdadmin dbCreateNew=false dbDelayConfig=true dbName=$SQLDB dbStorageGroup=$SQLSTO dbType=$DBPRODUCT dbConnectionLocation=$DBLOCATION dbJDBCClasspath=$DBJDBCCLASSPATH dbUserId=$DBUSER dbPassword=$DBPASS dbHostName=$DBHOSTNAME dbServerPort=$DBSERVERPORT dbSchemaName=$SQLID dbDriverType=2 3.3.3 Backing up the deployment manager’s ZFS Back up the deployment manager configuration ZFS data set again before running zWPSConfig.sh. We used the JCL shown in Example 3-2 on page 158 but changed the //DASD DD card to //DASD DD DSN=WASCFG.MDCELL.DUMP.DMGR.POSTWPS. 162 z/OS: WebSphere Business Process Management V6.2 Production Topologies 3.3.4 Running the zWPSConfig script After preparing the response file, we executed the shell script zWPSConfig.sh by submitting job MDDMCFG, as shown in Example 3-7. This job took about 6 minutes to run. Example 3-7 MDDMCFG JCL to run zWPSConfig for the DMGR //MDDMCFG JOB (0),CLASS=A,MSGCLASS=A,REGION=0M, // USER=MDADMIN,PASSWORD=MDADMIN /*JOBPARM SYSAFF=SC42 //***********************************************************/ //* Configure WPS in deployment manager mddmnode */ //***********************************************************/ //* Run zWPSConfig.sh */ //***********************************************************/ //INSTO EXEC PGM=IKJEFT01,REGION=0M,TIME=1440 //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BPXBATCH SH + cd /wasv61config/mdcell/mddmnode/DeploymentManager/bin; + ./zWPSConfig.sh + -response /var/WebSphere/home/MDCFG/wpswork/+ mdDmgrDB2.rsp + -augment + 1> /tmp/MDdmnodezWPSConfig_1.out + 2> /tmp/MDdmnodezWPSConfig_1.err; /* //***********************************************************/ //* STEPS TO COPY THE OUTPUT THE JOB LOG */ //***********************************************************/ //CPOUT EXEC PGM=IKJEFT01,REGION=0M //SYSEXEC DD DISP=SHR,DSN=BBO6142.MDDMNODE.SBBOEXEC //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BBOHFSWR '/tmp/MDdmnodezWPSConfig_1.out' BBOHFSWR '/tmp/MDdmnodezWPSConfig_1.err' /* Be sure to run the job using the WebSphere Administrator user ID. Ensure that the response file (mdDmgrDB2.rsp in Example 3-7 on page 163) is readable by the WebSphere administrator user ID. Chapter 3. Creating a WebSphere Process Server for z/OS network deployment cell 163 3.3.5 Starting the deployment manager Note: Once the deployment manager has been augmented for WebSphere Process Server for z/OS, do not start it until after the DB2 tables are created and verified which you should have done in 2.7, “Creating the DB2 databases and tables” on page 122. Now that the deployment manager’s node has been configured for WebSphere Process Server for z/OS, restart it to confirm that it initializes properly before federating the standalone node in to this new cell. Using the MVS console, or SDSF TSO windows, issue the start command for the node agent. On our system, the form of the command was as follows: START MDMGCR,JOBNAME=MDDMGR,ENV=MDCELL.MDDMNODE.MDDMGR Make sure the three address spaces start up. Example 3-8 on page 164 shows what they look like in the SDSF DA window for our deployment manager MDDMGR. Example 3-8 SDSF DA window for a deployment manager SDSF DA SC42 COMMAND INPUT NP JOBNAME MDDEMN MDDMGR MDDMGRS SC42 ===> JobID STC25962 STC25960 STC25965 PAG SysN SC42 SC42 SC42 0 CPU/L/Z 4/ 4/ 0 DATA SET SC CPU% Real StepName ProcStep Owner 0.00 8800 MDDEMN BBODAEMN MDACRU 0.00 84T MDDMGR BBOCTL MDACRU 0.00 159T MDDMGRS BBOSR MDASRU 3.3.6 Verifying the deployment manager runtime configuration Review the deployment manager’s log files (SYSPRINT on spool) for any error messages, and look in the Integrated Solutions Console for the settings of various components. This is quite easy if you know what you are looking for. See 2.8.1, “Review any error messages and repair” on page 138. 3.3.7 Backing up the HFS Back up the deployment manager’s configuration ZFS data set again after running zWPSConfig.sh. We used the JCL shown in Example 3-2 on page 158 but changed the //DASD DD card to //DASD DD DSN=WASCFG.MDCELL.DUMP.DMGR.POSTCFG. 164 z/OS: WebSphere Business Process Management V6.2 Production Topologies 3.4 Federating the standalone server into this ND cell Perform the following steps to federate the standalone application server’s node into the newly-created deployment manager’s cell. 1. Use the same spreadsheet you used for the standalone server, to select the response file variables, this time from the “Federate” tab. 2. Use the zPMT to create a profile to “Federate an application server”, and step through the windows, reviewing the values in each. 3. Generate the customized jobstreams and upload to the target MDFEDA.CNTL and MDFEBA.DATA datasets. 4. Follow the instructions generated by zPMT and select the BBOWMNAN job to federate the standalone server. 5. Add the user ID and password of the WebSphere administrator user to the BBOWMNAN job card. For example, the job card when we federated our standalone node into the MDCELL was as shown in Example 3-9 on page 165. Example 3-9 Job card //BBOWMNAN JOB (0),'WPS SETUP',CLASS=A,REGION=0M, // MSGCLASS=H,NOTIFY=&SYSUID, // USER=MDADMIN,PASSWORD=MDADMIN /*JOBPARM SYSADFF=SC42 Use a /*JOBPARM SYSAFF card to ensure that the BBOWMNAN job runs on the LPAR, which is the home of the primary node you are federating. If you chose the option to start the node agent after federation, the BBOWMNAN job will issue a start command for the node agent. BBOWMNAN must run on the LPAR where the node agent is started. Chapter 3. Creating a WebSphere Process Server for z/OS network deployment cell 165 Note: If you do not run the job under the WebSphere administrator user ID, and security is enabled in the cell, the BBOWMNAN job will fail with an error related to SSL. This is because the user ID running the job must have the deployment manager’s CA certificate in its keyring in order for an SSL handshake with the deployment manager to succeed. The WebSphere administrator user ID has such a keyring. A normal TSO user ID will not, unless you create a keyring for your TSO user ID and connect the cell’s CA certificate to the keyring. Running BBOWMNAN under the WebSphere administrator user ID avoids these problems. If you run the job under an incorrect user ID, you may also end up altering the ownership of various files in the deployment manager’s configuration HFS. If this happens, you may need to correct them with the bbowhfsb and bbowhfsc shell scripts. See “Fixing permissions & ownership of files in configuration HFS” on page 578. 6. Submit BBOWMNAN. If you chose to start the node agent during federation, the node agent is started by job BBOWMNAN just before it ends. If you have the OPERCMDS profile actively controlling access to MVS Start commands (like START.**) you must provide UPDATE access to the administrator’s user ID for this profile. 3.4.1 Updating the primary node’s node agent ports Most of the node agent ports match those you planned to use because they are configured during federation according the values that were passed into the zPMT tool when configuring the node. However, some of the ports take default values. Set these to the planned port numbers by performing the following steps, or use the updNewServer.py jython script available in Techdoc TD104066. 1. Log in to the WebSphere Integrated Solutions Console. 2. Navigate to System administration → Node agents and click the link to the first node agent. Note: If you were already logged into the WebSphere Integrated Solutions Console before the node was federated, you may not see any nodes listed when you navigate to System administration → Node agents. Log out of the Integrated Solutions Console and log in again. 166 z/OS: WebSphere Business Process Management V6.2 Production Topologies 3. Under Additional Properties, click Ports. The ports are listed as shown in Figure 3-2. Notice how some are taking default values. Figure 3-2 Node agent ports immediately after federation Note: The following ports are invalid for z/OS, and can be deleted: CSIV2_SSL_MUTUALAUTH_LISTENER_ADDRESS CSIV2_SSL_SERVERAUTH_LISTENER_ADDRESS SAS_SSL_SERVERAUTH_LISTENER_ADDRESS. The only one which we needed to update is DRS_CLIENT_ADDRESS. Chapter 3. Creating a WebSphere Process Server for z/OS network deployment cell 167 4. Click each of the port names in turn and delete them or set the correct value as shown for the DRS_CLIENT_ADDRESS in Figure 3-3. Figure 3-3 Correcting a node agent port 5. When you have finished, the ports must all be as planned on the cell-planning spreadsheet (Figure 3-4). Figure 3-4 Corrected node agent ports 168 z/OS: WebSphere Business Process Management V6.2 Production Topologies 6. Save the changes and synchronize to the nodes. You must restart the node agent to pick up the port number change. 3.4.2 Backing up the HFS Back up the deployment manager’s configuration ZFS data set again after running zWPSConfig.sh. We used the JCL shown in Example 3-2 on page 158 but changed the //DASD DD card to //DASD DD DSN=WASCFG.MDCELL.DUMP.DMGR.POSTFED. 3.5 Testing the WebSphere Process Server-configured server At this point we have a network deployment configuration with a deployment manager, node agent, and single server, all configured for WebSphere Process Server for z/OS on one system (LPAR) in the sysplex. 3.5.1 Starting the node agent Using the MVS console, or SDSF TSO windows, issue the start command for the node agent. On our system, the form of the command was as follows: START M1ACRA,JOBNAME=M1AGNTA,ENV=M1CELL.M1NODEA.M1AGNTA 3.5.2 Starting the server Using the Integrated Solutions Console, navigate to Server → Application servers, select the mdsr1a server, and click Start. Look for any errors as you did before with the standalone server as described on 2.8.1, “Review any error messages and repair” on page 138. 3.5.3 Testing sample applications Rerun the test applications you used in 2.9.1, “Installing a simple BPC application (WPSEcho.ear)” on page 142 and 2.9.2, “Installing a long-running BPEL application” on page 144. Chapter 3. Creating a WebSphere Process Server for z/OS network deployment cell 169 3.5.4 Backing up the ZFSes in this ND cell At this time, we decided to take a back-up of the network deployment configuration ZFS data set again before proceeding. We used the JCL shown in Example 3-10 on page 170 but changed the //DASD DD card as follows: //DASD DD DSN=WASCFG.MDCELL.DUMP.POSTFED.NODEA Example 3-10 Backing up the new MD deployment manager cell with a federated node //MDDUMPD1 JOB NOTIFY=&SYSUID,CLASS=A,MSGCLASS=A, // REGION=0M,TIME=1440 //* //* DUMP CONFIG HFS FOR DMGR AND NODEA //* //DUMP EXEC PGM=ADRDSSU //DASD DD DISP=(NEW,CATLG),DSN=WASCFG.MDCELL.DUMP.POSTFED.NODEA, // UNIT=3390,SPACE=(CYL,(1500,100),RLSE),VOL=SER=TSTOxx //SYSPRINT DD SYSOUT=* //SYSIN DD * DUMP DATASET( INCLUDE( WASCFG.MDCELL.MDDMNODE.CONFIG.ZFS WASCFG.MDCELL.MDNODEA.CONFIG.ZFS ) ) OUTDD(DASD) TOL(ENQF) OPTIMIZE(4) COMPRESS /* 3.6 Creating a cluster from the server You can now create a new cluster with the existing server that we just federated as its only member. Alternatively, you can wait until the empty node MDNODEB is federated into this cell, and create the cluster then. We did it here to verify that the server from the standalone server would retain all of the WebSphere Process Server for z/OS configured resources when it became a member of a cluster. 170 z/OS: WebSphere Business Process Management V6.2 Production Topologies Make certain that the server is stopped. 1. In the Integrated Solutions Console, navigate to Servers → Clusters, and select New. Enter the new cluster long and short names as in Figure 3-5. Figure 3-5 Step 1, Enter new cluster long and short names Chapter 3. Creating a WebSphere Process Server for z/OS network deployment cell 171 2. Specify your cluster long name (ours was mdsr1), and cluster short name (MDSR1). Click Next to advance to Step 2 in (Figure 3-6). Figure 3-6 Step 2, Create first cluster member 3. Click Create the member by converting an existing application server. and select the mdsr1a server. 172 z/OS: WebSphere Business Process Management V6.2 Production Topologies 4. Click Next. You will be asked to add additional members, but there are none, so click Next again to advance to the “Step 4: Summary” window (Figure 3-7). Figure 3-7 Cluster create summary 5. Click Finish to create the cluster, and save it with the Synch to nodes box checked. The transformation of a server into a clustered server changes many underlying resources and messaging engines, adding definitions to the Scope of the new cluster. 3.6.1 Checking the virtual host alias ports and short names When the server was promoted to a cluster member, several resources were redefined at the cluster level. Check the virtual host alias host names and port numbers. In the Integrated Solutions Console, navigate to Environment → Virtual Hosts → default_host → Host Aliases. Chapter 3. Creating a WebSphere Process Server for z/OS network deployment cell 173 3.6.2 Fixing the CEI messaging engine Note: This step is unnecessary if you are not starting with a standalone server, and you created a clustered server in the empty node. When the CEI messaging engine was moved to the cluster level, several attributes were lost: The schema name and Create Tables attributes were reset to their default values in the data store custom properties. We logged into the Integrated Solutions Console, in the CEI bus ME datasource. (Navigate to servers → server_name → Messaging engines → cei_bus_me_name → Data store.) We set the correct values for Schema (MDSR1C) and cleared the Create tables check box. The following destinations were lost in the CEI bus: BPCCEIConsumerQueueDestination_mdnodea_mdsr01a_WPS BPCTransformerQueueDestination_mdnodea_mdsr01a_WPS We added these queue destinations back using the Integrated Solutions Console. Figure 3-8 on page 175 shows the destinations with the BPCTransformerQueueDestination... missing. 174 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 3-8 CEI messaging engine destinations Chapter 3. Creating a WebSphere Process Server for z/OS network deployment cell 175 Click New. You will be asked whether it is a Queue, Topic Space, or Foreign. Select Queue and click Next. You then will enter the Create New Queue Wizard. Enter the name of the missing queue destination that you want to replace in the Identifier box, and click Next as in Figure 3-9. Figure 3-9 Create new queue steps Click Next, Next and Finish. Save the changes, synchronize to nodes, and re-cycle the adjunct. We encountered a problem when the CEI Bus could not obtain a lock on its data source. (CWSIS1501E: ...because the persistence layer has been stopped.) We dropped and re-created SIB Tables for CEI messaging engine to keep the database tables in sync with the data source. Note: We had another cell (the M1CELL) handy to examine the settings in that server. This proved to be valuable to have a back-up configuration to compare when things changed. 3.6.3 Restarting the cluster member After making the above repairs to the clustered server, we restarted the server and verified all was well by looking in the SYSPRINT and Integrated Solutions Console. 176 z/OS: WebSphere Business Process Management V6.2 Production Topologies 3.6.4 Backing up the ND cell containing the clustered server We decided to take a back-up of the network deployment configuration ZFS data set again before proceeding. We used the JCL shown in Example 3-2 on page 158 but changed the //DASD DD card as follows: //DASD DD DSN=WASCFG.MDCELL.DUMP.NODEA.POSTCLUS 3.7 Summary Figure 3-10 shows where we are in the process of growing our standalone server into a production topology. Deployment Manager MDDMGR Daemon MDDEMN DMgr Servant MDDMGRS Node Agent MDAGNTA Server MDSR1A Cluster MDSR1 Controller Servant MDSR1AS Adjunct MDSR1AA MDNODEA MDCELL Figure 3-10 Network deployment cell with a single cluster Chapter 3. Creating a WebSphere Process Server for z/OS network deployment cell 177 We now have a network deployment node on one LPAR with a single cluster containing one server with the following products installed: WebSphere Application Server for z/OS Feature Pack for Web Services WebSphere Process Server for z/OS WebSphere Business Services Fabric for z/OS In the next chapter, we add another LPAR with another node into our cell, and create another cluster member for the final single-cluster production topology on z/OS. 178 z/OS: WebSphere Business Process Management V6.2 Production Topologies 4 Chapter 4. Creating additional WebSphere Process Server nodes This chapter describes the third major phase in our road map to the production topology. In this chapter, we describe how to extend our network deployment configuration on a single system to a clustered set of WebSphere Process Server for z/OS servers spread across multiple systems. This chapter contains the following sections: Review and overview Creating an empty node Configuring the new node for WebSphere Process Server for z/OS Federating the new node into the network deployment cell Creating an additional cluster member Validating the new cluster © Copyright IBM Corp. 2009. All rights reserved. 179 4.1 Review and overview At this point in the process to create the production topology, we have a network deployment WebSphere Process Server for z/OS cell on a single LPAR with a deployment manager and a single (clustered) node, as shown in Figure 4-1. Deployment Manager MDDMGR Daemon MDDEMN DMgr Servant MDDMGRS Node Agent MDAGNTA Server MDSR1A Cluster MDSR1 Controller Servant MDSR1AS Adjunct MDSR1AA MDNODEA MDCELL Figure 4-1 The WebSphere Process Server for z/OS single system network deployment configuration In the following sections of this chapter, we perform the following steps to create a secondary node with WebSphere Process Server for z/OS to establish our production topology as shown in Figure 4-2 on page 181. 180 z/OS: WebSphere Business Process Management V6.2 Production Topologies Deployment Manager MDDMGR Node Agent Node Agent MDAGNTA MDAGNTB Server Server MDSR1A MDSR1B Controller Controller Servant Servant MDSR1AS MDSR1BS Adjunct Adjunct MDSR1AA MDSR1BA MDNODEA MDNODEB Cluster MDSR1 MDCELL SYSA SYSB Figure 4-2 The finished WebSphere Process Server for z/OS production topology 1. Create an empty node with the zPMT-generated jobstreams. 2. Augment the empty node with the zSMPInstall.sh and zWPSConfig.sh scripts. 3. Update the new servant and adjunct started procedures with STEPLIBs for DB2 libraries. 4. Federate the new empty node into the production cell. 5. Clean up ports and other settings. 6. Validate the new configuration by running sample applications. 7. Back up the configuration ZFS’s at each milestone. Chapter 4. Creating additional WebSphere Process Server nodes 181 4.2 Creating an empty node Use the spreadsheet and zPMT to configure a secondary empty node up through the job that completes the empty node’s file system initialization (BBOMHFSB), but do not federate the node into the cell. That is, do not run the BBOWMNAN job. The node must be augmented to support WebSphere Process Server for z/OS before it is federated in 4.4, “Federating the new node into the network deployment cell” on page 188. If you want to have more nodes in the cell, repeat the tasks in this section for each secondary node. We will use the familiar tools (spreadsheet, zPMT, and customized jobs to create a secondary node. 4.2.1 Using the spreadsheet to plan the server configuration In the same spreadsheet used in 2.2.2, “Using a spreadsheet to plan names for configuration objects” on page 45, you can find the response file definitions for a secondary node in the tab along the EmptyNode_S button. Click that tab, click the column A heading, and copy it to your clipboard. Open up a text editor, paste it there, and save it as a file in the directory where you keep your response files. 4.2.2 Using the zPMT to create the installation jobs Perform the following steps to use the zPMT to create the installation jobs: 1. Start the WebSphere Customization Tool, and select the zPMT. 2. Select the MDCELL in the top of the window, and click Create in the bottom of the window. The “Environment Selection” window (Figure 4-3 on page 183) displays. 182 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 4-3 zPMT Environment Selection for an empty node with Feature Pack for Web Services Chapter 4. Creating additional WebSphere Process Server nodes 183 3. Select Managed (custom) node with Feature Pack for Web Services. Click Next and Browse for the location of your response file that you copied from the spreadsheet. Follow the same steps to generate and upload the customized jobstreams to your z/OS target system. 4.2.3 Preparing the new z/OS system As for the first system where we built the standalone server, we must also define the alias names for the WebSphere Application Server for z/OS load libraries for this new node. See 2.3.4, “Planning z/OS dataset names and aliases” on page 61. This must be done before the configuration jobs are run. Specifically, before the BBOMHFSA job can run. A symbolic link for the WebSphere Process Server for z/OS SMP/E product ZFS must be defined in the new empty node’s mount point. See 2.3.6, “Using intermediate symbolic links” on page 65 for details. 4.2.4 Running the configuration jobs to create an empty node Once the customized jobs and property files are uploaded to the z/OS partitioned dataset, you can go to your TSO session and open the target .CNTL PDS, and find the instructions in the BBOMDINS member. Follow the instructions, with these alterations: BBOSBRAJ and BBOSBRAK These can be run as described by the instructions. If you used the alternative RACF commands from Techdoc WP101427, then these are unnecessary. BBOSBRAM Pay attention to where the home directories get created. Even though we ran this same job for the standalone server on the first system, the same home directories for this node must be created because they are in the /var/ root, which is a system-specific file system: /SC53/var/. This is where you can put the .profile and .setup files. BBOCBRAJ and BBOCBRAK These are unnecessary if you used the generic set of commands from Techdoc WP101427. 184 z/OS: WebSphere Business Process Management V6.2 Production Topologies BBOWCPYM This copies the procedures to Proclib. Note that these are different from the similar procedures we created for the standalone server. (There is a “B” in the sixth position of all the proclib members.) BBOMCFS This creates the ZFS or HFS filesystem and mounts it. BBOMHFSA If you are using alias names for the WebSphere BBO* load libraries, you must first run the job to create the aliases and symbolic links. See 4.2.3, “Preparing the new z/OS system” on page 184. The remaining jobs can be run without special attention. BBOWWPFM This job sets up the runtime file system. BBOMHFSB This job corrects the file ownership and permission settings of directories and file in the configuration ZFS. Important: Do not run the BBOWMNAN job at this time. We must first augment this empty node with WebSphere Process Server for z/OS before we federate it. 4.2.5 Customizing the servant and adjunct procedures for DB2 Servers in the new node will need access to the DB2 libraries for several activities when installing WebSphere Process Server for z/OS. If your DB2 libraries are not in link list or LPA, you must add them to your deployment manager procedures and setupCmdLine.sh script. See 2.5.7, “Customizing the new server” on page 112. Chapter 4. Creating additional WebSphere Process Server nodes 185 4.3 Configuring the new node for WebSphere Process Server for z/OS When you have created a new empty node, you can augment it for WebSphere Process Server for z/OS. Adding WebSphere Process Server for z/OS to the server’s node consists of the following steps (as we did to the standalone node and deployment manager): 1. Run the zSMPInstall,sh script 2. Copy the ManagedDB2.rsp file from the /usr/lpp/zWPSMP/V6R2/zos.config/ directory to your working directory, tailor it for your configuration, and save it with read permission the administrator’s user ID running the zWPSConfig.sh script. 3. Run the zWPSConfig.sh script See 2.6, “Adding WebSphere Process Server for z/OS” on page 114 for details. 4.3.1 Running the zSMPInstall script Start the augmentation of your new node for WebSphere Process Server for z/OS by running the zSMPInstall.sh script. Specify the root filesystem for WebSphere Process Server for z/OS and the new node’s filesystem root when you run the script, as shown in Example 4-1 Example 4-1 Running zSMPInstall.sh from a telnet command line su b6admin (pwd = 'xxxxxxx') cd /wasv61config/mdcell/mdnodeb/wps_smpe/zos.config/bin/ ./zSMPInstall.sh -smproot /wasv61config/mdcell/mdnodeb/wps_smpe \ -runtime <server_root> -install Note: The back-slash (‘\’) in Example 4-1 indicates that the command is continued on the next line. Do not type the back-slash if you are entering the command all on one line You can also run the script from a batch job. Refer to 2.6.1, “Running the zSMPInstall script” on page 114 for details. 186 z/OS: WebSphere Business Process Management V6.2 Production Topologies 4.3.2 Tailoring the response file for the empty node Parameters and their values are passed to the zWPSConfig.sh in a response file. A sample response file called ManagedDB2.rsp is provided with WebSphere Process Server for z/OS for augmenting a standalone server when you will use DB2 for z/OS as a data store. Perform the following steps to tailor the response file for the empty node 1. Copy the ManagedDB2.rsp response file from /<wps_install_root>/zos.config/ to a work directory and rename it mdnodebManagedDB2.rsp. In our case, the work directory was /var/WebSphere/home/MDCFG/wpswork. 2. Customize the properties in the MmdnodebManagedDB2.rsp response file as shown in Example 4-2. Example 4-2 Response file mdnodebManagedDB2.rsp for empty node DBPRODUCT=DB2UDBOS390_V8_1 DBJDBCCLASSPATH=/usr/lpp/db2/d9fg/db2910_jdbc/classes augment profileName=default profilePath=/wasv61config/mdcell/mdnodeb/AppServer/profiles/default templatePath=/wasv61config/mdcell/mdnodeb/AppServer/profileTemplates/ma naged.wbiserver cellName=bbotemp nodeName=mdnodeb dbType=$DBPRODUCT dbJDBCClasspath=$DBJDBCCLASSPATH dbDriverType=2 Note: The value of “bbotemp” is assigned to the cellName by the WCT/zPMT V7.0.0.1. A cell short name of BBOTEMP and a cell long name of bbotemp will be assigned to the unfederated managed node. These names will no longer be used after the managed node is federated into a network deployment cell. 3. Save this file in your wpswork directory with read permissions (644) so it can be read by the administrator’s ID (MDADMIN) when running the zWPSConfig.sh script in the next step. Chapter 4. Creating additional WebSphere Process Server nodes 187 4.3.3 Running zWPSConfig script After preparing the response file, we executed the shell script zWPSConfig.sh in a telnet session (Example 4-3), or in a batch job, as discussed in 2.6.3, “Running the zWPSConfig.sh script” on page 120. Example 4-3 Running the zWPSConfig shell script for the empty node in a telnet session su mdamin password: xxxx export LIBPATH=/usr/lpp/db2/d9fg/db2910_jdbc/lib:$LIBPATH cd /wasv61config/mdcell/mdnodea/AppServer/bin ./zWPSConfig.sh \ -response /var/WebSphere/home/MDCFG/wpswork/mdnodebManagedDB2.rsp \ -augment 4.3.4 Backing up the configuration ZFS Back up the server’s configuration ZFS data set again after running zWPSConfig.sh. We used the JCL shown in Example 2-15 on page 114 and changed the DASD DD statement to have a meaningful suffix like “NODEB.POSTWPS”. 4.4 Federating the new node into the network deployment cell Perform the following steps to federate the newly-defined empty node into the deployment manager’s cell. 1. Verify that the MDEMPTYB.DATA(BBOWMNPF) member does not contain the following variable: IPC_CONNECTOR_ADDRESS= This was added by a down-level version of the WCT zPMT (7.0.0.0). Delete the line or comment it out. See “BBOWMNAN job fails” on page 596. 2. Return to the MDEMPTYB.CNTL dataset that contains the jobs for configuring the primary empty node. Locate the BBOWMNAN job, and edit it. 3. Do not forget to set the user ID and password of the WebSphere administrator user to the job card. Make certain that the /*JOBPARM card specifies the system where the empty node was configured (SC53), as shown in Example 4-4 on page 189. 188 z/OS: WebSphere Business Process Management V6.2 Production Topologies Example 4-4 Job card //MDOWMNAN JOB (0),'WPS SETUP',CLASS=A,REGION=0M, // MSGCLASS=H,NOTIFY=&SYSUID, // USER=MDADMIN,PASSWORD=MDADMIN /*JOBPARM SYSADFF=SC53 4. Submit BBOWMNAN. If you chose to start the node agent during federation, the node agent is started by job BBOWMNAN just before it ends (assuming the MDADMIN user ID is allowed to issue MVS start commands and not restricted by the RACF OPERCMDS profile (MVS.** or MVS.START.**...) 4.5 Creating an additional cluster member You can create this cluster after the first server is federated, or wait until the second node is federated before you create the cluster. We chose to create the cluster before this new node MDNODEB was federated. We now have a network deployment cell on system SC42, and an empty node on SC53 federated into our cell, but the new node (MDNODEB) does not have any servers. The next thing to do is add another member to our cluster MDSR1, which we can do quite easily through the Integrated Solutions Console as described below: Perform the following steps to create a cluster. Note: We created the initial cluster in 3.6, “Creating a cluster from the server” on page 170. If you did not yet create the cluster, we describe it here. 1. In the Integrated Solutions Console, navigate to Servers → Clusters. 2. Click Clusters. Chapter 4. Creating additional WebSphere Process Server nodes 189 3. In the Server clusters dialog box Click New (Figure 4-4). Figure 4-4 Create a new cluster 4. Specify the following options in the dialog box (Figure 4-5): a. Cluster name: MDSR1 b. Cluster short name: mdrs1 c. Select the Prefer local. Specifies whether enterprise bean requests will be routed to the node on which the client resides when possible check box. d. Clear the Configure HTTP session memory-to-memory replication check box. Figure 4-5 Cluster definition 5. Click Next. 190 z/OS: WebSphere Business Process Management V6.2 Production Topologies 6. Specify the following options in the dialog box (Figure 4-6): a. Select the Create the member by converting an existing application server check box. b. Select mdcell/mdnodea from the Create the member by converting an existing application server check box drop-down menu. c. Accept all the defaults that are selected based on the above choice. This selection will convert the stand-alone server in node A to a clustered server. Figure 4-6 Specify the first cluster member 7. Click Next. Chapter 4. Creating additional WebSphere Process Server nodes 191 8. Specify the following options in the “Step 3: Create additional cluster members” window (Figure 4-7): a. b. c. d. e. Select mdnodeb from the Select node drop-down menu. Enter mdrs01b_WPS for the Member name. Enter for the MDSR1B Short name. Enter 2 for the Weight. Clear the Generate unique HTTP ports check box. Figure 4-7 Create additional cluster member 192 z/OS: WebSphere Business Process Management V6.2 Production Topologies 9. Click Add Member (Figure 4-8). Figure 4-8 Cluster member definitions 10.Click Next. Chapter 4. Creating additional WebSphere Process Server nodes 193 11.Review options on the “Step 4: Summary” window (Figure 4-9). Figure 4-9 Cluster create summary 12.Click Finish. 13.Select the Synchronize changes with Nodes check box. 194 z/OS: WebSphere Business Process Management V6.2 Production Topologies 14.Click Save (Figure 4-10). Figure 4-10 Cluster create results 15.Click OK. 16.In the deployment manager console, navigate to Servers → Application servers. 17.Click Application servers. 18.Select the mdsr01a_WPS and mdrs01b_WPS check boxes. Chapter 4. Creating additional WebSphere Process Server nodes 195 19.Click Start (Figure 4-11). Figure 4-11 Application servers starting 4.6 Validating the new cluster We have now configured our production topology with a single cluster across two systems. 4.6.1 Starting the node agent Using the MVS console, or SDSF TSO windows, issue the start command for the node agents. On our system, this was the form of the following commands: SC42 START MDACRA,JOBNAME=MDAGNTA,ENV=MDCELL.MDNODEA.MDAGNTA SC53 START MDACRB,JOBNAME=MDAGNTB,ENV=MDCELL.MDNODEB.MDAGNTB 4.6.2 Starting the new cluster member Using the Integrated Solutions Console, navigate to the Server → Application servers, select the mdsr01b_WPS server, and click Start. Look for any errors as you did before with the standalone server as described in section 2.8.1, “Review any error messages and repair” on page 138. 196 z/OS: WebSphere Business Process Management V6.2 Production Topologies 4.6.3 Looking at the final topology The production topology for WebSphere Process Server for z/OS is as shown in Figure 4-12. Controller Servant MDSR1AS (WPS) Servant MDSR1BS (WPS) BPC (HTM and BFM) CEI BPC Explorer BRM Business Space BPC (HTM and BFM) CEI BPC Explorer BRM Business Space SCA SYS ME CEI ME MDNODEA MDNODEB CEI Bus Adjunct MDSR1BA BPC Bus Adjunct MDSR1AA SCA.SYSTEM Bus SCA APP ME Single Cluster mdsr1 SCA.APPLICATION Bus Controller Figure 4-12 The final topology 4.6.4 Testing sample applications Rerun the test applications you used in 2.9.1, “Installing a simple BPC application (WPSEcho.ear)” on page 142 and 2.9.2, “Installing a long-running BPEL application” on page 144. It is a good idea to test the applications using both the sysplex distributor's host address, and each server's host address. Chapter 4. Creating additional WebSphere Process Server nodes 197 198 z/OS: WebSphere Business Process Management V6.2 Production Topologies Part 3 Part 3 Extending production topologies © Copyright IBM Corp. 2009. All rights reserved. 199 200 z/OS: WebSphere Business Process Management V6.2 Production Topologies 5 Chapter 5. WebSphere Business Services Fabric for z/OS This chapter provides detailed instructions on how to incorporate WebSphere Business Services Fabric into a production WebSphere Process Server for z/OS topology. The WebSphere Business Services Fabric product consists of three distinct sets of functions: IBM Business Services Foundation Pack IBM Business Services Foundation Pack provides the runtime components for installing and managing WebSphere Business Services Fabric components, and the primary subject of this chapter. IBM Business Services Tool Pack IBM Business Services Tool Pack provides tooling extensions for creating business services. These tooling extensions run only on workstation platforms and are given a full treatment in WebSphere Business Process Management V6.2 Production Topologies, SG24-7732. Industry content packs Industry content packs are industry-specific solutions provided as a starting point for common patterns and implementations. We discuss the installation and configuration of the content packs on z/OS in 7.4, “Configuring the WebSphere Business Services Fabric for z/OS Content Packs” on page 451. This chapter contains the following sections: 5.1, “Introduction” on page 202 5.2, “Install WebSphere Business Services Fabric for z/OS in a standalone node” on page 206 5.3, “Optional configuration steps for stand-alone server” on page 259 5.4, “Troubleshooting the WebSphere Business Services Fabric for z/OS installation” on page 298 5.5, “WebSphere Business Services Fabric events” on page 298 Chapter 5. WebSphere Business Services Fabric for z/OS 201 For additional information refer to the following: For information about installing WebSphere Business Services Fabric for z/OS in a cluster, see Chapter 6, “Incorporating WebSphere Business Services Fabric for z/OS into the cluster” on page 299. For information about installing and testing applications for use with WebSphere Business Services Fabric for z/OS, see Chapter 7, “Installing and testing WebSphere Business Services Fabric for z/OS” on page 403. 5.1 Introduction WebSphere Business Services Fabric for z/OS is a comprehensive Service-Oriented Architecture (SOA)-based offering to deliver dynamic SOA that uses existing IT assets and delivers business value incrementally. For detailed information about WebSphere Business Services Fabric concepts, architecture, and the development of dynamic SOA applications, refer to Redbooks publication WebSphere Business Process Management V6.2 Production Topologies, SG24-7732. This chapter focuses on incorporating WebSphere Business Services Fabric into an existing WebSphere Process Server production topology. We tested in a single-topology with WebSphere Process Server for z/OS and WebSphere Business Services Fabric for z/OS in the same cluster. This is the BPM reference topology for z/OS as explained in 1.5, “BPM reference topology for z/OS and how we will use it in this book” on page 30 In this topology, a single cluster runs the WebSphere Process Server for z/OS support and messaging infrastructure along with components supporting Business Process Choreographer (BPC), WebSphere Business Services Fabric for z/OS, and the applications. Figure 5-1 on page 203 shows the BPM reference topology for z/OS. It also shows where WebSphere Business Services Fabric for z/OS components are added to it. 202 z/OS: WebSphere Business Process Management V6.2 Production Topologies Controller MDSR1A Controller MDSR1B Servant MDSR1AS (WPS) Servant MDSR1BS (WPS) BPC (HTM and BFM) CEI BPC Explorer BRM Business Space Fabric BPC (HTM and BFM) CEI BPC Explorer BRM Business Space Fabric Fabric ME Fabric Bus mdnodeb CEI Bus Adjunct MDSR1BA BPC Bus mdnodea BPC ME SCA.SYSTEM Bus Adjunct MDSR1AA CEI ME SCA.APPLICATION Bus SCA APP SCA SYS ME ME Single Cluster mdsr1 Figure 5-1 WebSphere Business Services Fabric for z/OS in a WebSphere network deployment cell When you add WebSphere Business Services Fabric for z/OS to the topology, the following components are added: A DB2 database is added to the DB2 server. We called ours MDFABDB. A service integration bus, named fabricbus, is added. In the BPM reference topology for z/OS, we added this locally to the cluster running the other BPM components (mdsr1 cluster in Figure 5-1). The WebSphere Business Services Fabric for z/OS core application EAR files are added to the cluster. In the BPM reference topology for z/OS the Fabric EAR files are deployed to the cluster running the other BPM components, (mdsr1 in Figure 5-1). WebSphere Business Services Fabric for z/OS events are emitted to the JMS destinations present in the cluster. Chapter 5. WebSphere Business Services Fabric for z/OS 203 5.1.1 Planning and preparing the Fabric installation This section describes the software pre-requisites and the installation of the product using SMP/E. It introduces the additional material we provide with this Redbooks publication. Finally there are some comments about creating the deployment directory. 5.1.2 Software pre-requisites The pre-requisites for WebSphere Business Services Fabric for z/OS are found in the IBM support article, System requirements for WebSphere Business Services Fabric V6.2 on z/OS, available at the following Web page: http://www.ibm.com/support/docview.wss?rs=36&uid=swg27013591 The key requirement is WebSphere Process Server for z/OS V6R2M0. The requirement for LDAP depends on whether you decide to use LDAP or a local SAF user registry. 5.1.3 Installing WebSphere Business Services Fabric for z/OS in SMP/E The instructions for installing WebSphere Business Services Fabric for z/OS into SMP/E are found in WebSphere Business Service Fabric 6.2.0 Program Directory, GI11-2874-01, which is shipped with the product. The program directory can also be downloaded from IBM Publications at the following Web page: http://www.ibm.com/shop/publications/order The product ZFS occupies around 2500 CYL of disk space. When SMP/E work is finished, the WebSphere Business Services Fabric product ZFS is typically mounted read-only at /usr/lpp/webify/V6R2M0. That copy is shared across the sysplex. Configuration of WebSphere Business Services Fabric for z/OS starts once the product has been made available in the UNIX System Services file system. 204 z/OS: WebSphere Business Process Management V6.2 Production Topologies A word about service levels: We experienced a significant amount of pain during the writing of this book because of incorrect and misleading service levels of the code in support of WebSphere Business Services Fabric for z/OS. We recommend that before you install the product, you ask your IBM representative to do a thorough search of current product databases to ensure you have the most current code levels. At the time of this writing, the most current level of software for WebSphere Business Services Fabric for z/OS was Fabric_PTF. In addition, a support PTF for WebSphere Process Server for z/OS is also required for proper operation of WebSphere Business Services Fabric for z/OS. This PTF number is UK42437. Additional service was expected for WebSphere Business Services Fabric for z/OS and WebSphere Process Server for z/OS, but the exact PTF numbers were not available before this book went to press. 5.1.4 Sample JCL and shell scripts used in this chapter The sample JCL and shell scripts described in this chapter can be found on the /zos_wbsf directory in the additional materials disk. The JCL is in the WBSF.CNTL library, the DDL is in the WBSF.SQL library, and the shell scripts in the /wpswork directory under /zos_wbsf. Problem diagnosis information can be located in Chapter 10, “Problem diagnosis and prevention” on page 571. 5.1.5 Creating the deployment directory Under the heading “Creating the Deployment Directory,” the WebSphere Business Services Fabric for z/OS Information Center recommends that you take copies of the product ZFS so that you can customize files such as the data definition language (DDL) statements and the JACL. However, after completing the configuration we found that it was not necessary to take a copy of the entire ZFS. It is sufficient to copy any files you need to change into a work directory where you can edit them and transfer them into the appropriate directories within the node’s file system. We will highlight those files as we progress through the configuration steps. Chapter 5. WebSphere Business Services Fabric for z/OS 205 5.2 Install WebSphere Business Services Fabric for z/OS in a standalone node This section describes the steps in installing WebSphere Business Services Fabric into a clustered environment. The initial steps are: 1. Copy the Fabric artifacts. See 5.2.1, “Copy Fabric artifacts” on page 207 2. Configure security. See 5.2.2, “Configure security” on page 209. 3. Configure databases. See 5.2.3, “Configure databases” on page 214. The majority of the actual configuration for WebSphere Business Services Fabric for z/OS occurs during the profile augmentation process. This process is unique for each node and the deployment manager. We describe this process for a standalone node in 5.2.4, “Augment WebSphere Process Server for z/OS standalone node profile” on page 220. We go through a series of verification steps to ensure the augmentation worked in the manner expected. We then start the server. Post augmentation steps are as follows: 1. Recreating the Fabric datasource. See “Delete fabric datasource.” on page 235. 2. Modifying the SI Bus created for Fabric. See “Modify fabric bus” on page 245. 3. EJBRole mapping. See “Map Fabric EJB roles to RACF EJB roles” on page 246. 4. Final verification. See 5.2.7, “Verify the WebSphere Business Services Fabric for z/OS installation and configuration” on page 252. 5. Product registration updates. See 5.2.8, “Create symlink for fabric product registration” on page 258. 206 z/OS: WebSphere Business Process Management V6.2 Production Topologies 5.2.1 Copy Fabric artifacts To install WebSphere Business Services Fabric for z/OS, several files and directories have to be copied to the WebSphere Process Server for z/OS nodes from the WebSphere Business Services Fabric for z/OS product ZFS. The WebSphere Business Services Fabric for z/OS Information Center lists the files to be copied. It is necessary to copy the registration file WBSF.registrar into each node’s properties/version directory. We discuss this process in 5.2.8, “Create symlink for fabric product registration” on page 258. We created a shell script called copyfabric.sh (Example 5-1) to do the necessary copies. The copy commands (cp) in Example 5-1 go on one line but some have been split across 2 lines to fit on the page. Line breaks are indicated by a ‘\’. Before you run the shell script, make sure you have a work directory defined and it you have R/W access to it. On our system our work files are located in the /var/WebSphere/home/MDCFG/wpswork directory. Refer to “Installing the sample shell scripts on a z/OS system” on page 623 for instructions on how to create and populate this directory from the Additional Materials available for this Redbooks publication. Example 5-1 Sample shell script copyfabric.sh #!/bin/sh # copyfabric.sh # # Copy fabric artifacts # #---------------------------------------------------------------------# Setup # # Set the following variables to match your environment # WORK=/tmp/workFabric FABRIC=/usr/lpp/webifyMP/V6R2M0 DMNODE=/wasv61config/mdcell/mddmnode/DeploymentManager NODEA=/wasv61config/mdcell/mdnodea/AppServer NODEB=/wasv61config/mdcell/mdnodeb/AppServer OWNER=MDADMIN GROUP=MDCFG # # Set the correct owner:group on the CHOWN statements at the bottom # # echo 'Copying files to work directory' Chapter 5. WebSphere Business Services Fabric for z/OS 207 cp -R $FABRIC/configuration/DB2Scripts/define_db.sql $WORK/. cp -R $FABRIC/configuration/DB2Scripts/define_tables.sql $WORK/. cp -R $FABRIC/configuration/script/Augment_Adv_Fabric_Profile.sh $WORK/. echo 'Copying templates for profile augmentation' echo 'Copying to '$DMNODE'/profileTemplates' cp -R $FABRIC/WBSFabric $DMNODE/profileTemplates/. cp -R $FABRIC/dmgr.wbsfabric $DMNODE/profileTemplates/. cp -R $FABRIC/managed.wbsfabric $DMNODE/profileTemplates/. echo 'Copying to '$NODEA'/profileTemplates' cp -R $FABRIC/WBSFabric $NODEA/profileTemplates/. cp -R $FABRIC/dmgr.wbsfabric $NODEA/profileTemplates/. cp -R $FABRIC/managed.wbsfabric $NODEA/profileTemplates/. echo 'Copying to '$NODEB'/profileTemplates' cp -R $FABRIC/WBSFabric $NODEB/profileTemplates/. cp -R $FABRIC/dmgr.wbsfabric $NODEB/profileTemplates/. cp -R $FABRIC/managed.wbsfabric $NODEB/profileTemplates/. echo 'Creating symbolic links' ln -s $FABRIC/configuration/Runtime/zos/WBSF.registrar \ $DMNODE/properties/version/WBSF.registrar ln -s $FABRIC/configuration/Runtime/zos/WBSF.registrar \ $NODEA/properties/version/WBSF.registrar ln -s $FABRIC/configuration/Runtime/zos/WBSF.registrar \ $NODEB/properties/version/WBSF.registrar echo 'Setting permissions' chmod -Rh 775 $WORK/* chown -Rh $OWNER:$GROUP $WORK/* chmod -Rh 775 $DMNODE/profileTemplates/* chown -Rh $OWNER:$GROUP $DMNODE/profileTemplates/* chmod -Rh 775 $NODEA/profileTemplates/* chown -Rh $OWNER:$GROUP $NODEA/profileTemplates/* chmod -Rh 775 $NODEB/profileTemplates/* chown -Rh $OWNER:$GROUP $NODEB/profileTemplates/* chmod -Rh 775 $DMNODE/properties/version/WBSF.registrar chown -Rh $OWNER:$GROUP $DMNODE/properties/version/WBSF.registrar chmod -Rh 775 $NODEA/properties/version/WBSF.registrar chown -Rh $OWNER:$GROUP $NODEA/properties/version/WBSF.registrar chmod -Rh 775 $NODEB/properties/version/WBSF.registrar chown -Rh $OWNER:$GROUP $NODEB/properties/version/WBSF.registrar You can invoke copyfabric.sh from the OMVS command line or run sample job MDCPYFAB, which invokes copyfabric.sh. 208 z/OS: WebSphere Business Process Management V6.2 Production Topologies Note: All of the JCL jobs, SQL, and sample shell scripts mentioned in this chapter and in chapters Chapter 6, “Incorporating WebSphere Business Services Fabric for z/OS into the cluster” on page 299 and Chapter 7, “Installing and testing WebSphere Business Services Fabric for z/OS” on page 403 are available in the Additional Materials shipped with this Redbooks publication. See Appendix A, “Additional material” on page 615 for details. The JCL jobs are in WBSF.CNTL. The SQL is in WBSF.SQL. The shell scripts are in zos_wpswork. 5.2.2 Configure security The WebSphere Business Services Fabric for z/OS Information Center describes how to use four different types of user registries. Standalone LDAP Local operating system Federated repositories Federated repositories with LDAP For our Fabric installation we will focus on using the local operating system option. On z/OS this means RACF. When using RACF as the user registry for the cell, consult the Information Center article Configuring WebSphere Process Server security to use local operating system, available at the following Web page: http://publib.boulder.ibm.com/infocenter/dmndhelp/v6r2mx/topic/com.ibm. ws.fabric.install.doc/fpiZOS/task/t_%20configuring_security.html Some RACF groups and user IDs are required for J2C authentication aliases. Plan some groups and user IDs to permit to various roles. We planned the groups and user IDs we would use with a spreadsheet. The spreadsheet is included in Appendix A, “Additional material” on page 615. Add RACF users and groups We created sample job MDRACF (Example 5-2) to define all the groups and user IDs we required for use with Fabric and MDHOME to define their home directories. Example 5-2 Sample job MDRACF to define user IDs and groups for Fabric //MDRACF JOB (0),'MDADMIN',CLASS=A,MSGCLASS=H,REGION=0M, // NOTIFY=&SYSUID /*JOBPARM SYSAFF=SC42 Chapter 5. WebSphere Business Services Fabric for z/OS 209 //***********************************************************/ //* Make sure you edit this file with CAPS OFF. */ //* Run this job using a userid with RACF SPECIAL. */ //* Change MD to your two character cell prefix. */ //* For example, MD. (Use upper case.) */ //* Set the OWNER as you want. */ //* Set the UIDs and GIDs so they are unique for this cell.*/ //* Run job MDHOME after this to create home directories. */ //***********************************************************/ //* //* Add groups and userids for roles related to Fabric //* //RACF EXEC PGM=IKJEFT01,DYNAMNBR=20 //SYSTSPRT DD SYSOUT=* //SYSUDUMP DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSTSIN DD * /* Group for FAB administrators */ AG MDFADMG SUPGROUP(MDCFG) OMVS(AUTOGID) + DATA('MD Fabric Admins') /* Fabric admin userid */ AU MDFABADM DFLTGRP(MDFADMG) OMVS(AUTOUID + HOME(/var/WebSphere/home/MDFADMG) + PROGRAM(/bin/sh)) + NAME('MD Fabric Admin') + OWNER(MDCFG) PW USER(MDFABADM) NOINTERVAL ALU MDFABADM PASSWORD(MDFABADM) NOEXPIRED /* Group for Fabric Governance admininstrators */ AG MDFGADMG SUPGROUP(MDCFG) OMVS(AUTOGID) + DATA('MD Fabric Governance Admins') /* Fabric governance admin userid */ AU MDFGADM DFLTGRP(MDFGADMG) OMVS(AUTOUID + HOME(/var/WebSphere/home/MDFGADMG) + PROGRAM(/bin/sh)) + NAME('MD Fab Gov Adm') + OWNER(MDCFG) PW USER(MDFGADM) NOINTERVAL ALU MDFGADM PASSWORD(MDFGADM) NOEXPIRED /* Group for Fabric Performance Users */ AG MDFPUG SUPGROUP(MDCFG) OMVS(AUTOGID) + DATA('MD FAB PERF USER') /* Fabric Performance User userid */ AU MDFPU DFLTGRP(MDFPUG) OMVS(AUTOUID + HOME(/var/WebSphere/home/MDFPUG) + 210 z/OS: WebSphere Business Process Management V6.2 Production Topologies PROGRAM(/bin/sh)) + NAME('MD Fabric Performance Users') + OWNER(MDCFG) PW USER(MDFPU) NOINTERVAL ALU MDFPU PASSWORD(MDFPU) NOEXPIRED /* Group for Fabric Subscriber Managers */ AG MDFSMG SUPGROUP(MDCFG) OMVS(AUTOGID) + DATA('MD Fabric Subscriber Managers') /* Fabric Subscriber Manager userid */ AU MDFSMU DFLTGRP(MDFSMG) OMVS(AUTOUID + HOME(/var/WebSphere/home/MDFSMG) + PROGRAM(/bin/sh)) + NAME('MD FAB SUB MGR') + OWNER(MDCFG) PW USER(MDFSMU) NOINTERVAL ALU MDFSMU PASSWORD(MDFSMU) NOEXPIRED /* Group for Fabric Basic Users */ AG MDFBUG SUPGROUP(MDCFG) OMVS(AUTOGID) + DATA('MD Fabic Basic Users') /* Fabric Basic User userid */ AU MDFBU DFLTGRP(MDFBUG) OMVS(AUTOUID + HOME(/var/WebSphere/home/MDFBUG) + PROGRAM(/bin/sh)) + NAME('MD FAB Base User') + OWNER(MDCFG) PW USER(MDFBU) NOINTERVAL ALU MDFBU PASSWORD(MDFBU) NOEXPIRED /* Group for Fabric Studio Users */ AG MDFSUG SUPGROUP(MDCFG) OMVS(AUTOGID) + DATA('MD Fabric Studio Users') /* Fabric Studio User userid */ AU MDFSU DFLTGRP(MDFSUG) OMVS(AUTOUID + HOME(/var/WebSphere/home/MDFSUG) + PROGRAM(/bin/sh)) + NAME('MD FAB Studio User') + OWNER(MDCFG) PW USER(MDFSU) NOINTERVAL ALU MDFSU PASSWORD(MDFSU) NOEXPIRED /* Permit the role groups to the APPL profile */ PE MD CLASS(APPL) ID(MDFADMG, MDFGADM, MDFPUG, MDFSMG) PE MD CLASS(APPL) ID(MDFBUG, MDFSUG) SETR RACLIST(APPL) REFRESH /* Fabric Bus Schema group */ AG MD01F SUPGROUP(MDCFG) OMVS(AUTOGID) + DATA('MD SIB01 FABRIC SCHEMA') Chapter 5. WebSphere Business Services Fabric for z/OS 211 /* /* /* /* Allow alias to switch to bus schema This connect is needed in DB2 V8.1 but not in DB2 9.1 CONNECT MDDBU GROUP(MD01F) */ */ */ */ /* Create home directories for the users We created sample job MDHOME to make home directories for the users that were created by job MDRACF. Add RACF EJBROLEs We created sample job MDEJBROL (Example 5-3) to define RACF EJBROLE profiles for the roles related to WebSphere Business Services Fabric. This job also permitted the necessary groups to use the roles. Example 5-3 Sample job MDEJBROL to define EJBROLE profiles for Fabric //MDEJBROL JOB (0),'RACFADM',CLASS=A,MSGCLASS=A,REGION=0M, // NOTIFY=&SYSUID //***********************************************************/ //* Make sure you edit this file with CAPS OFF. */ //* Run this job using a userid with RACF SPECIAL. */ //* The RACF userids and groups used here must exist. */ //* Run job XXRACF first. */ //* Change MD to your two character cell prefix. */ //* For example, MD. (Use upper case.) */ //* Set the OWNER as you want. */ //***********************************************************/ //* //* Create EJB role profiles for WPS //* //RACFDBA EXEC PGM=IKJEFT01,DYNAMNBR=20 //SYSTSPRT DD SYSOUT=* //SYSUDUMP DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSTSIN DD * /* Define FabricAdministrator role */ RDEFINE EJBROLE MD.FabricAdministrator + UACC(NONE) OWNER(MDCFG) DATA('MD FabricAdministrator role') PERMIT MD.FabricAdministrator CLASS(EJBROLE) RESET 212 z/OS: WebSphere Business Process Management V6.2 Production Topologies + + + + PERMIT MD.FabricAdministrator + CLASS(EJBROLE) + ID(MDFADMG) ACCESS(READ) /* Define FabricGovernanceAdministrator role */ RDEFINE EJBROLE MD.FabricGovernanceAdministrator + UACC(NONE) + OWNER(MDCFG) + DATA('MD FabricGovernanceAdministrator role') PERMIT MD.FabricGovernanceAdministrator + CLASS(EJBROLE) + RESET PERMIT MD.FabricGovernanceAdministrator + CLASS(EJBROLE) + ID(MDFADMG,MDFGADMG) + ACCESS(READ) /* Define FabricPerformanceUser role */ RDEFINE EJBROLE MD.FabricPerformanceUser + UACC(NONE) + OWNER(MDCFG) + DATA('MD FabricPerformanceUser role') PERMIT MD.FabricPerformanceUser + CLASS(EJBROLE) + RESET PERMIT MD.FabricPerformanceUser + CLASS(EJBROLE) + ID(MDFPUG,MDFADMG) + ACCESS(READ) /* Define FabricSubscriberManager role */ RDEFINE EJBROLE MD.FabricSubscriberManager + UACC(NONE) + OWNER(MDCFG) + DATA('MD FabricSubscriberManager role') PERMIT MD.FabricSubscriberManager + CLASS(EJBROLE) + RESET PERMIT MD.FabricSubscriberManager + CLASS(EJBROLE) + ID(MDFSMG,MDFADMG) + ACCESS(READ) /* Define FabricBasicUser role */ RDEFINE EJBROLE MD.FabricBasicUser + UACC(NONE) + OWNER(MDCFG) + DATA('MD FabricBasicUser role') PERMIT MD.FabricBasicUser + Chapter 5. WebSphere Business Services Fabric for z/OS 213 CLASS(EJBROLE) RESET PERMIT MD.FabricBasicUser CLASS(EJBROLE) ID(MDFBUG,MDFADMG) ACCESS(READ) /* Define FabricStudioUser role */ RDEFINE EJBROLE MD.FabricStudioUser UACC(NONE) OWNER(MDCFG) DATA('MD FabricStudioUser role') PERMIT MD.FabricStudioUser CLASS(EJBROLE) RESET PERMIT MD.FabricStudioUser CLASS(EJBROLE) ID(MDFSUG,MDFADMG) ACCESS(READ) SETR RACLIST(EJBROLE) REFRESH SEARCH MASK(MD) CLASS(EJBROLE) /* + + + + + + + + + + + + 5.2.3 Configure databases WebSphere Business Services Fabric for z/OS requires a database for its tables. You will probably want to create a second database to hold the tables related to the Fabric Service Integration Bus. The names we used for these two databases accessed by our MD cell were MDFABDB and MD01F respectively. Copy database DDLs The DDL is provided at /usr/lpp/webify/V6R1M2/configuration/DB2Scripts. The copyfabric.sh script copied the contents of this directory to our temporary work directory /tmp/workFabric. We created sample job MDCPYDDL to convert the DDL from ASCII to EBCDIC and copy it from /tmp/workFabric to z/OS sequential files so we could edit it. The source file names and the z/OS sequential dataset names are shown in Table 5-1. 214 z/OS: WebSphere Business Process Management V6.2 Production Topologies Table 5-1 Source DDL files and their equivalent EBCDIC z/OS files Description Source file name z/OS sequential dataset name Database, stogroup and table spaces define_db.sql WASCFG.MDCELL.SQL.MDFABDB Tables define_tables.sql WASCFG.MDCELL.SQL.MDFABTB Tailor database DDLs Before creating the Fabric databases, edit the DDL that creates the database, storage group, and table spaces to match your requirements (Source file define_db.sql and z/OS file WASCFG.MDCELL.SQL.MDFABDB). The following list details the changes we made to the MDFABDB file: Commented out the SET CURRENT SQLID at the start of all the DDL files because it is better to specify this in the JCL that are used to execute the DDL. Made the following changes: – Changed the STOGROUP name to our STOGRP, MDWPSSG. – VOLUMES from (WBIUS5,WBIUS4,WBIUS3) VCAT WSDB2 to our values. We commented out the CREATE STOGROUP statement because we were using a STOGROUP that already exists. Commented out the GRANT USE OF STOGROUP because we are using a STOGROUP that already exists, and the same schema as the one you are using for the other WebSphere Process Server databases. The grant will have already been issued in this case. Commented out the CREATE DATABASE statement because we created the database separately using job MDCREDB. Note: Leaving correct but commented-out CREATE DATABASE and CREATE STOGROUP statements in the DDL serves to act as documentation of which storage group is being used. This is better than deleting the statements entirely. Changed the database name from FABRICDB to MDFABDB. Changed the BUFFERPOOL assignment to the bufferpool planned (BP11) and added an INDEXBP BP12 statement to the CREATE DATABASE. Changed all PRIQTY and SECQTY statements to -1. Chapter 5. WebSphere Business Services Fabric for z/OS 215 Create fabric databases and tables 1. Create sample job MDCREDB (Example 5-4) to create the MDFABDB and MD01F databases, and to issue required GRANTs to the schema groups. Example 5-4 Sample job MDCREDB to create the Fabric databases //MDCREDB JOB (0),'MDADMIN',CLASS=A,REGION=0M,NOTIFY=&SYSUID, // USER=SENIOKJ,PASSWORD=SENIOKJ /*JOBPARM S=SC42 //***********************************************************/ //* Run this job using a userid with DB2 SYSADM authority */ //* or grant DBADM to the <sqlid> on SET CURRENT SQLID. */ //* Set the SYSAFF to the LPAR for this node. */ //* Change <xx> to your two character cell prefix. */ //* For example, MD. (Use upper case.) */ //* The DB2 system administrator should review all the */ //* names and bufferpools used. */ //* Change all the dataset names and plan name to match */ //* those in use with your DB2 system. */ //* Set the SYSIN to the dataset holding the DDL. */ //***********************************************************/ //JOBLIB DD DISP=SHR,DSN=DB9F9.SDSNEXIT // DD DISP=SHR,DSN=DB9F9.SDSNLOAD //* //* Before running this: //* The userid on the SET CURRENT SQLID statement must //* have DB2 SYSADM authority and EXECUTE on PLAN DSNTEP2 //* //* Create Fabric database and tablespaces. //* //CREATE EXEC PGM=IKJEFT01,DYNAMNBR=20 //SYSTSPRT DD SYSOUT=* //SYSUDUMP DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSTSIN DD * DSN SYSTEM(D9FG) RUN PROGRAM(DSNTEP2) PLAN(DSNTEP91) LIB('DB9FU.RUNLIB.LOAD') //SYSIN DD * CREATE DATABASE MDFABDB STOGROUP MDWPSSG BUFFERPOOL BP11 INDEXBP BP12 CCSID UNICODE; 216 z/OS: WebSphere Business Process Management V6.2 Production Topologies GRANT DBADM ON DATABASE MDFABDB TO MDWPS; CREATE DATABASE MDSIBDB STOGROUP MDWPSSG BUFFERPOOL BP3 INDEXBP BP4; GRANT DBADM ON DATABASE MDSIBDB /* TO MD01F; 2. Edit the DDL that creates the tables to match your requirements. (Source file define_tables.sql and z/OS file WASCFG.MDCELL.SQL.MDFABTB.) We changed the database name from FABRICDB to MDFABDB. 3. Review the other DDL files. We found that no further editing was required. 4. Copy the customized DDL from the variable-length sequential files to a fixed-block, LRECL 80 partitioned dataset so it could be executed by DSNTEP2. The partitioned dataset was called WASCFG.MDCELL.WBSF.SQL. 5. Create sample job MDDDLFAB (Example 5-5) in WASCFG.MDCELL.WBSF.CNTL to execute the DDL. Make sure that you execute the DDL that creates the table spaces first (MDFABDB) followed by the DDL that creates the tables (MDFABTB), and the other DDL that creates the indexes. Note the order of the concatenation in the SYSIN DD of job MDDDLFAB. Example 5-5 Sample job MDDDLFAB to execute the Fabric DDL //MDDDLFAB JOB (0),'MDADMIN',CLASS=A,REGION=0M,NOTIFY=&SYSUID, // USER=SENIOKJ,PASSWORD=SENIOKJ /*JOBPARM S=SC42 //***********************************************************/ //* Run this job using a userid with DB2 SYSADM authority */ //* or grant DBADM to the <sqlid> on SET CURRENT SQLID. */ //* Set the SYSAFF to the LPAR for this node. */ //* Change <xx> to your two character cell prefix. */ //* For example, MD. (Use upper case.) */ //* The DB2 system administrator should review all the */ //* names and bufferpools used. */ //* Change all the dataset names and plan name to match */ //* those in use with your DB2 system. */ //* Set the SYSIN to the dataset holding the DDL. */ //***********************************************************/ //JOBLIB DD DISP=SHR,DSN=DB9F9.SDSNEXIT // DD DISP=SHR,DSN=DB9F9.SDSNLOAD Chapter 5. WebSphere Business Services Fabric for z/OS 217 //* //* Before running this: //* The userid on the SET CURRENT SQLID statement must //* have DB2 SYSADM authority and EXECUTE on PLAN DSNTIA91 //* //* Create Fabric tables and indexes etc. //* //CREATE EXEC PGM=IKJEFT01,DYNAMNBR=20 //SYSTSPRT DD SYSOUT=* //SYSUDUMP DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSTSIN DD * DSN SYSTEM(D9FG) RUN PROGRAM(DSNTEP2) PLAN(DSNTEP91) LIB('DB9FU.RUNLIB.LOAD') //SYSIN DD * SET CURRENT SQLID = 'MDWPS' ; // DD DISP=SHR,DSN=WASCFG.MDCELL.WBSF.SQL(MDFABDB) // DD DISP=SHR,DSN=WASCFG.MDCELL.WBSF.SQL(MDFABTB) 6. If your DB2 for z/OS system is using RACF for authorization checking, and if you have connected the database J2C authentication alias user ID to the RACF group of the schema (MDWPS for our cell), then the database authentication alias will need no more authority because it has privileges equivalent to the owner of the tables. If you have not connected the database J2C authentication alias user ID to the schema’s RACF group because you do not want the authentication alias to have implicit owner privileges, issue GRANT statements for the Fabric tables and sequences. Create the necessary grant statements in member GRANTFAB in the WASCFG.MDCELL.WPS.SQL dataset. Then we created sample job MDGRANT in the .CNTL dataset to execute these statements. Execute MDGRANT now if you need to GRANT table and sequence privileges to the database J2C authentication alias user ID. Create message bus tables To create the tables for the Fabric Service Integration Bus, we copied the DDL we had used for one of the other buses and changed the schema to MD01F (where the “F” stands for Fabric). You could also run the sibDDLGenerator.sh to generate new DDL. We created sample job MDDDLSI (Example 5-6) to execute this DDL. Example 5-6 Sample job MDDDLSI to execute the Fabric bus DDL 218 z/OS: WebSphere Business Process Management V6.2 Production Topologies //MDDDLSI JOB (0),'MDADMIN',CLASS=A,REGION=0M,NOTIFY=GATES, // USER=SENIOKJ,PASSWORD=SENIOKJ /*JOBPARM S=SC42 //***********************************************************/ //* Run this job using a userid with DB2 SYSADM authority */ //* or grant DBADM to the <sqlid> on SET CURRENT SQLID. */ //* Set the SYSAFF to the LPAR for this node. */ //* Change <xx> to your two character cell prefix. */ //* For example, MD. (Use upper case.) */ //* The DB2 system administrator should review all the */ //* names and bufferpools used. */ //* Change all the dataset names and plan name to match */ //* those in use with your DB2 system. */ //* Set the SYSIN to the dataset holding the DDL. */ //***********************************************************/ //JOBLIB DD DISP=SHR,DSN=DB9F9.SDSNEXIT // DD DISP=SHR,DSN=DB9F9.SDSNLOAD //* //* Before running this: //* The userid on the SET CURRENT SQLID statement must //* have DB2 SYSADM authority and EXECUTE on PLAN DSNTEP2 //* //* Create database and tables for Fabric SIB //* //CREATE EXEC PGM=IKJEFT01,DYNAMNBR=20 //SYSTSPRT DD SYSOUT=* //SYSUDUMP DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSTSIN DD * DSN SYSTEM(D9FG) RUN PROGRAM(DSNTEP2) PLAN(DSNTEP91) LIB('DB9FU.RUNLIB.LOAD') //SYSIN DD * SET CURRENT SQLID = 'MD01F' ; // DD DISP=SHR,DSN=WASCFG.MDCELL.WBSF.SQL(MD01F) Note: There are GRANT statements for the SIB tables at the end of the MD01F DDL member, so it is not necessary to run a separate GRANT job for the SIB tables. Chapter 5. WebSphere Business Services Fabric for z/OS 219 5.2.4 Augment WebSphere Process Server for z/OS standalone node profile Previous versions of WebSphere Business Services Fabric for z/OS required many manual configuration steps to properly set up and configure a WebSphere Process Server for z/OS server for Fabric. Improvements for V.6.2 have removed most of these requirements. Most configuration for WebSphere Business Services Fabric for z/OS V.6.2 is accomplished through profile augmentation. Next, we describe how to augment an existing WebSphere Process Server for z/OS profile to contain WebSphere Business Services Fabric for z/OS functions. The augmentation process uses the manageprofiles.sh command shipped with base WebSphere Application Server for z/OS. Prerequisites for profile augmentation Before augmenting a profile, ensure you have completed the following steps: 1. Ensure you have fully configured your WebSphere Application Server for z/OS and WebSphere Business Services Fabric for z/OS standalone server environment. 2. Ensure you have completed the security requirements for your Fabric installation. See 5.2.2, “Configure security” on page 209. 3. Ensure you have created the required Fabric and message bus databases. See 5.2.3, “Configure databases” on page 214. 4. Ensure SMPE work for Fabric has been completed. 5. Ensure you ran the MDCPYFAB job or issued the copyfabric.sh script from a UNIX command shell to copy the Fabric profile templates to the proper locations in your WebSphere Application Server for z/OS configuration HFS. 6. Ensure you shut down the Servants associated with the profile you will augment. 7. Ensure you have enough space in the target HFS and in /tmp to complete the augmentation process. Note: The Fabric applications will require a lot more space in your ZFS file systems so they may need to extend during the deployment process. After installing Fabric, our deployment manager’s node ZFS occupied 1800 CYLs while each node occupied more than 2000 CYLs. To ensure a clean deployment, ensure that the volumes that host your cell’s ZFS datasets have enough free space to allow them to obtain secondary extents and grow to the sizes mentioned above. 220 z/OS: WebSphere Business Process Management V6.2 Production Topologies Using the manageprofiles.sh command The manageprofiles.sh command is located in the /WPS_HOME/bin directory. On our system, the location for our WebSphere standalone node is /wasv61config/mdcell/mdnodea/AppServer/bin. The command used to augment the profile for WebSphere Business Services Fabric for z/OS uses similar parameters as in the base WebSphere Process Server installation as well as some additional Fabric specific parameters. Options for the manageprofiles.sh command are explained in Table 5-2 on page 221. All parameters are case sensitive. Table 5-2 manageprofiles command options Parameters Description -augment Use the augment parameter to make changes to an existing profile with an augmentation template. The augment parameter causes the manageprofiles command to update or augment the profile identified in the -templatePath parameter. The augmentation templates that you can use are determined by which IBM products and versions are installed in your environment. Note: Do not manually modify the files that are located in the /wasv61config/mdcell/mdnodea/AppServer/profileTemplates directory. Specify the fully qualified path for -templatePath. -templatePath Specifies the directory path to the template files in the installation root directory. Within the profileTemplates directory are various directories that correspond to different profile types and vary with the type of product installed. The profile directories are the paths that you indicate when using the -templatePath option. You can specify profile templates that lie outside the installation root if you have any. Use absolute paths. This parameter must exist as a directory and point to a valid template directory. Use the following templates with WebSphere Business Services Fabric for z/OS: WBSFabric: For a WebSphere Business Services Fabric for z/OS stand-alone server profile, which defines a stand-alone server. dmgr.wbsfabric: For a WebSphere Business Services Fabric for z/OS deployment manager profile, which defines a deployment manager node. managed.wbsfabric: For a WebSphere Business Services Fabric for z/OS managed node profile, which defines a managed node. -profileName Specifies the name of the profile. Use the value Default, which is the only valid value for WebSphere Application Server for z/OS. Chapter 5. WebSphere Business Services Fabric for z/OS 221 Parameters Description -cellName Specifies the cell name for the cell to be augmented. -nodeName Specifies the node name for the node to be augmented. -serverName Specifies the server name of the server to be augmented. -configureRESTSecurity Configure security for the Fabric Rest Services gateway. Default is True. -adminUserName Specifies the user ID that is used for administrative security. When augmenting a profile that has administrative security enabled, this parameter is required. -adminPassword Specifies the password that is used for administrative security. When augmenting a profile that has administrative security enabled, this parameter is required. -fabricDbCreateNew Specifies if the user wants to create the Fabric database automatically. For z/OS this parameter must always be set to false, because DB2 for z/OS databases are created through JCL by the z/OS database administrator. -fabricDbType Specifies the Fabric database type. Valid value for z/OS is DB2_UNIVERSAL. -fabricDbDriverType Specifies the driver type for the database. Valid values are 2 and 4. Note: We specify type 2 for the driver type. Due to a bug in the way the Fabric datasource gets created, we must recreate it properly post augmentation. See “Rebuild fabric datasource using non-XA DB2 UDB JCC driver” on page 237. We could have avoided this problem had we chosen to use a type 4 driver. Because this is a DB2 z/OS implementation, we chose to use type 2 and manually correct the problem. -fabricDbName Specifies the name of the Fabric database. For a z/OS database, this value is always set to the DB2 subsystem location name. -fabricDbUser Specifies the user name that is used to access the DB2 z/OS subsystem. -fabricDbPassword Specifies the password that is used to access the DB2 z/OS subsystem -fabricDbServerPort Specifies the port where the TCP/IP service is assigned. For a DB2 z/OS database, this is the DB2 DDF listener port. -fabricDbHostName Specifies the database server host name where the TCP/IP service is assigned. For a DB2 z/OS database, this is the name of the z/OS host where the DB2 DDF server is running. -fabricDbJDBCClasspath The location of the DB2 z/OS JDBC jar file. 222 z/OS: WebSphere Business Process Management V6.2 Production Topologies Parameters Description -fabricDbCommonForME Specifies the message store to be used for the Fabric message bus. Note: Coding server name and port numbers: For a type 2 datasource connecting to DB2 z/OS you still have to code the server name and port numbers of the DB2 z/OS Distributed Data Facility server (DDF), even though these are normally required by a type 4 datasource. These values do not have to be correct but they cannot be blank. We coded the correct values for our DB2 DDF server to act as documentation. Find the host name and port that DDF is listening on by looking at the initialization messages of the DB2 MSTR address space. On our system we saw these messages in the joblog of our D9F1MSTR address space: DSNL004I -D9F1 DDF START COMPLETE 768 LOCATION DB9F LU USIBMSC.SCPD9F1 GENERICLU -NONE DOMAIN wtsc42.itso.ibm.com TCPPORT 37893 SECPORT 0 RESPORT 37894 IPNAME -NONE Issue the manageprofiles.sh command We created sample job MDAUGFAB (Example 5-7) to execute the manageprofiles command through the BPXBATCH interface. We chose to use JCL rather than execute the command from a UNIX command shell so we could capture the output of the command in the joblog. Note: The values we specified on the manageprofiles.sh command are specific for our environment. Make sure you change the parameters to reflect your WebSphere Application Server for z/OS and WebSphere Process Server for z/OS installation. Example 5-7 Sample job MDAUGFAB to execute the manageprofiles.sh command //MDAUGFAB JOB (0),CLASS=A,MSGCLASS=A,REGION=0M, // USER=MDADMIN,PASSWORD=MDADMIN, // NOTIFY=GATES /*JOBPARM SYSAFF=SC42 //***********************************************************/ Chapter 5. WebSphere Business Services Fabric for z/OS 223 //* Configure Fabric in standalone server mdnodea */ //***********************************************************/ //INSTO EXEC PGM=IKJEFT01,REGION=0M,TIME=1440 //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BPXBATCH SH + export LIBPATH=/usr/lpp/db2/d9fg/db2910_jdbc/lib:$LIBPATH; + cd /wasv61config/mdcell/mdnodea/AppServer/bin; + ./manageprofiles.sh + -augment + -templatePath /wasv61config/mdcell/mdnodea/AppServer+ /profileTemplates/WBSFabric + -profileName default + -cellName mdbasea + -nodeName mdnodea + -serverName mdsr01a_WPS + -configureRESTSecurity true + -adminUserName mdadmin + -adminPassword mdadmin + -fabricDbCreateNew false + -fabricDbType DB2_UNIVERSAL + -fabricDbDriverType 2 + -fabricDbName DB9F + -fabricDbUser MDDBU + -fabricDbPassword MDDBU + -fabricDbServerPort 37893 + -fabricDbHostName wtsc42.itso.ibm.com + -fabricDbJDBCClasspath /usr/lpp/db2/d9fg/db2910_jdbc/classes + 1> /tmp/mdnodeaFabricConfig_0A11.out + 2> /tmp/mdnodeaFabricConfig_0A11.err; /* //***********************************************************/ //* STEPS TO COPY THE OUTPUT THE JOB LOG */ //***********************************************************/ //CPOUT EXEC PGM=IKJEFT01,REGION=0M //SYSEXEC DD DISP=SHR,DSN=BBO6142.MDNODEA.SBBOEXEC //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BBOHFSWR '/tmp/mdnodeaFabricConfig_0A11.out' BBOHFSWR '/tmp/mdnodeaFabricConfig_0A11.err' /* // 224 z/OS: WebSphere Business Process Management V6.2 Production Topologies After you run the MDAUGFAB job, check the joblog for the job. You should see the following message in the output: INSTCONSUCCESS: Profile augmentation succeeded. Ensure the augmentation process did what it was supposed to by following 5.2.5, “Verify the augmentation process succeeded” on page 225. 5.2.5 Verify the augmentation process succeeded The profile augmentation process creates several artifacts with the WebSphere Application Server for z/OS stand-alone server profile. We can verify what was created by using the WebSphere Application Server for z/OS administrative console. Start the stand-alone server Using the MVS console, or SDSF TSO windows, issue the start command for the node agent. On our system, the form of the command was as follows: START MDACRA,JOBNAME=MDSR1A,ENV=MDBASEA.MDNODEA.MDSR1A Make sure the four address spaces start up. Example 5-8 is what they look like in the SDSF DA window for our standalone application server MDS1A. Example 5-8 SDSF DA window for a standalone server SDSF DA SC42 (ALL) COMMAND INPUT ===> PREFIX=MD* DEST=(ALL) NP JOBNAME JobID MDDEMNA STC25300 MDSR1A STC25296 MDSR1AA STC25304 MDSR1AS STC25301 PAG 0 CPU/L/Z 4/ 4/ 0 LINE 1-4 SC OWNER=* SORT=SysName/A JOBNAME/A SYSNA SysN CPU% Real StepName ProcStep Owner SC42 0.00 8801 MDDEMNA BBODAEMN MDACRU SC42 0.03 65T MDSR1A BBOCTL MDACRU SC42 0.12 105T MDSR1AA BBOSR MDASRU SC42 0.12 146T MDSR1AS BBOSR MDASRU Review any error messages and repair When the server finishes initialization, look in the server’s log files for any error messages, and examine the Integrated Solutions Console for the settings of various components. This is quite easy if you know what you are looking for. Examine the server logs on JES spool Using SDSF Display Active window, select your application server’s started tasks with the PREfix or Select MD* commands and look in the control region (MDSR1A), control region adjunct (MDSR1AA), and servant regions (MDSR1S). Chapter 5. WebSphere Business Services Fabric for z/OS 225 Select each with the S action character, and look for any sever error messages with the FIND SEVERE ALL command. Hopefully, there will be none. If there are, note the number displayed by SDSF, and step through each one and start your problem determination process. Back up the stand-alone server’s ZFS data set At each major configuration milestone, and before proceeding, we find it a good idea to back up our configuration HFS. We used the JCL shown in Example 5-9 to perform the back up. Example 5-9 JCL to back up the stand-alone server node //MDDUMP JOB NOTIFY=&SYSUID,CLASS=A,MSGCLASS=A, // REGION=0M,TIME=1440 //* //* DUMP CONFIG HFS FOR STANDALONE NODE A //* //DUMP EXEC PGM=ADRDSSU //DASD DD DISP=(NEW,CATLG),DSN=WASCFG.MDCELL.DUMP.POSTAUG, // UNIT=3390,SPACE=(CYL,(500,100),RLSE) //SYSPRINT DD SYSOUT=* //SYSIN DD * DUMP DATASET( INCLUDE( WASCFG.MDCELL.MDNODEA.CONFIG.ZFS ) ) OUTDD(DASD) TOL(ENQF) OPTIMIZE(4) COMPRESS /* 226 z/OS: WebSphere Business Process Management V6.2 Production Topologies Login to administrative console Next, we must login to the admin server for the stand-alone server. See Figure 5-2. Figure 5-2 Login to stand-alone server admin console Next, verify that the Fabric resources were created properly by the profile augmentation process. Verify Fabric datasource was created Verify the Fabric datasource was created. A fabric_pm datasource will be created at the server level. This datasource will be connected to the Fabric database in the DB2 z/OS subsystem. The Fabric database is used by the Fabric performance manager. Chapter 5. WebSphere Business Services Fabric for z/OS 227 To verify this datasource was created, perform the following steps: 1. Navigate to Resources → JDBC in the Integrated Solutions Console. 2. Click Data sources. 3. Click fabric pm. See Figure 5-3. Figure 5-3 Fabric datasource on stand-alone server Verify the Fabric JAAS-J2C authentication alias was created A FABRIC_JDBC_AUTH J2C alias is created based on the user ID and password used to access the DB2 z/OS database defined for Fabric’s use. It is connected to the fabric_pm datasource. See Figure 5-4. Figure 5-4 Fabric datasource definition 228 z/OS: WebSphere Business Process Management V6.2 Production Topologies 1. On the fabric pm datasource window verify that the component-managed authentication alias is set to FABRIC JDBC AUTH. 2. Click JAAS - J2C authentication data to see the definition of the Fabric authentication alias, as shown in Figure 5-5. Figure 5-5 List of JAAS-J2C authentication aliases 3. Click FABRIC JDBC AUTH. See Figure 5-6 for the definition of the Fabric JAAS-J2C alias. Figure 5-6 Fabric JAAS-J2C authentication alias for stand-alone server Chapter 5. WebSphere Business Services Fabric for z/OS 229 Verify the Fabric JDBC path variable was set The JDBC provider specified in the manageprofiles.sh command is listed as a JDBC driver for the server being augmented. This JDBC provider is present on z/OS servers by default and is not created as a result of the profile augmentation. To verify the JDBC path variable, perform the following steps: 1. Navigate to Resources → JDBC in the Integrated Solutions Console. 2. Click JDBC Providers. A list of JDBC Providers is displayed. The datasource we specified on the manageprofiles.sh command, DB2 Universal JDBC Driver Provider, is in the list. See Figure 5-7 Figure 5-7 JDBC providers on the stand-alone server 230 z/OS: WebSphere Business Process Management V6.2 Production Topologies Verify the Fabric .jar files were copied The following two .jar files should be copied to the plugins directory of the WebSphere Process Server for z/OS stand-alone server (/wasv61config/mdcell/mdnodea/AppServer/plugins): com.ibm.ws.repository_6.2xxx.jar com.ibm.ws.fabric.catalog_6.2.0.jar See Figure 5-8. Figure 5-8 Fabric jar files in plugins directory of stand-alone server Chapter 5. WebSphere Business Services Fabric for z/OS 231 Verify the Fabric SCA jars were copied Verify the following jars were copied to the lib/ext directory of the WebSphere Process Server for z/OS stand-alone server (/wasv61config/mdcell/mdnodea/AppServer/lib/ext): fabric-da-api.jar fabric-da-model.jar fabric-da-sca.jar fabric-da-scdl.jar fabric-types.jar See Figure 5-9. Figure 5-9 Fabric SCA jars in the /lib/ext directory of the stand-alone server 232 z/OS: WebSphere Business Process Management V6.2 Production Topologies Verify the Fabric applications were deployed The following Fabric applications should be deployed on the stand-alone server. Fabric_Catalog Fabric_Engine Fabric_REST_Services Fabric_Tools Fabric_Tools_Help To verify these were deployed, perform the following steps: 1. Navigate to Applications → Enterprise Applications in the Integrated Solutions Console. 2. Click Enterprise Applications. A list of deployed applications is displayed. See Figure 5-10 on page 234. Chapter 5. WebSphere Business Services Fabric for z/OS 233 Figure 5-10 Fabric applications deployed on the stand-alone server 234 z/OS: WebSphere Business Process Management V6.2 Production Topologies 5.2.6 Post augmentation tasks There are several tasks that must be completed after profile augmentation to complete the configuration of Fabric on the stand-alone server. These steps are outlined below. Delete fabric datasource. For DB2 z/OS databases we have two choices when defining the JDBC datasources. Type 2 This JDBC driver takes advantage of z/OS cross memory services to provide a high speed connector between the Java process and the DB2 main address space where SQL command will be executed. This driver does not use network protocols and is generally the most efficient in terms of resources consumed. Both the database subsystem and the Java process (in this case the WebSphere Application Server for z/OS server) must reside in the same LPAR to make a type 2 connection. Type 4 This JDBC driver uses a gateway function to provide remote access to DB2 z/OS resources. Type 4 connections use a distributed DRDA® protocol to complete their connection. This connection type is more expensive in terms of resources and end to end throughput. It should only be used when the database subsystem and Java process are not in the same LPAR. Our implementation uses DB2 shared data and a DB2 subsystem on every LPAR. For this reason we chose to use a type 2 connection when augmenting our WebSphere Application Server for z/OS stand-alone server profile. See “Issue the manageprofiles.sh command” on page 223. We experienced a defect in the way the type 2 JDBC driver was created by the manageprofiles.sh command. As implemented, it uses the XA version of the type 2 driver. We verified this by performing the following steps: 1. Navigate to Resources → JDBC in the Integrated Solutions Console. 2. Click Data sources. Chapter 5. WebSphere Business Services Fabric for z/OS 235 3. Click fabric pm. In Figure 5-11 below, we can see the incorrect JDBC driver specified in the Provider field. Figure 5-11 Incorrect JDBC driver provider for fabric pm database This driver is implemented on distributed systems, but not on z/OS. For this reason, we must drop this datasource and recreate it using the z/OS version of the type 2 driver. First we deleted the fabric_pm datasource created by the manageprofiles.sh command. 1. 2. 3. 4. 5. 236 Navigate to Resources → JDBC in the Integrated Solutions Console. Click Data sources. Select the fabric pm check box. Click Delete. Click Save. z/OS: WebSphere Business Process Management V6.2 Production Topologies See Figure 5-12. Figure 5-12 Delete incorrect fabric pm datasource Rebuild fabric datasource using non-XA DB2 UDB JCC driver We will now rebuild the fabric pm datasource using the correct JDBC type driver. 1. In the Integrated Solutions Console, navigate to Resources → JDBC. 2. Click Data sources. Note: The scope is often set to All scopes. Select a scope before you click New. Otherwise you will receive the following message: Select a valid scope from drop down list. "All scopes" is not a valid scope for this operation. Chapter 5. WebSphere Business Services Fabric for z/OS 237 3. Click New. See Figure 5-13. Figure 5-13 Create a new fabric pm datasource 238 z/OS: WebSphere Business Process Management V6.2 Production Topologies 4. Fill in the following parameters on “Step 1: Enter basic data source information” window (Figure 5-14): – scope: accept default value, cells:mdbasea:nodes:mdnodea:servers:mdsr01a_WPS – Data source name: fabric pm – JNDI name: jdbc/fabric/pm – Component-managed authentication alias and XA recovery authentication alias: Select FABRIC_JDBC_AUTH from the drop-down menu Click Next. Figure 5-14 Enter fabric pm datasource information Chapter 5. WebSphere Business Services Fabric for z/OS 239 5. On the “Step 2: Select JDBC provider” window (Figure 5-15) perform the following steps: a. Select the Select an existing JDBC provider check box. b. Select DB2 Universal JDBC driver provider from the drop-down menu. Click Next. Figure 5-15 Specify JDBC provider for fabric pm database 240 z/OS: WebSphere Business Process Management V6.2 Production Topologies 6. On the “Step 3: Enter database specific properties for the data source” window (Figure 5-16), specify the following parameters: – – – – Database name: DB2 location name (DB9F in our case.) Driver type: Select 2 from the drop-down menu. Server name: wtsc42.itso.ibm.com Port number: 37893 Clear the Use this data source in container managed persistence (CMP) check box and click Next. Figure 5-16 Specify database parameters of fabric pm datasource Chapter 5. WebSphere Business Services Fabric for z/OS 241 7. Verify fabric pm datasource parameters in the “Step 4: Summary” window (Figure 5-17). Figure 5-17 Verify new fabric pm datasource parameters 8. Click Finish to create the data source. 9. Click Save. 242 z/OS: WebSphere Business Process Management V6.2 Production Topologies 10.Verify the new fabric pm datasource is using the correct JDBC type 2 driver by performing the following steps a. In the Integrated Solutions Console, navigate to Resources → JDBC. b. Click Data sources. c. Click fabric pm. d. Verify the Provider field is set to DB2 Universal JDBC Driver Provider. See Figure 5-18. Figure 5-18 Verify new fabric pm datasource JDBC provider Chapter 5. WebSphere Business Services Fabric for z/OS 243 Update SQLID for fabric datasource Datasource creation does not support the specification of a schema ID. Because we created our fabric database with a schema, we must set the currentSschema for the fabric pm datasource to match this SQLID. To set the currentSchema perform the following steps: 1. In the Integrated Solutions Console, navigate to Resources → JDBC. 2. Click Data sources. 3. Click fabric pm. 4. Click Custom properties. 5. Click the currentSchema parameter. 6. In the Value box, specify your SQLID. We used MDWPS. See Figure 5-19. Figure 5-19 Specify currentSQLID for fabric pm datasource 7. Click Apply. 8. Click Save to save your configuration changes. 244 z/OS: WebSphere Business Process Management V6.2 Production Topologies 9. Verify your changes in the fabric pm datasource custom properties window (Figure 5-20). Figure 5-20 Verify currentSQLID setting for fabric pm datasource Modify fabric bus We created our fabric bus using the same parameters we used to create the system buses for our WebSphere Process Server for z/OS installation. We chose to put each of our buses in its own database with its own schema. We must therefore change the schema ID in the fabric bus definition. We must also clear Create Tables check box as DB2 z/OS database tables are created through JCL by the database administrator. To make these changes, perform the following steps: 1. In the Integrated Solutions Console, navigate to Servers → Application servers. 2. Click mdsr01A_WPS. 3. Click Messaging engines. 4. Click the Fabric bus, mdnodea.mdsr01a_WPS-Fabric.mdbasea.Bus. 5. Click Message store. 6. Change schema name to MD01F, the name of our backing store for the Fabric bus. 7. Clear the Create tables check box. DB2 z/OS tables are created through JCL by the database administrator. The final configuration should look like Figure 5-21 on page 246. Chapter 5. WebSphere Business Services Fabric for z/OS 245 Figure 5-21 Create fabric ME with correct SQLID and clear the Create Tables check box. Map Fabric EJB roles to RACF EJB roles When using a LocalOS, SAF-based user registry such as RACF with WebSphere Process Server, the role-based authorization checks are normally performed by SAF-based user registry. When using RACF, roles are defined in RACF profiles in class EJBROLE and user ID and groups are permitted access to these profiles. When using a non-SAF user registry, the Integrated Solutions Console is used to permit users and groups to roles. For example, only users with FabricAdminstrator role for the Fabric_Tool can access the Fabric console. For our cell called MD we created a RACF EJBROLE profile called MD.FabricAdminstrator and a RACF group called MDFADMG was permitted access to this profile. We created a user called MDFABADM, which has MDFABADG as its DFLTGRP. Therefore, user ID MDFABADM had the FabricAdminstrator role. See the jobs MDRACF (Example 5-2 on page 209) and MDEJBROL (Example 5-3 on page 212) for details of this setup for all the roles. To map the EJBROLEs defined in RACF to the roles specified for Fabric, it is necessary to use the Integrated Solutions Console. In order to map the users, groups defined in RACF to the roles defined for Fabric, perform the following steps: 1. In the Integrated Solutions Console, navigate to Applications → Enterprise Applications. 246 z/OS: WebSphere Business Process Management V6.2 Production Topologies 2. Choose Fabric Tools. 3. Click Security role to user/group mapping. 4. On the window shown in Figure 5-22, select the check boxes next to each of the listed roles, and click Look up users. 5. All of our defined EJBROLEs in RACF start with MD. In the Search string box change the search string to MD*. Click Search. A list of available user IDs will be displayed. 6. Choose the ID from the list and use the >> box to move the ID to the selected column. Click OK. Figure 5-22 EJB Role to user mapping Chapter 5. WebSphere Business Services Fabric for z/OS 247 7. Repeat the above steps for each of the remaining EJB roles. The completed mappings should look like Figure 5-23 Figure 5-23 Fabric EJB role mappings Update xml file for Business Spaces As part of the configuration for Business Space powered by WebSphere, you must specify endpoints for all of the configured services. The configuration for WebSphere Business Services Fabric for z/OS is contained in the wbsfEndpoints XML file contained in the profile template used to configure Fabric. On our system, after profile augmentation for the stand-alone server, this file is located in /wasv61config/mdcell/mdnodea/AppServer/profiles/default/BusinessSpace/regis tryData/wbsfEndpoints.xml. Modify this file to contain the URL of the server where WebSphere Business Services Fabric for z/OS is deployed. Refer to Example 5-10 on page 249. Note this is an ASCII file and must be edited through an ASCII editor. (See“Dealing with ASCII files” on page 581.) 248 z/OS: WebSphere Business Process Management V6.2 Production Topologies Example 5-10 wbsfEndpoints XML file for Business Space powered by WebSphere configuration <?xml version="1.0" encoding="UTF-8" ?> - <!-START NON-TRANSLATABLE --> - <tns:BusinessSpaceRegistry xmlns:tns="http://com.ibm.bspace/BusinessSpaceRegistry" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://com.ibm.bspace/BusinessSpaceRegistry BusinessSpaceRegistry.xsd"> - <tns:Endpoint> <tns:id>{com.ibm.bspace}bspaceBusinessVariableServiceRootId</tns:id> <tns:version>1.0.0.0</tns:version> <tns:url>https://wtsc42.itso.ibm.com:20049/fabricrest</tns:url> <tns:description>Location of Business Space Business Policy server</tns:description> </tns:Endpoint> </tns:BusinessSpaceRegistry> An alternative for editing ASCII files In z/OS 1.8 ISPF added direct support for editing ASCII based files. In Figure 5-24 on page 250 below, we show how to edit the wbsfEndpoints.xml file from ISPF. 1. Simply navigate to the directory where the file to be edited resides. In this case: /wasv61config/mdcell/mdnodea/AppServer/profiles/default/BusinessSpace/re gistryData/ directory 2. Edit the file with the ea command. Chapter 5. WebSphere Business Services Fabric for z/OS 249 Figure 5-24 Editing an ASCII file with ISPF 3.17 Stop the stand-alone server The server must be stopped and restarted to pick up the changes we have made. To stop the stand-alone server issue the P MDSR1A command. An alternative to the traditional Stop command is shown in Example 5-11 on page 251. 1. From the SDSF DA window, issue the prefix MD* command to narrow down the selection list. 2. Select the control region (MDSR1A below) with a Y. Issuing a Y from the SDSF DA display effectively stops the control region and all associated servants. 250 z/OS: WebSphere Business Process Management V6.2 Production Topologies Example 5-11 Issue the Y command on the SDSF DA window SDSF DA SC42 (ALL) PAG 0 CPU/L/Z 4/ 4/ 0 LINE 1-4 COMMAND INPUT ===> SC PREFIX=MD* DEST=(ALL) OWNER=* SORT=SysName/A JOBNAME/A SYSNA NP JOBNAME JobID SysN CPU% Real StepName ProcStep Owner MDDEMNA STC25300 SC42 0.00 8801 MDDEMNA BBODAEMN MDACRU Y MDSR1A STC25296 SC42 0.03 65T MDSR1A BBOCTL MDACRU MDSR1AA STC25304 SC42 0.12 105T MDSR1AA BBOSR MDASRU MDSR1AS STC25301 SC42 0.12 146T MDSR1AS BBOSR MDASRU Start the stand-alone server Using the MVS console, or SDSF TSO windows, issue the start command for the stand-alone server. On our system, the form of the command was as follows: START MDACRA,JOBNAME=MDSR1A,ENV=MDBASEA.MDNODEA.MDSR1A Make sure the four address spaces start up. See Example 5-12. Example 5-12 SDSF DA window for a standalone server SDSF DA SC42 (ALL) COMMAND INPUT ===> PREFIX=MD* DEST=(ALL) NP JOBNAME JobID MDDEMNA STC25300 MDSR1A STC25296 MDSR1AA STC25304 MDSR1AS STC25301 PAG 0 CPU/L/Z 4/ 4/ 0 LINE 1-4 SC OWNER=* SORT=SysName/A JOBNAME/A SYSNA SysN CPU% Real StepName ProcStep Owner SC42 0.00 8801 MDDEMNA BBODAEMN MDACRU SC42 0.03 65T MDSR1A BBOCTL MDACRU SC42 0.12 105T MDSR1AA BBOSR MDASRU SC42 0.12 146T MDSR1AS BBOSR MDASRU Review any error messages and repair When the server finishes initialization, look in the server’s log files for any error messages, and examine the Integrated Solutions Console for the settings of various components. This is quite easy if you know what you are looking for. Examine the server logs on JES spool Using the SDSF display active window, select your application server’s started tasks with the PREfix or Select MD* commands and look in the control region (MDSR1A), control region adjunct (MDSR1AA), and servant regions (MDSR1AS). Select each one of them with the S action character, and look for any sever error messages with the FIND SEVERE ALL command. Hopefully, there will be none. If there are, note the number displayed by SDSF, and step through each one and start your problem determination process. Chapter 5. WebSphere Business Services Fabric for z/OS 251 We found it useful to review any Java exceptions in the servant region joblogs. These exceptions do not always get flagged as severe errors, but that does not mean they are not important. Back up the stand-alone server’s ZFS data set At each major configuration milestone, and before proceeding, we backed up our configuration HFS. We used the JCL in Example 5-9 on page 226 to perform the back up. Example 5-13 JCL to back up the stand-alone server node //MDDUMP JOB NOTIFY=&SYSUID,CLASS=A,MSGCLASS=A, // REGION=0M,TIME=1440 //* //* DUMP CONFIG HFS FOR DMGR AND NODE A //* //DUMP EXEC PGM=ADRDSSU //DASD DD DISP=(NEW,CATLG),DSN=WASCFG.MDCELL.DUMP.POSTFIX, // UNIT=3390,SPACE=(CYL,(500,100),RLSE) //SYSPRINT DD SYSOUT=* //SYSIN DD * DUMP DATASET( INCLUDE( WASCFG.MDCELL.MDDMNODE.CONFIG.ZFS ) ) OUTDD(DASD) TOL(ENQF) OPTIMIZE(4) COMPRESS /* 5.2.7 Verify the WebSphere Business Services Fabric for z/OS installation and configuration This section describes the steps to verify the installation and configuration of WebSphere Business Services Fabric. 1. Log in to the Integrated Solutions Console. 2. Navigate to Applications → Enterprise applications. 3. Click Enterprise applications. 252 z/OS: WebSphere Business Process Management V6.2 Production Topologies 4. Verify the Fabric applications are started, as shown in Figure 5-25. Figure 5-25 Fabric application .ears deployed on stand-alone server Chapter 5. WebSphere Business Services Fabric for z/OS 253 5. In the Integrated Solutions Console, perform the following steps: a. Navigate to Service integration → Buses. b. Click Fabric.mdbasea.Bus. c. Click Messaging engines. d. The mdnodea.mdsr01a_WPS-Fabric.mdbasea.Bus messaging engine will be up and running, as shown in Figure 5-26. Figure 5-26 mdnodea.mdsr01a_WPS-Fabric.mdbasea.Bus messaging engine status 254 z/OS: WebSphere Business Process Management V6.2 Production Topologies Log in to fabric console Type the following URL in the browser: http://wtsc42.itso.ibm.com:20048/fabric. You will see the login window (Figure 5-27). Log in with MDBSPADM, the user ID you added to RACF as the Fabric administrator user ID. Figure 5-27 WebSphere Business Services Fabric for z/OS login window Chapter 5. WebSphere Business Services Fabric for z/OS 255 If the user ID does not have access to the required role, you will receive the error shown in Figure 5-28. Figure 5-28 WebSphere Business Services Fabric for z/OS invalid login When the WebSphere Process Server cell is using a RACF user registry, go to the z/OS SYSLOG, where you see a related ICH408I error as shown in Example 5-14. Example 5-14 RACF ICH408I error when unauthorized to Fabric role(s) ICH408I USER(MDFABADM) GROUP(MDFADMG ) NAME(MD FABRIC ADMIN ) MD.FabricStudioUser CL(EJBROLE ) INSUFFICIENT ACCESS AUTHORITY ACCESS INTENT(READ ) ACCESS ALLOWED(NONE ) +BBOO0222I: SECJ0129E: Authorization failed for MDFABADM while invoking GET on default_host:fabric/app, Authorization failed, Not granted any of the required roles: FabricAdministrator FabricGovernanceAdministrator FabricPerformanceUser FabricSubscriberManager FabricBasicUser FabricStudioUser 256 z/OS: WebSphere Business Process Management V6.2 Production Topologies To resolve this error, permit the user ID to one of the EJBROLE profiles. In this case the user ID MDFABADM, which we wanted to be the Fabric administrator user ID, had not been granted to any of the roles. Refer to “Map Fabric EJB roles to RACF EJB roles” on page 246 for details on how to set up the role to user ID relationships. The first time you log in, you are prompted to register your name and email address, as shown in Figure 5-29. Figure 5-29 First time login user account creation Chapter 5. WebSphere Business Services Fabric for z/OS 257 After registering, the “Welcome to the WebSphere Business Services Fabric” window (Figure 5-30) is displayed. Figure 5-30 WebSphere Business Services Fabric for z/OS successful login 5.2.8 Create symlink for fabric product registration The sub-capacity reporting tool (SCRT) allows you to generate sub-capacity reports. You must submit your SCRT reports to IBM each month to qualify for sub-capacity workload license charges, sub-capacity entry workload license charges, as well as other sub-capacity workload license charges. The SCRT generates reports based on registration records recorded in the IBM System Management Facility (SMF). These records are generated by enabled products. WebSphere Application Server for z/OS, WebSphere Process Server for z/OS, and WebSphere Business Services Fabric for z/OS are all enabled for SCRT reporting. To register WebSphere Business Services Fabric for z/OS for sub-capacity reporting through SCRT, a registration record must be copied into a WebSphere Application Server for z/OS directory. Rather than actually copy this record, we 258 z/OS: WebSphere Business Process Management V6.2 Production Topologies created a symbolic link from the installed file in the WebSphere Business Services Fabric for z/OS SMPE-managed HFS to the WebSphere Process Server for z/OS HFS. Refer to Example 5-15 for a sample of the commands we used. Example 5-15 Create a symbolic link for the WBSF registrar file for SCRT reporting ln -s /usr/lpp/webifyMP/configuration/Runtime/zos/WBSF.registrar /wasv61config/mdcell/mdnodea/AppServer/properties/version/WBSF.registrar It is necessary to change the permissions for this symbolic link. Make sure you have R/W access to the WebSphere HFS before you execute this command (Example 5-16). Example 5-16 Change file permissions for the registrar symbolic link. chmod 755 /wasv61config/mdcell/mdnodea/AppServer/properties/version/WBSF.registrar Verify registration succeeded After you start your server, look for a valid message in the joblog for the servant region. You should see the message shown in Example 5-17 if the registration succeeds. Example 5-17 Successful registration for WebSphere Business Services Fabric for z/OS BBOO0332I PRODUCT 'WBSF for z/OS IFAUSAGE SERVICE. ' SUCCESSFULLY REGISTERED WITH 687 5.3 Optional configuration steps for stand-alone server There are several additional configuration steps you can take for your stand-alone server: 1. Change WebSphere Application Server for z/OS servant regions to run in 64-bit mode (5.3.1, “Configure WebSphere Application Server for z/OS servants for 64-bit addressing” on page 260) 2. Configure message engines to use MQ shared queues (5.3.2, “Configure message engines to use MQ shared queues” on page 265) 3. Configure HTTP servers or proxies (5.3.3, “Define HTTP servers or proxies” on page 278 4. Configure LDAP Security (5.3.4, “Configure LDAP security” on page 280) Chapter 5. WebSphere Business Services Fabric for z/OS 259 5.3.1 Configure WebSphere Application Server for z/OS servants for 64-bit addressing Address spaces on z/OS are limited to 31 bits or to 64 bits, depending on the addressing mode selected for the server address space. WebSphere Application Server Network Deployment for z/OS address spaces in V.6.1 start in 31-bit mode by default. Recall that 31 bits of addressing limits the total memory space in a z/OS address space to 2 GB of storage, This is roughly half of comparable 32-bit systems. Java heap in a 31-bit address space has a practical upper limit of 768 MB. Beyond this value, unpredictable results can occur as z/OS attempts to satisfy storage requests. For our purposes in deploying WebSphere Business Services Fabric for z/OS, we would like to have more memory available. Fortunately, WebSphere Application Server for z/OS allows us to run in 64-bit addressing mode. We set up 64-bit addressing as follows for the relevant address spaces: Daemon: 31-bit Deployment manager controller: 31-bit Deployment manager servant: 31-bit Node agents: 31-bit Node server controller: 31-bit Node server adjunct: 31-bit (could be 64, but we chose not to change) Node server servant: 64 bit In the steps below we show how to set up a servant for 64-bit addressing. Refer to Techdocs publication WP100920, WAS 6.1 64-bit Addressing Support, available from the IBM Technical Sales Library at the following Web page: http://www.ibm.com/support/techdocs The steps to configure 64-bit addressing for a WebSphere Application Server for z/OS server are as follows: 1. Ensure the WebSphere Application Server for z/OS libraries for 64-bit support are specified in your JCL started PROC for the server. See Example 5-18. Example 5-18 Server MDASRZ JCL from SYS1.PROCLIB //* //*Steplib //* //STEPLIB // // 260 Setup DD DISP=SHR,DSN=BBO6142.MDNODEA.SBBOLD2 DD DISP=SHR,DSN=BBO6142.MDNODEA.SBBOLOAD DD DISP=SHR,DSN=BBO6142.MDNODEA.SBBGLOAD z/OS: WebSphere Business Process Management V6.2 Production Topologies // // // DD DISP=SHR,DSN=DB9F9.SDSNEXIT DD DISP=SHR,DSN=DB9F9.SDSNLOAD DD DISP=SHR,DSN=DB9F9.SDSNLOD2 2. Add AMODE=64 to the JCL-started PROC for the server. See Example 5-19. Example 5-19 Server MDSRA JCL from SYS1.PROCLIB //MDASRA PROC ENV=,Z=MDASRAZ,AMODE=64 // SET ROOT='/wasv61config/mdcell/mdnodea' //BBOPASR EXEC PGM=BBOOMTST,PARM='&AMODE.' // INCLUDE MEMBER=&Z //TEST1 IF (BBOPASR.RC = 0) THEN //BBOSR EXEC PGM=BBOSR,REGION=0M,TIME=NOLIMIT, // PARM='ENVAR("_EDC_UMASK_DFLT=007") /' //BBOENV DD PATH='&ROOT/&ENV/was.env' // INCLUDE MEMBER=&Z // ELSE //BBGSR EXEC PGM=BBGSR,REGION=0M,TIME=NOLIMIT,MEMLIMIT=NOLIMIT, // PARM='ENVAR("_EDC_UMASK_DFLT=007") /' //BBOENV DD PATH='&ROOT/&ENV/was.env' // INCLUDE MEMBER=&Z // ENDIF Chapter 5. WebSphere Business Services Fabric for z/OS 261 3. In the Integrated Solutions Console, perform the following steps: a. b. c. d. Navigate to Servers → Application Servers. Click Application Servers. Click one of the servers. We chose server mdsr01a_WPS. Select the Run in 64 bit JVM mode check box (Figure 5-31). Figure 5-31 Run server in 64-bit mode 262 z/OS: WebSphere Business Process Management V6.2 Production Topologies e. In the Server Infrastructure section, expand Java and Process Management (Figure 5-32). Figure 5-32 Navigate to Java and Process Management f. Click Process Definition. Notice all server processes for server mdsr01a_WPS are listed (Figure 5-33). Figure 5-33 Server mdsr01a_WPS processes Chapter 5. WebSphere Business Services Fabric for z/OS 263 g. Click Servant. Examine all parameters. Make sure startCommandArgs includes AMODE=64. See Figure 5-34. Figure 5-34 Servant process parameters h. Click Java Virtual Machine. i. We changed the values for Initial Heap Size and Maximum Heap Size as follows: • • Initial Heap Size: 512 Maximum Heap Size: 1024 j. Click OK, and Save. 4. Stop and restart the stand-alone server. 264 z/OS: WebSphere Business Process Management V6.2 Production Topologies 5. Check for the message shown in Example 5-20 in the JES joblog for the servant. Example 5-20 JES joblog output for MDSR1A BBOO0004I WEBSPHERE FOR Z/OS SERVANT PROCESS 332 MDCELL/MDNODEA/MDSR1/MDSR1A IS STARTING. BBOO0239I WEBSPHERE FOR Z/OS SERVANT PROCESS 333 mdcell/mdnodea/mdsr01a_WPS IS STARTING. BBOO0308I SERVANT PROCESS MDCELL/MDNODEA/MDSR1/MDSR1A IS EXECUTING IN 64-BIT ADDRESSING MODE. Your servant for server mdsr01a_WPS is started in 64-bit addressing mode. 5.3.2 Configure message engines to use MQ shared queues If you are able to use WebSphere MQ as a message provider, the potential bottleneck of running one message engine in an adjunct disappears. For this reason, if your applications make heavy use of messaging, it is more advantageous to use WebSphere MQ as a message provider than to split the message engines into a separate cluster or consider partitioning. There are five busses in our environment, and many queues defined in support of them. We chose to focus our efforts on the fabric.mdcell.bus. There is one inbound queue and one outbound queue defined for the Fabric bus. The resources were created by the Fabric profile augmentation process and use the WebSphere Default Messaging Provider. We suggest you navigate through these resource definitions using the Integrated Solutions Console and save the values for reference. We will be creating a new inbound queue using MQ shared queues. In the steps below we detail how to use the installed WebSphere MQ Messaging Provider to set up the Fabric queues to use our shared MQ infrastructure. This involves defining multiple resources and changing our fabric bus to use them. There is a fair amount of information about this topic available. Your best resources are the WebSphere Application Server for z/OS Information Center and this Redbooks publication. We follow a step by step approach: 1. Ensure the MQ libraries are specified in your JCL started PROCs for the servers. See Example 5-21 on page 266. You must update the JCL for the controllers, adjuncts, and servants. Chapter 5. WebSphere Business Services Fabric for z/OS 265 Example 5-21 Server MDASRZ JCL from SYS1.PROCLIB //STEPLIB // // // // // // // DD DD DD DD DD DD DD DD DISP=SHR,DSN=BBO6153.MDNODEB.SBBOLD2 DISP=SHR,DSN=BBO6153.MDNODEB.SBBOLOAD DISP=SHR,DSN=BBO6153.MDNODEB.SBBGLOAD DISP=SHR,DSN=DB9F9.SDSNEXIT DISP=SHR,DSN=DB9F9.SDSNLOAD DISP=SHR,DSN=DB9F9.SDSNLOD2 DISP=SHR,DSN=MQ600.SCSQANLE DISP=SHR,DSN=MQ600.SCSQAUTH 2. Ensure that the program attribute bit is set for the JMS runtime libraries. Our JMS libraries are mounted at /usr/lpp/mqm/mqfg. a. Navigate to the java/lib directory. b. Ensure the +a and +p bits are turned on for all jms files that end in .so. See Figure 5-35 for an example of how to do this. Figure 5-35 Verify JMS runtime libraries 266 z/OS: WebSphere Business Process Management V6.2 Production Topologies 3. Create a new MQ server for our shared queue implementation. 4. In the Integrated Solutions Console, perform the following steps: a. Navigate to Servers → WebSphere MQ Servers. b. Click New. c. Specify the following parameters: • • • • • • • Name: MDCELL.MQF1 Description: MQ shared queue WebSphere MQ server name: MQF1 Server Type: Queue sharing group Select Use bindings transport mode if available WebSphere MQ host: wpsplex.itso.ibm.com WebSphere MQ port: 1415 Accept default values for the remainder of the fields. See Figure 5-36 on page 268. Chapter 5. WebSphere Business Services Fabric for z/OS 267 Figure 5-36 Create MQ Server d. Click Test Connection. The result should be a good connection to the MQ shared queue specified (Figure 5-37). Figure 5-37 Successful connection to MQ 268 z/OS: WebSphere Business Process Management V6.2 Production Topologies e. Click OK and Save. See Figure 5-38. Figure 5-38 MQ server is created 5. Add a new bus member to the fabric.mdcell.bus. 6. In the Integrated Solutions Console, perform the following steps a. b. c. d. e. f. Navigate to Service Integration → Buses. Click Buses. Click Fabric.mdcell.bus. Click Bus Members. Click Add. Select WebSphere MQ server and MDCELL.MQF1 from the drop-down menu (Figure 5-39). Figure 5-39 Specify bus member parameters Chapter 5. WebSphere Business Services Fabric for z/OS 269 g. Click Next. See Figure 5-40. Figure 5-40 Specify bus member parameters h. Click Next. See Figure 5-41. Figure 5-41 Review bus member parameters 270 z/OS: WebSphere Business Process Management V6.2 Production Topologies i. Click Finish and Save. See Figure 5-42. Figure 5-42 New Bus member created 7. Add a new inbound queue to the bus member. 8. In the Integrated Solutions Console, perform the following steps a. b. c. d. e. f. Navigate to Service Integration → Buses. Click Buses. Click Fabric.mdcell.bus. Click Destinations. Click New. Select Queue (Figure 5-43). Figure 5-43 Create a new queue destination g. Click Next. Chapter 5. WebSphere Business Services Fabric for z/OS 271 h. Specify the following values in the “Set queue attributes” window (Figure 5-44). • • Identifier: HUB Request Queue MQ. Description: MQ Request Queue for Fabric HUB. Figure 5-44 Specify queue destination name i. Click Next. j. Specify Bus Member: WebSphere MQ server = MDCELL.MQF1. See Figure 5-45. Figure 5-45 Assign destination to bus member k. Click Next. 272 z/OS: WebSphere Business Process Management V6.2 Production Topologies l. For WebSphere MQ queue name, select other, please specify from the drop-down menu. Enter HUB.Request.Queue in the field. See Figure 5-46. Figure 5-46 Specify queue name Chapter 5. WebSphere Business Services Fabric for z/OS 273 m. Click Next. See Figure 5-47. Figure 5-47 Review parameters n. Click Finish and click Save. See Figure 5-48. Figure 5-48 New destination is created You should now have a MQ shared queue destination defined for the Fabric.mdcell.bus message bus. 274 z/OS: WebSphere Business Process Management V6.2 Production Topologies 9. Change the existing JMS resource definition for the inbound Fabric queue to use our new MQ queue. 10.In the Integrated Solutions Console, perform the following steps: a. b. c. d. Navigate to Resources → JMS. Click Queues. Select HUB Request Queue. In the Queue name drop-down menu, change the value from HUB.Request.Queue to HUB.Request.Queue.MQ. See Figure 5-49. Figure 5-49 Specify new queue Chapter 5. WebSphere Business Services Fabric for z/OS 275 11.Click OK and click Save. You have now completed configuration or changing the inbound queue for the Fabric queue to use MQ-shared queues instead of the WebSphere Default messaging provider. Stop the stand-alone server The server must be stopped and restarted to pick up the changes we have made. To stop the stand-alone server issue the P MDSR1A command: An alternative to the traditional Stop command is shown in Example 5-11 on page 251. 1. From the SDSF DA window, issue the prefix MD* command to narrow the selection list. 2. Select the control region (MDSR1A below) with a Y (Example 5-22). Issuing a Y from the SDSF DA display effectively stops the control region and all associated servants. Example 5-22 Issue the Y command on the SDSF DA window SDSF DA SC42 (ALL) PAG 0 CPU/L/Z 4/ 4/ 0 LINE 1-4 COMMAND INPUT ===> SC PREFIX=MD* DEST=(ALL) OWNER=* SORT=SysName/A JOBNAME/A SYSNA NP JOBNAME JobID SysN CPU% Real StepName ProcStep Owner MDDEMNA STC25300 SC42 0.00 8801 MDDEMNA BBODAEMN MDACRU Y MDSR1A STC25296 SC42 0.03 65T MDSR1A BBOCTL MDACRU MDSR1AA STC25304 SC42 0.12 105T MDSR1AA BBOSR MDASRU MDSR1AS STC25301 SC42 0.12 146T MDSR1AS BBOSR MDASRU Start the stand-alone server Using the MVS console, or SDSF TSO windows, issue the start command for the stand-alone server. On our system, this was the form of the following command: START MDACRA,JOBNAME=MDSR1A,ENV=MDBASEA.MDNODEA.MDSR1A Make sure the four address spaces start up. See Example 5-12 on page 251. 276 z/OS: WebSphere Business Process Management V6.2 Production Topologies Example 5-23 SDSF DA window for a standalone server SDSF DA SC42 (ALL) COMMAND INPUT ===> PREFIX=MD* DEST=(ALL) NP JOBNAME JobID MDDEMNA STC25300 MDSR1A STC25296 MDSR1AA STC25304 MDSR1AS STC25301 PAG 0 CPU/L/Z 4/ 4/ 0 LINE 1-4 SC OWNER=* SORT=SysName/A JOBNAME/A SYSNA SysN CPU% Real StepName ProcStep Owner SC42 0.00 8801 MDDEMNA BBODAEMN MDACRU SC42 0.03 65T MDSR1A BBOCTL MDACRU SC42 0.12 105T MDSR1AA BBOSR MDASRU SC42 0.12 146T MDSR1AS BBOSR MDASRU Review any error messages and repair When the server finishes initialization, look in the server’s log files for any error messages, and examine the Integrated Solutions Console for the settings of various components. This is quite easy if you know what you are looking for. Examine the server logs on JES spool Using SDSF Display Active window, select your application server’s started tasks with the PREfix or Select MD* commands and look in the control region (MDSR1A), control region adjunct (MDSR1AA), and servant regions (MDSR1AS). Select each one of them with the S action character, and look for any sever error messages with the FIND SEVERE ALL command. Hopefully, there will be none. If there are, note the number displayed by SDSF, and step through each one and start your problem determination process. We found it useful to review any Java exceptions in the servant region joblogs. These exceptions do not always get flagged as severe errors, but that does not mean they are not important. Back up the stand-alone server’s ZFS data set At each major configuration milestone, and before proceeding, we find it a good idea to back up our configuration HFS. We used the JCL shown in Example 5-24 to perform the back up. Example 5-24 JCL to back up the stand-alone server node //MDDUMP JOB NOTIFY=&SYSUID,CLASS=A,MSGCLASS=A, // REGION=0M,TIME=1440 //* //* DUMP CONFIG HFS FOR DMGR AND NODE A //* //DUMP EXEC PGM=ADRDSSU //DASD DD DISP=(NEW,CATLG),DSN=WASCFG.MDCELL.DUMP.POSTMQ, Chapter 5. WebSphere Business Services Fabric for z/OS 277 // UNIT=3390,SPACE=(CYL,(500,100),RLSE) //SYSPRINT DD SYSOUT=* //SYSIN DD * DUMP DATASET( INCLUDE( WASCFG.MDCELL.MDDMNODE.CONFIG.ZFS ) ) OUTDD(DASD) TOL(ENQF) OPTIMIZE(4) COMPRESS /* - 5.3.3 Define HTTP servers or proxies When you have a cluster, it is usually necessary to define one or more HTTP Servers, or some kind of proxy, in front of the cluster. The HTTP Servers run the WebSphere plugin, which is responsible for distributing work and maintaining affinity between a client and a cluster member. Alternatively, you can use the on-demand router (ODR) that is part of WebSphere Extended Deployment for z/OS Virtual Enterprise, or perhaps use some other third-party proxy. IBM White Paper PRS2663, WebSphere z/OS - Comparing Front End HTTP Options, will help you understand the differences between these solutions. You can obtain that paper from the following Web page: http://www.ibm.com/support/techdocs When you run HTTP servers on z/OS you have a choice between the older IBM HTTP Server for z/OS (based on Lotus® Domino® Go technology) and the newer IBM HTTP Server (based on Apache). You can also choose to run the HTTP Server on some other platform. We configured the IBM HTTP Server for z/OS based on Lotus Domino Go. (We refer to this HTTP Server as “IBM HTTP Server for z/OS (LGW)” hereafter.) HTTP Servers were configured on the LPARs hosting the nodes of our MD and MP cells. We do not include step-by-step instructions on configuring an HTTP Server in this Redbooks publication. Instead, we describe the two z/OS HTTP Server options briefly, and provide references to existing step-by-step guides for each of these options. 278 z/OS: WebSphere Business Process Management V6.2 Production Topologies IBM HTTP Server for z/OS powered by Apache The newer HTTP Server is shipped as one component of the WebSphere Application Server for z/OS optional materials and is usually found in the file system at /usr/lpp/zWebSphere_OM/V6R1/HTTP/Server. The IBM HTTP Server for z/OS Powered by Apache is FMID JIWO610, Component IDs 5655I3510 and 5655I3511. IBM White Paper WP101170, The IBM HTTP Server for z/OS Powered by Apache, describes how to configure it. You can download that White Paper from the following Web page: http://www.ibm.com/support/techdocs The service level of the HTTP Server powered by Apache needs to be kept in step with the underlying WebSphere Application Server for z/OS. You can find details of the corresponding service levels at the following Web page: http://www.ibm.com/support/docview.wss?&uid=swg27009131 IBM HTTP Server for z/OS (LGW) The IBM HTTP Server for z/OS (LGW) has been shipped as part the z/OS base operating system for many years. There are three FMIDs: HIMW530 JIMW53B JIMW531 The LGW is typically found in the file system at /usr/lpp/internet. You can find a step-by-step guide to configuring the IBM HTTP Server for z/OS (LGW) in IBM Redpaper WebSphere Process Server for z/OS: Configuring a Network Deployment Environment, REDP-4388. Although that paper discusses WebSphere Process Server for z/OS V6.0.2, the instructions are largely the same for V6.2. Chapter 5. WebSphere Business Services Fabric for z/OS 279 5.3.4 Configure LDAP security In preparing this Redbooks publication, we used a localOS user registry based on RACF by default. We felt this was the most logical choice because the focus of this book is z/OS. There are, however, reasons why you might choose another option. A few examples are as follows: Your corporate mandate is to use LDAP, and you do not want to try to manage multiple different user registries and the federation of user identities across those registries. You plan to deploy WebSphere Business Services Fabric for z/OS and need to define users and groups in an external LDAP federation project. You plan to use WebSphere Business Monitor and require cross cell authentication between the WebSphere Application Server for z/OS cell supporting WebSphere Process Server for z/OS and the WebSphere Application Server cell supporting WebSphere Business Monitor, therefore requiring an LDAP server implementation. We use the remainder of this section to describe the steps we took to enable security using an LDAP user registry. Choosing an LDAP server As outlined in “Why use Tivoli Directory Server for z/OS?” on page 68, there are two LDAP servers available for use on z/OS 1.8: z/OS Integrated Security Services (ISS) LDAP IBM Tivoli Directory Server (TDS) for z/OS V6.0 LDAP We chose to use IBM Tivoli Directory Server for z/OS LDAP for the reasons listed above. Configuring users and groups The WPS_LDAP worksheet on the spreadsheet described in 2.2.2, “Using a spreadsheet to plan names for configuration objects” on page 45, contains entries in ldif format for loading into Tivoli Directory Server for z/OS. Details about the LDAP server are set on the Variables worksheet. The users and groups are derived from the other worksheets. Most of them come from the WPS_RACF worksheet where the majority of WebSphere Process Server for z/OS and WebSphere Business Services Fabric for z/OS user IDs and groups are planned. The .ldif files we used to populate our LDAP server are called zos_MD_itso.ldif and zos_MD_itso_wbsf.ldif. Both files are described in Appendix A, “Additional material” on page 615. You must populate your LDAP directory with the required user IDs and groups, before you attempt to enable WebSphere to use LDAP. 280 z/OS: WebSphere Business Process Management V6.2 Production Topologies Configuring WebSphere to use a LDAP user registry We took the following steps to enable our WebSphere Application Server for z/OS configuration to use an LDAP user registry. 1. While not strictly required, we got much better results with configuring security when we started with security turned off. To turn security off perform the following steps: a. In the Integrated Solutions Console, navigate to Security→ Secure administration, applications, and infrastructure. b. In the Administrative security section, clear the Enable administrative security check box. See Figure 5-50. Figure 5-50 Turn off administrative security c. Click Apply and click Save. d. Stop and restart the stand-alone server to pick up the new security settings. e. Review and resolve any error messages. Chapter 5. WebSphere Business Services Fabric for z/OS 281 f. Examine the server logs on JES spool. Look for the message shown in Example 5-25 in the servant log: Example 5-25 Verify security is turned off Trace: 2009/03/26 13:56:09.392 01 t=7FF028 c=UNK key=P8 (13007002) ThreadId: 0000000a FunctionName: com.ibm.ws.security.core.distSecurityComponentImpl SourceId: com.ibm.ws.security.core.distSecurityComponentImpl Category: INFO ExtendedMessage: BBOO0222I: SECJ0243I: Security service started successfully Trace: 2009/03/26 13:56:09.392 01 t=7FF028 c=UNK key=P8 (0000000A) Description: Log Java Message Message: BBOO0222I: SECJ0243I: Security service started successfully Trace: 2009/03/26 13:56:09.394 01 t=7FF028 c=UNK key=P8 (13007002) ThreadId: 0000000a FunctionName: com.ibm.ws.security.core.distSecurityComponentImpl SourceId: com.ibm.ws.security.core.distSecurityComponentImpl Category: INFO ExtendedMessage: BBOO0222I: SECJ0210I: Security enabled false Trace: 2009/03/26 13:56:09.394 01 t=7FF028 c=UNK key=P8 (0000000A) Description: Log Java Message Message: BBOO0222I: SECJ0210I: Security enabled false 282 z/OS: WebSphere Business Process Management V6.2 Production Topologies g. Verify security is turned off by logging into the administrative console. If your login window is as shown in Figure 5-51, security is disabled for your server. Figure 5-51 Verify security is disabled h. Back up the stand-alone server’s ZFS data set. To enable your server to use an LDAP user registry, follow these steps. 1. Log on into the Integrated Solutions Console. Note: Logon is still required even when security is disabled. This is so that changes made during the session can be associated with a user. Note that no password is required. 2. In the Integrated Solutions Console, perform the following steps: a. Navigate to Security→ Secure administration, applications, and infrastructure. b. In the User account repository section, ensure the Current realm definition will be set to Local operating system (assuming you previously configured your server for use with RACF). c. In the drop-down menu for Available realm definitions, select Standalone LDAP registry. Chapter 5. WebSphere Business Services Fabric for z/OS 283 d. Click Configure. See Figure 5-52. Figure 5-52 Configure LDAP user registry Note: Figure 5-53 displays a warning that appears at the top of the Integrated Solutions Console. This is expected and will be resolved below. Figure 5-53 Configuration warning message 284 z/OS: WebSphere Business Process Management V6.2 Production Topologies 3. Configure the LDAP server by performing the following steps: a. Enter the following values: Note: These values are correct for our installation and match the values we used to set up our LDAP server with. Your values may be different. • • • • • Type of LDAP server: Custom Primary administrative user name: uid=mdadmin, cn=People,o=ibm Server user identity: Server identity that is stored in the repository Server user ID or administrative user on a version 6.0.x node: uid=mdadmin, cn=People,o=ibm Password: mdadmin Note: The preceding values are not strictly required for our 6.1 WebSphere Application Server for z/OS installation. We could have chosen to specify Automatically generated server identity. We include these values for completeness. • • • • • • Host: wpsplex.itso.ibm.com Port: 2389 Base distinguished name (DN): o=ibm Bind distinguished name (DN): uid=ldapadm,o=ibm Bind password: ldapadm Search timeout: 120 b. Select the Reuse connection check box. c. Select the Ignore case for authorization check box. d. If you do not want to use SSL, clear the SSL enabled check box. We chose not use SSL in our installation. Your window should look similar to Figure 5-54 on page 286. Chapter 5. WebSphere Business Services Fabric for z/OS 285 Figure 5-54 Specify LDAP server parameters 4. Click Test connection to verify your LDAP server connection (Figure 5-55). Figure 5-55 Verify LDAP server connection 286 z/OS: WebSphere Business Process Management V6.2 Production Topologies 5. Click OK. If there are no errors, you should see the image shown in Figure 5-56. If there are errors, double-check your LDAP server values specified in your imported .ldif definitions. To enable the LDAP user registry you must exit this window with no errors. Figure 5-56 Verify LDAP server parameters 6. Click Save. 7. Set the filters for our LDAP server. 8. In the Integrated Solutions Console, perform the following steps: a. Navigate to Security→ Secure administration, applications, and infrastructure. b. Ensure Current realm definition is set to Standalone LDAP registry. c. Select Standalone LDAP registry from the Available realm definitions drop-down menu. d. Click Configure. e. Click Advanced lightweight directory access protocol (LDAP) user registry settings. f. In the General properties windows, make sure the following values are set. These values are specific for our environment. Ensure you use the values that match your LDAP server definitions. • • • • • User filter: (&(uid=%v)(objectclass=inetOrgPerson)) Group filter: (&(cn=%v)(objectclass=groupOfUniqueNames)) User ID map: :uid Group ID map: :cn Group member ID map: groupOfUniqueNames: uniqueMember See Figure 5-57 on page 288. Chapter 5. WebSphere Business Services Fabric for z/OS 287 Figure 5-57 LDAP Server filters i. Click OK and click Save. g. Click OK. h. Click Save. 9. Change some Custom properties to disable SAF checking. 10.In the Integrated Solutions Console, perform the following steps: a. Navigate to Security→ Secure administration, applications, and infrastructure. b. Click Custom properties. c. Enter the following parameters, clicking OK after each one: • • com.ibm.security.SAF.authroization: false com.ibm.security.SAF.delegation: false See Figure 5-58. Figure 5-58 Change Custom properties 288 z/OS: WebSphere Business Process Management V6.2 Production Topologies d. Click Save. 11.Enable security using our newly configured LDAP user registry. 12.In the Integrated Solutions Console, perform the following steps: a. Navigate to Security→ Secure administration, applications, and infrastructure. b. In the Administrative security section, select the Enable administrative security check box. c. In the Application security section, select the Enable application security check box. d. In the Java 2 security section, ensure the Use Java 2 security to restrict application access to local resources check box is cleared. Java 2 security is beyond the scope of this exercise and is not required by default. e. In the User account repository section, ensure the Current realm definition is set to Standalone LDAP registry. f. In the drop-down menu for Available realm definitions, select Standalone LDAP registry. See Figure 5-59 on page 290. Chapter 5. WebSphere Business Services Fabric for z/OS 289 Figure 5-59 Enable LDAP user registry g. Click Apply. 290 z/OS: WebSphere Business Process Management V6.2 Production Topologies h. Read the generated warnings (Figure 5-60) concerning restarting servers and synching changes. Figure 5-60 LDAP configuration warnings 13.Click Save. 14.Examine the server logs on JES spool. Look for the message shown in Example 5-26 in the servant log. Example 5-26 Verify LDAP user registry is initialized Trace: 2009/03/26 15:02:43.406 01 t=7B09D8 c=UNK key=P8 (13007002) ThreadId: 00000050 FunctionName: com.ibm.ws.security.registry.UserRegistryImpl SourceId: com.ibm.ws.security.registry.UserRegistryImpl Category: AUDIT ExtendedMessage: BBOO0222I: SECJ0136I: Custom Registry:com.ibm.ws.security.registry.ldap.LdapRegistryImpl has been initialized 15.Stop and restart the stand-alone server to pick up the new security settings. 16.Review and resolve any error messages. 17.Examine the server logs on JES spool. Look for the message shown in Example 5-27 in the servant log: Example 5-27 JES spool server log message Trace: 2009/03/26 15:11:02.307 01 t=7FF028 c=UNK key=P8 (13007002) Chapter 5. WebSphere Business Services Fabric for z/OS 291 ThreadId: 0000000a FunctionName: com.ibm.ws.security.registry.UserRegistryImpl SourceId: com.ibm.ws.security.registry.UserRegistryImpl Category: AUDIT ExtendedMessage: BBOO0222I: SECJ0136I: Custom Registry:com.ibm.ws.security.registry.ldap.LdapRegistryImpl has been initialized Trace: 2009/03/26 15:11:02.356 01 t=7FF028 c=UNK key=P8 (13007002) ThreadId: 0000000a FunctionName: com.ibm.ws.security.registry.ldap.LdapRegistryImpl SourceId: com.ibm.ws.security.registry.ldap.LdapRegistryImpl Category: AUDIT ExtendedMessage: BBOO0222I: SECJ0419I: The user registry is currently connected to the LDAP server ldap://wpsplex.itso.ibm.com:2389 18.Back up the stand-alone server’s ZFS data set. 19.Log on to the Integrated Solutions Console using your cell administrative ID (Figure 5-61). Figure 5-61 Logon to Integrated Solutions Console You have now successfully enabled your cell to use an LDAP user registry (Figure 5-62 on page 293). 292 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 5-62 Successful login to Integrated Solutions Console using LDAP user registry Configuring roles There are many required role definitions. Even if you previously defined these roles in RACF, you may have reason to modify them for use with your LDAP user registry. WebSphere Application Server for z/OS. If you have not previously defined the required roles and their relationship to the defined user IDs and groups in your LDAP directory, complete that task now. The role definitions below are based on those created by the MDEJBROL job described in “Using EJBROLE profiles” on page 71. The roles are required to support WebSphere Process Server for z/OS, including CEI, Business Space, BPC Explorer, and REST services, as well as those required for WebSphere Business Services Fabric for z/OS. As in the case of a RACF based user registry, it is a good idea to create all the required user IDs, and groups before you configure WebSphere Process Server for z/OS or WebSphere Business Services Fabric for z/OS. The EJB roles are specified by the deployed applications and must be mapped to the user IDs and groups in LDAP after the LDAP user registry has been enabled. You can do this mapping through the Integrated Solutions Console or through scripting. The user IDs, groups, and roles we created are summarized in Table 5-3. Table 5-3 user IDs, groups, and roles Group or User Description EJB roles Group ID, User ID BPE API User Group BPEAPIUser MDBPEADM BPE Administrator’s Group BPESystemAdministrator MDBPEADG BPE System Monitor’s Group BPESystemMonitor MDBPESMG Chapter 5. WebSphere Business Services Fabric for z/OS 293 Group or User Description EJB roles Group ID, User ID Human Task API User Group TaskAPIUser UACC(READ) Human Task Administrator’s Group TaskSystemAdministrator MDHTADMG Human Task System Monitor’s Group TaskSystemMonitor MDHTSMG BPC Web Client Role Group WebClientUser MDWCUG BPC IVT User Role Group BPCIVTUser MDWCUG MDB JMSAPIUser Role JMSAPIUser MDJMSAPI MDB Escalation User EscalationUser MDESCAL CEI Event Administrator Group eventAdministrator MDEVTADG CEI Event Consumer Group eventConsumer MDEVTCO CEI Event Creator Group eventCreator MDEVTCRG CEI Event Updater Group eventUpdater MDEVTUPG CEI Catalog Administrator Group catalogAdministrator MDCATADG CEI Catalog Reader Group CatalogReader MDCATRG Failed Event Manager User Group WBIOperator MDCFG, MDBPEADG, MDEVTADG Business Space Administrator Group Administrator MDBSPADG Rest Services Gateway Group RestServicesUser MDFABADM, MDFADMG Fabric Administrator Group FabricAdministrator MDFABADM, MDFADMG Fabric Basic User FabricBasicUser MDFBU, MDFBUG Fabric Governanace Administrator FabricGovernanaceAdministrator MDFGADM, MDFGADMG Fabric Performance User FabricPerformanceUser MDFPU, MDFPUG Fabric Subscriber Manager FabricSubscriberManager MDFSM, MDFSMG Fabric Studio User FabricStudioUser MDFSU, MDFSUG 294 z/OS: WebSphere Business Process Management V6.2 Production Topologies Map security roles to user IDS and groups For each of the roles specified in Table 5-3 on page 293, the following steps are required. 1. In the Integrated Solutions Console, perform the following steps: a. Navigate to Applications → Enterprise applications. b. Choose an application that needs to have its role definitions mapped. We use Fabric REST Services for example. Click Fabric REST Services. 2. Click Security role to user/ group mapping. See Figure 5-63. Figure 5-63 Select role to user mapping 3. Select the RESTServicesUser check box as shown in Figure 5-64 on page 296. Chapter 5. WebSphere Business Services Fabric for z/OS 295 Figure 5-64 Select role 4. Click Look up users. 5. Click Search to display user IDs and groups in the LDAP registry. See Figure 5-65 on page 297. 296 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 5-65 Display user IDs and groups 6. Select the user IDs to map to the RESTSerrviceUser role. – uid=mdfabadm,cn=People,o=ibm – uid=mdadmin,cn=People,o=ibm 7. Press the >> button to assign the user IDs to the role. 8. Click OK. 9. Clear the Everyone check box. Chapter 5. WebSphere Business Services Fabric for z/OS 297 10.Clear the All authenticated check box. 11.Click OK. 12.Click Save. 13.Stop and restart the Fabric REST Services application. Repeat the above steps for all applications and user roles described in Table 5-3 on page 293. 5.4 Troubleshooting the WebSphere Business Services Fabric for z/OS installation This section describes how to diagnose problems with the installation. There are a series of problems and solutions in Chapter 10, “Problem diagnosis and prevention” on page 571. Go to DB2 and verify the creation of eight tables for the fabricbus messaging engine. For our cell, these were in database MDSIMDB and they had a schema of MD01F. Go to the Integrated Solutions Console and verify the scope of the all JMS provider resources created for fabric. Verify the JNDI Names are the same for the JMS Provider resources. Verify the Destination JNDI Names are given properly for the Activation specifications. Check the server logs for any java.io.FilePermisssion exceptions for Fabric applications related to Java security. If exceptions are present, grant the permissions in the policy file or disable Java 2 security and restart all the servers. Make sure to investigate all ICH408I messages in the z/OS system log. The problems these messages highlight must be resolved, as they typically point to configuration and file permission problems concerning important z/OS resources. 5.5 WebSphere Business Services Fabric events WebSphere Business Services Fabric generates several Common Base Events (CBEs) that can be monitored by WebSphere Business Monitor. For details on how to configure WebSphere Business Services Fabric to generate events, see Chapter 8, “Monitoring the Business Process Management environment” on page 465. 298 z/OS: WebSphere Business Process Management V6.2 Production Topologies 6 Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster This chapter contains the following sections: 6.1, “Configuring WebSphere Business Services Fabric for z/OS in a clustered environment” on page 300 6.2, “Install WebSphere Business Services Fabric for z/OS in the deployment manager” on page 301 6.3, “Install WebSphere Business Services Fabric for z/OS in a managed node” on page 327 Figure 6.5 on page 343 6.6, “Creating a WebSphere Business Services Fabric for z/OS cluster configuration with empty managed nodes” on page 345 6.7, “Configuring a WebSphere Business Services Fabric for z/OS cluster from a stand-alone node” on page 361 6.8, “Verify basic server operation” on page 397 Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 299 6.1 Configuring WebSphere Business Services Fabric for z/OS in a clustered environment We took a staged approach to building out our WebSphere Business Services Fabric for z/OS cluster. We outline the steps we took below: 1. We built a stand-alone server as documented in Chapter 2, “Configuring a WebSphere Process Server for z/OS standalone server” on page 39. a. We built a stand-alone WebSphere Application Server for z/OS called basea. b. We augmented the stand-alone server with WebSphere Process Server for z/OS. c. We augmented the stand-alone server with WebSphere Business Services Fabric for z/OS as documented in 6.2.1, “Augment WebSphere Process Server for z/OS deployment manager profile” on page 302. 2. We built a deployment manager as documented in Chapter 3, “Creating a WebSphere Process Server for z/OS network deployment cell” on page 151. a. We built a WebSphere Application Server for z/OS deployment manager called dmgr. b. We augmented the deployment manager with WebSphere Process Server for z/OS. 3. We augmented the deployment manager with WebSphere Business Services Fabric for z/OS as documented in 6.2.1, “Augment WebSphere Process Server for z/OS deployment manager profile” on page 302. 4. We built an empty node as documented in Chapter 4, “Creating additional WebSphere Process Server nodes” on page 179. a. We built an empty WebSphere Application Server for z/OS node called nodeb. b. We augmented the empty node with WebSphere Process Server for z/OS. 5. We federated our nodes and created a two server cluster. a. We federated the stand-alone server into the cell (3.4, “Federating the standalone server into this ND cell” on page 165). This node became node A. b. We federated the empty node into the cell (4.4, “Federating the new node into the network deployment cell” on page 188). This became node B. c. We built a cluster with managed nodes node A and node B (3.6, “Creating a cluster from the server” on page 170) and (4.5, “Creating an additional cluster member” on page 189). 300 z/OS: WebSphere Business Process Management V6.2 Production Topologies 6. We augmented node B with WebSphere Business Services Fabric for z/OS as documented in 6.3.1, “Augment WebSphere Process Server for z/OS managed node profile” on page 328. 7. We performed cluster related configuration for WebSphere Process Server for z/OS as documented in Figure 6.5 on page 343. 6.2 Install WebSphere Business Services Fabric for z/OS in the deployment manager This section describes the steps for installing WebSphere Business Services Fabric into a deployment manager node. This is the first step in building a clustered environment. Most of the initial configuration steps were completed when configuring a stand-alone server. These steps do not need to be repeated for the deployment manager node. 1. Copy the Fabric artifacts. See 5.2.1, “Copy Fabric artifacts” on page 207. 2. Configure security. See 5.2.2, “Configure security” on page 209. 3. Configure databases. See 5.2.3, “Configure databases” on page 214. The majority of the actual configuration for WebSphere Business Services Fabric for z/OS occurs during the profile augmentation process. This process is unique for each node and the deployment manager. We describe this process for a deployment manager node in 6.2.1, “Augment WebSphere Process Server for z/OS deployment manager profile” on page 302. We go through a series of verification steps to ensure the augmentation worked the way we expected it to. This process is described in “Using the manageprofiles.sh command” on page 303. We then proceed to start the server. Post augmentation steps include recreating the Fabric datasource. See “Delete fabric datasource” on page 316 for details. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 301 Note: In this book, we outline two approaches for creating a clustered environment for WebSphere Business Services Fabric for z/OS. In the first instance, we build a cluster using the stand-alone node we previously configured in 5.2, “Install WebSphere Business Services Fabric for z/OS in a standalone node” on page 206. In this example, most of the resources created for the stand-alone node augmentation process are federated into the deployment manager, but not at the cluster level. We must undertake specific actions to correct these resources. We outline the steps we took in 6.7, “Configuring a WebSphere Business Services Fabric for z/OS cluster from a stand-alone node” on page 361. In the second instance, we federate two empty nodes into our cluster. We first augment the deployment manager, and augment both nodes through the deployment manager. In this case, we run two Fabric provided scripts to complete the process of creating the resources for the cluster. We outline these steps in Figure 6.5 on page 343. Make sure you understand which path you intend to take when building your cluster. We urge you to take the empty node approach as it is significantly easier to complete the configuration successfully. 6.2.1 Augment WebSphere Process Server for z/OS deployment manager profile Previous versions of WebSphere Business Services Fabric for z/OS required many manual configuration steps to properly set up and configure a WebSphere Process Server for z/OS server for Fabric. Improvements for V6.2 have removed most of these requirements. Most configuration for WebSphere Business Services Fabric for z/OS V6.2 is accomplished through profile augmentation. This section describes how to augment an existing WebSphere Process Server for z/OS profile to contain WebSphere Business Services Fabric for z/OS functions. The augmentation process uses the manageprofiles.sh command shipped with base WebSphere Application Server for z/OS. Before augmenting a profile, ensure you have completed the following steps: 1. Ensure you have fully configured your WebSphere Application Server for z/OS and WebSphere Business Services Fabric for z/OS stand-alone server environment. 2. Ensure you have completed the security requirements for your Fabric installation. See 5.2.2, “Configure security” on page 209. 302 z/OS: WebSphere Business Process Management V6.2 Production Topologies 3. Ensure you have created the required Fabric and message bus databases. See 5.2.3, “Configure databases” on page 214. 4. SMPE work for Fabric has been completed. 5. Ensure you ran the MDCPYFAB job or issued the copyfabric.sh script from a UNIX command shell to copy the Fabric profile templates to the proper locations in your WebSphere Application Server for z/OS configuration HFS. 6. Ensure you shut down the Servants associated with the profile you will augment. 7. Ensure you have enough space in the target HFS and in /tmp to complete the augmentation process. Note: The Fabric applications will require a lot more space in your ZFS file systems so they may need to extend during the deployment process. After installing Fabric, our deployment manager’s node ZFS occupied 1800 CYLs while each node occupied more than 2000 CYLs. To ensure a clean deployment, ensure that the volumes that host your cell’s ZFS datasets have enough free space to allow them to obtain secondary extents and grow to the sizes mentioned above. Using the manageprofiles.sh command The manageprofiles.sh command is located in the /WPS_HOME/bin directory. On our system the location for our WebSphere deployment manager node is /wasv61config/mdcell/mddmnode/AppServer/bin. The command used to augment the WebSphere Process Server for z/OS profile for WebSphere Business Services Fabric for z/OS uses the same parameters used in the base WebSphere Process Server installation as well as some additional Fabric specific parameters. Options for the manageprofiles.sh command are detailed in Table 5-2 on page 221. Note that all parameters are case sensitive. Issue the manageprofiles.sh command We created sample job MDAGDMGR (Example 6-1) to execute the manageprofiles command through the BPXBATCH interface. We chose to use JCL rather than execute the command from a UNIX command shell so we could capture the output of the command in the joblog. Note: The values we specified on the manageprofiles.sh command are specific for our environment. Make sure you change the parameters to reflect your WebSphere Application Server for z/OS and WebSphere Process Server for z/OS installation. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 303 Example 6-1 Sample job MDAGDMGR to execute the manageprofiles.sh command //MDAGDMGR JOB (0),CLASS=A,MSGCLASS=A,REGION=0M, // USER=MDADMIN,PASSWORD=MDADMIN, // NOTIFY=GATES /*JOBPARM SYSAFF=SC42 //***********************************************************/ //* Configure Fabric in deployment manager */ //***********************************************************/ //INSTO EXEC PGM=IKJEFT01,REGION=0M,TIME=1440 //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BPXBATCH SH + export LIBPATH=/usr/lpp/db2/d9fg/db2910_jdbc/lib:$LIBPATH; + cd /wasv61config/mdcell/mddmnode/DeploymentManager/bin; + ./manageprofiles.sh + -augment + -templatePath /wasv61config/mdcell/mddmnode/DeploymentManager+ /profileTemplates/dmgr.wbsfabric + -profileName default + -cellName mdcell + -nodeName mddmnode + -serverName dmgr + -configureRESTSecurity true + -adminUserName mdadmin + -adminPassword mdadmin + -fabricDbCreateNew false + -fabricDbType DB2_UNIVERSAL + -fabricDbDriverType 2 + -fabricDbName DB9F + -fabricDbUser MDDBU + -fabricDbPassword MDDBU + -fabricDbServerPort 37893 + -fabricDbHostName wtsc42.itso.ibm.com + -fabricDbJDBCClasspath /usr/lpp/db2/d9fg/db2910_jdbc/classes + 1> /tmp/mddmnodeFabricConfig_0A11.out + 2> /tmp/mddmnodeFabricConfig_0A11.err; /* //***********************************************************/ //* STEPS TO COPY THE OUTPUT THE JOB LOG */ //***********************************************************/ //CPOUT EXEC PGM=IKJEFT01,REGION=0M //SYSEXEC DD DISP=SHR,DSN=BBO6142.MDDMNODE.SBBOEXEC //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * 304 z/OS: WebSphere Business Process Management V6.2 Production Topologies BBOHFSWR '/tmp/mddmnodeFabricConfig_0A11.out' BBOHFSWR '/tmp/mddmnodeFabricConfig_0A11.err' /* // After you run the MDAGDMG job check the joblog for the job. You should see the following message in the output: INSTCONSUCCESS: Profile augmentation succeeded. To ensure the augmentation process did what it was supposed to, continue with 6.2.2, “Verify the augmentation process succeeded” on page 305. 6.2.2 Verify the augmentation process succeeded The profile augmentation process creates several artifacts with the WebSphere Application Server for z/OS deployment manager profile. The server must be started to pick up the changes we have made. Start the deployment manager Using the MVS console, or SDSF TSO windows, issue the start command for the deployment manager. On our system, the command was as follows: START MDMGCR,JOBNAME=MDDMGR,ENV=MDCELL.MDDMNODE.MDDMGR Make sure the three address spaces start up. See Example 6-2 on page 305. Example 6-2 SDSF DA window for a deployment manager SDSF DA SC42 SC* COMMAND INPUT ===> PREFIX=MD* DEST=(ALL) NP JOBNAME StepName MDDMGR MDDMGR MDDEMN MDDEMN MDDMGRS MDDMGRS PAG 0 OWNER=* ProcStep BBOCTL BBODAEMN BBOSR CPU/L/Z 6/ 5/ SORT=SysName/A JobID Owner STC01031 MDACRU STC01032 MDACRU STC01033 MDASRU Review any error messages and repair When the server finishes initialization, look in the server’s log files for any error messages, and examine the Integrated Solutions Console for the settings of various components. This is quite easy if you know what you are looking for. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 305 Examine the server logs on JES spool Using SDSF Display Active window, select your application server’s started tasks with the PREfix or Select MD* commands and look in the control region (MDDMGR), and servant region (MDDMGRS). Select each one of them with the S action character, and look for any sever error messages with the FIND SEVERE ALL command. Hopefully, there will be none. If there are, note the number displayed by SDSF, and step through each one and start your problem determination process. We also find it useful to review any Java exceptions in the servant region joblogs. These exceptions do not always get flagged as severe errors, but that does not mean they are not important. Back up the deployment manager’s ZFS data set At each major configuration milestone, and before proceeding, we find it a good idea to back up our configuration HFS. We used the JCL shown in Example 6-3 on page 306 to perform the back up. Example 6-3 JCL to back up the deployment manager node //MDDUMP JOB NOTIFY=&SYSUID,CLASS=A,MSGCLASS=A, // REGION=0M,TIME=1440 //* //* DUMP CONFIG HFS FOR DMGR //* //DUMP EXEC PGM=ADRDSSU //DASD DD DISP=(NEW,CATLG),DSN=WASCFG.MDCELL.DUMP.POSTAUG, // UNIT=3390,SPACE=(CYL,(500,100),RLSE) //SYSPRINT DD SYSOUT=* //SYSIN DD * DUMP DATASET( INCLUDE( WASCFG.MDCELL.MDDMGR.CONFIG.ZFS ) ) OUTDD(DASD) TOL(ENQF) OPTIMIZE(4) COMPRESS /* 306 z/OS: WebSphere Business Process Management V6.2 Production Topologies Log in to administrative console Log in to the admin server for the deployment manager. See Figure 6-1. Figure 6-1 Login to deployment manager admin console Verify that the Fabric resources were created properly by the profile augmentation process. Verify Fabric datasource was created As a result of federating the stand-alone node into the cluster, there are several datasources present for Fabric. A fabric_pm datasource is defined for each node and for the cluster. Augmenting the deployment manager creates an additional fabric_pm datasource at the cell level. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 307 To see these datasources, perform the following steps: 1. In the Integrated Solutions Console, navigate to Resources → JDBC. 2. Click Data sources. See Figure 6-2. Figure 6-2 fabric_pm datasources after deployment manager augmentation This datasource, as with the others, will be connected to the Fabric database in the DB2 z/OS subsystem. The Fabric database is used by the Fabric performance manager. To verify this datasource was created, perform the following steps: 1. In the Integrated Solutions Console, navigate to Resources → JDBC. 2. Click Data sources. 3. Click the fabric pm cell level datasource. See Figure 6-3. Figure 6-3 fabric_pm cell level datasource 308 z/OS: WebSphere Business Process Management V6.2 Production Topologies We will cleanup these extra fabric_pm datasources in our post cluster configuration steps (6.7, “Configuring a WebSphere Business Services Fabric for z/OS cluster from a stand-alone node” on page 361). Verify the Fabric JAAS-J2C authentication alias was created A FABRIC_JDBC_AUTH J2C alias was created by the augmentation of the stand-alone server. It is not re-created when augmenting the deployment manager. The J2C alias is created based on the user ID and password used to access the DB2 z/OS database defined for Fabric’s use. It is connected to the fabric_pm datasource. See Figure 6-4. Figure 6-4 Fabric datasource definition Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 309 1. On the fabric pm datasource window verify that the component-managed authentication alias is set to FABRIC JDBC AUTH. 2. Click JAAS - J2C authentication data to see the definition of the Fabric authentication alias, as shown in Figure 6-5. Figure 6-5 List of JAAS-J2C authentication aliases 3. Click FABRIC JDBC AUTH. See Figure 6-6 for the definition of the Fabric JAAS-J2C alias. Figure 6-6 Fabric JAAS-J2C authentication alias for deployment manager 310 z/OS: WebSphere Business Process Management V6.2 Production Topologies Verify the Fabric JDBC path variable was set The JDBC provider specified in the manageprofiles.sh command is listed as a JDBC driver for the server being augmented. This JDBC provider is present on z/OS servers by default and is not created as a result of the profile augmentation. To verify the JDBC path variable perform the following steps: 1. In the Integrated Solutions Console, navigate to Resources → JDBC. 2. Click JDBC Providers. A list of JDBC Providers is displayed. The datasource we specified on the manageprofiles.sh command, DB2 Universal JDBC Driver Provider, is in the list. See Figure 6-7. Figure 6-7 JDBC Providers on the deployment manager Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 311 Verify the Fabric .jar files were copied The following two .jar files should be copied to the plugins directory of the WebSphere Process Server for z/OS deployment manager: /wasv61config/mdcell/mddmnode/DeploymentManager/plugins. com.ibm.ws.repository_6.2xxx.jar com.ibm.ws.fabric.catalog_6.2.0.jar See Figure 6-8. Figure 6-8 Fabric jar files in plugins directory of deployment manager 312 z/OS: WebSphere Business Process Management V6.2 Production Topologies Verify the Fabric SCA jars were copied Verify the following jars were copied to the lib/ext directory of the WebSphere Process Server for z/OS stand-alone server: /wasv61config/mdcell/mddmnode/DeploymentManager/lib/ext fabric-da-api.jar fabric-da-model.jar fabric-da-sca.jar fabric-da-scdl.jar fabric-types.jar See Figure 6-9. Figure 6-9 Fabric SCA jars in the /lib/ext directory of the deployment manager Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 313 Verify the Fabric applications were deployed The Fabric applications were deployed on the stand-alone server. As part of the federation of the stand-alone server into the cluster, the following Fabric applications should still be deployed in the cell. Fabric_Catalog Fabric_Engine Fabric_REST_Services Fabric_Tools Fabric_Tools_Help To verify the applications are still there, perform the following steps: 1. In the Integrated Solutions Console, navigate to Applications → Enterprise Applications. 2. Click Enterprise Applications. A list of deployed applications is displayed. See Figure 6-10 on page 315. 314 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 6-10 Fabric applications deployed on the deployment manager Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 315 6.2.3 Post augmentation tasks There are several tasks that must be completed after profile augmentation to complete the configuration of Fabric on the deployment manager. These steps are outlined below. Delete fabric datasource Just as for the datasources created for the stand-alone server, for the deployment manager, for DB2 z/OS databases we have two choices when defining the JDBC datasources. Type 2 This JDBC driver takes advantage of z/OS cross memory services to provide a high-speed connector between the Java process and the DB2 main address space where SQL command will be executed. This driver does not use network protocols and is generally the most efficient in terms of resources consumed. Both the database subsystem and the Java process (in this case the WebSphere Application Server for z/OS server) must reside in the same LPAR to make a type 2 connection. Type 4 This JDBC driver uses a gateway function to provide remote access to DB2 z/OS resources. Type 4 connections use a distributed DRDA protocol to complete their connection. This connection type is more expensive in terms of resources and end-to-end throughput. It should only be used when the database subsystem and Java process are not in the same LPAR. Our implementation uses DB2 shared data and a DB2 subsystem on every LPAR. For this reason we chose to use a type 2 connection when augmenting our WebSphere Application Server for z/OS deployment manager server profile. See “Issue the manageprofiles.sh command” on page 303. We experienced a defect in the way the type 2 JDBC driver was created by the manageprofiles.sh command. As implemented, it uses the XA version of the type 2 driver. We verified this by performing the following steps: 1. In the Integrated Solutions Console, navigate to Resources → JDBC. 2. Click Data sources. 3. Click the fabric pm cell level resource. 316 z/OS: WebSphere Business Process Management V6.2 Production Topologies In Figure 6-11 below, we can see the incorrect JDBC driver specified in the Provider field. Figure 6-11 Incorrect JDBC driver provider for fabric pm database This driver is implemented on distributed systems, but not on z/OS. For this reason, we must drop this datasource and recreate it using the z/OS version of the type 2 driver. First we deleted the cell level fabric_pm datasource created by the manageprofiles.sh command. 1. In the Integrated Solutions Console, navigate to Resources → JDBC. 2. Click Data sources. 3. Select the fabric pm check box. 4. Click Delete. 5. Click Save. See Figure 6-12 on page 318. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 317 Figure 6-12 Delete incorrect fabric pm datasource Rebuild fabric datasource using non-XA DB2 UDB JCC driver We will now rebuild the cell level fabric pm datasource using the correct JDBC type driver. 1. In the Integrated Solutions Console, navigate to Resources → JDBC. 2. Click Data sources. Note: The scope is often set to “All scopes”. Select a scope before you click “New”... otherwise you will receive the following message: Select a valid scope from drop down list. "All scopes" is not a valid scope for this operation. 318 z/OS: WebSphere Business Process Management V6.2 Production Topologies 3. Click New. See Figure 6-13. Figure 6-13 Create a new fabric pm datasource Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 319 4. Fill in the following parameters on this window: a. Data source name: fabric pm b. JNDI name: jdbc/fabric/pm c. Component-managed authentication alias and XA recovery authentication alias: FABRIC_JDBC_AUTH Click Next. See Figure 6-14. Figure 6-14 Enter fabric pm datasource information 320 z/OS: WebSphere Business Process Management V6.2 Production Topologies 5. On the “Select JDBC provider” window (Figure 6-15), perform the following steps: a. Select the Select an existing JDBC provider check box. b. Select DB2 Universal JDBC driver provider from the drop-down menu. Figure 6-15 Specify JDBC provider for fabric pm database c. Click Next. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 321 6. On the “Enter database specific properties for the data source” window, perform the following steps: a. Specify the following parameters on the database panel: • • • • Database name: DB2 location name (DB9F in our case) Driver type: 2 Server name: wtsc42.itso.ibm.com Port number: 37893 b. Clear the Use this data source in container managed persistence (CMP) check box. See Figure 6-16. Figure 6-16 Specify database parameters for fabric pm datasource c. Click Next. 322 z/OS: WebSphere Business Process Management V6.2 Production Topologies 7. Verify fabric pm datasource parameters. See Figure 6-17. Figure 6-17 Verify new fabric pm datasource parameters 8. Click Finish to create the data source. 9. Click Save. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 323 Verify the new fabric pm datasource is using the correct JDBC type 2 driver by performing the following steps: 1. In the Integrated Solutions Console, navigate to Resources → JDBC. 2. Click Data sources. 3. Click the fabric pm cell level resource. 4. Verify the Provider field is set to DB2 Universal JDBC Driver Provider. See Figure 6-18. Figure 6-18 Verify new fabric pm datasource JDBC provider Update SQLID for fabric datasource Datasource creation does not support the specification of a schema ID. Because we created our fabric database with a schema, we must set the currentSchema for the fabric pm datasource to match this SQLID. To set the currentSchema perform the following steps: 1. In the Integrated Solutions Console, navigate to Resources → JDBC. 2. Click Data sources. 324 z/OS: WebSphere Business Process Management V6.2 Production Topologies 3. Click the fabric pm cluster=MDRS1 level resource. 4. Click Custom properties. 5. Click the currentSchema parameter. 6. In the Value box specify your SQLID. We use MDWPS. See Figure 6-19. Figure 6-19 Specify currentSQLID for fabric pm datasource 7. Click Apply. 8. Click Save to save your configuration changes. 9. Verify your changes in the fabric pm datasource Custom properties window, as shown in Figure 6-20. Figure 6-20 Verify currentSQLID setting for fabric pm datasource Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 325 Stop the deployment manager Using the MVS console, or SDSF TSO panels, issue the stop command for the deployment manager. On our system, the command was as follows: P MDMGCR Start the deployment manager Using the MVS console, or SDSF TSO panels, issue the start command for the deployment manager. On our system, the command was as follows: START MDMGCR,JOBNAME=MDDMGR,ENV=MDCELL.MDDMNODE.MDDMGR Make sure the three address spaces start up. See Example 6-4. Example 6-4 SDSF DA panel for a deployment manager SDSF DA SC42 SC* COMMAND INPUT ===> PREFIX=MD* DEST=(ALL) NP JOBNAME StepName MDDMGR MDDMGR MDDEMN MDDEMN MDDMGRS MDDMGRS PAG 0 OWNER=* ProcStep BBOCTL BBODAEMN BBOSR CPU/L/Z 6/ 5/ SORT=SysName/A JobID Owner STC01031 MDACRU STC01032 MDACRU STC01033 MDASRU Review any error messages and repair When the server finishes initialization, look in the server’s log files for any error messages, and examine the Integrated Solutions Console for the settings of various components. This is quite easy if you know what you are looking for. Examine the server logs on JES spool Using SDSF Display Active panel, select your application server’s started tasks with the PREfix or Select MD* commands and look in the control region (MDDMGR), and servant region (MDDMGRS). Select each one of them with the S action character, and look for any sever error messages with the FIND SEVERE ALL command. Hopefully, there will be none. If there are, note the number displayed by SDSF, and step through each one and start your problem determination process. We also find it useful to review any Java exceptions in the servant region joblogs. These exceptions do not always get flagged as severe errors, but that does not mean they are not important. 326 z/OS: WebSphere Business Process Management V6.2 Production Topologies Back up the stand-alone server’s ZFS data set At each major configuration milestone, and before proceeding, we find it a good idea to back up our configuration HFS. We used the JCL shown in Example 6-3 on page 306 to perform the back up. Example 6-5 JCL to back up the deployment manager node //MDDUMP JOB NOTIFY=&SYSUID,CLASS=A,MSGCLASS=A, // REGION=0M,TIME=1440 //* //* DUMP CONFIG HFS FOR DMGR //* //DUMP EXEC PGM=ADRDSSU //DASD DD DISP=(NEW,CATLG),DSN=WASCFG.MDCELL.DUMP.POSTFIX, // UNIT=3390,SPACE=(CYL,(500,100),RLSE) //SYSPRINT DD SYSOUT=* //SYSIN DD * DUMP DATASET( INCLUDE( WASCFG.MDCELL.MDDMGR.CONFIG.ZFS ) ) OUTDD(DASD) TOL(ENQF) OPTIMIZE(4) COMPRESS /* At this point the cell should be ready for additional nodes to be configured. See 6.3, “Install WebSphere Business Services Fabric for z/OS in a managed node” on page 327 for details. 6.3 Install WebSphere Business Services Fabric for z/OS in a managed node This section describes the steps for installing WebSphere Business Services Fabric into a managed node. Most of the initial configuration steps were completed when configuring a stand-alone server. These steps do not need to be repeated for additional nodes. 1. Copy the Fabric artifacts. See 5.2.1, “Copy Fabric artifacts” on page 207. 2. Configure security. See 5.2.2, “Configure security” on page 209. 3. Configure databases. See 5.2.3, “Configure databases” on page 214. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 327 The majority of the actual configuration for WebSphere Business Services Fabric for z/OS occurs during the profile augmentation process. This process is unique for each node and the deployment manager. We describe this process for a managed node in 6.3.1, “Augment WebSphere Process Server for z/OS managed node profile” on page 328. We then go through a series of verification steps to ensure the augmentation worked the way we expected it to. This process is described in 6.3.2, “Verify the augmentation process succeeded” on page 332. 6.3.1 Augment WebSphere Process Server for z/OS managed node profile Previous versions of WebSphere Business Services Fabric for z/OS required many manual configuration steps to properly setup and configure a WebSphere Process Server for z/OS server for Fabric. Improvements for V6.2 have removed most of these requirements. Most configuration for WebSphere Business Services Fabric for z/OS V.6.2 is accomplished through profile augmentation. We now describe how to augment an existing WebSphere Process Server for z/OS profile to contain WebSphere Business Services Fabric for z/OS functions. The augmentation process uses the manageprofiles.sh command shipped with base WebSphere Application Server for z/OS. Before augmenting a profile, ensure you have completed the following steps: 1. Make sure you have fully configured your WebSphere Application Server for z/OS and WebSphere Business Services Fabric for z/OS managed node environment. 2. Ensure you have completed the security requirements for your Fabric installation. See 5.2.2, “Configure security” on page 209. 3. Ensure you have created the required Fabric and message bus databases. See 5.2.3, “Configure databases” on page 214. 4. SMPE work for Fabric has been completed. 5. You ran the MDCPYFAB job or issued the copyfabric.sh script from a UNIX command shell to copy the Fabric profile templates to the proper locations in your WebSphere Application Server for z/OS configuration HFS. 6. You shut down the Servants associated with the profile you will augment. 7. You have enough space in the target HFS and in /tmp to complete the augmentation process. 328 z/OS: WebSphere Business Process Management V6.2 Production Topologies Note: The Fabric applications will require a lot more space in your ZFS file systems so they may need to extend during the deployment process. After installing Fabric, our deployment manager’s node ZFS occupied 1800 CYLs while each node occupied more than 2000 CYLs. To ensure a clean deployment, ensure that the volumes that host your cell’s ZFS datasets have enough free space to allow them to obtain secondary extents and grow to the sizes mentioned above. Using the manageprofiles.sh command The manageprofiles.sh command is located in the /WPS_HOME/bin directory. On our system the location for our WebSphere empty node is /wasv61config/mdcell/mdnodeb/AppServer/bin. The command used to augment the profile for WebSphere Business Services Fabric for z/OS uses similar parameters used in the base WebSphere Process Server installation as well as some additional Fabric specific parameters. Options for the manageprofiles.sh command are explained in “Using the manageprofiles.sh command” on page 221. Note that all parameters are case sensitive. Issue the manageprofiles.sh command We created two sample jobs, MDAGNODA (Example 6-6 on page 330), and MDAGNODB (Example 6-7 on page 331) to execute the manageprofiles command through the BPXBATCH interface. If you are creating two empty nodes, execute both MDAGNODA and MDAGNODB. If you are planning to federate your stand-alone server created previously, then only run the job for node B, MDAGNODB. We chose to use JCL rather than execute the command from a UNIX command shell so we could capture the output of the command in the joblog. Note: The values we specified on the manageprofiles.sh command are specific for our environment. Make sure you change the parameters to reflect your WebSphere Application Server for z/OS and WebSphere Process Server for z/OS installation. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 329 Note: Ensure you specify your deployment manager JMX™ SOAP port correctly on the manageprofiles command. We incorrectly specified the deployment manager admin port and our first manageprofiles command attempt failed. You can obtain this value from the spreadsheet and response file generated by the zPMT for your WebSphere Process Server for z/OS configuration. Refer to (2.2.2, “Using a spreadsheet to plan names for configuration objects” on page 45) for details. Example 6-6 Sample job MDAGNODA to execute the manageprofiles.sh command for node A //MDAGNODA JOB (0),CLASS=A,MSGCLASS=A,REGION=0M, // USER=MDADMIN,PASSWORD=MDADMIN, // NOTIFY=&SYSUID /*JOBPARM SYSAFF=SC42 //***********************************************************/ //* Configure Fabric in empty node node a */ //***********************************************************/ //INSTO EXEC PGM=IKJEFT01,REGION=0M,TIME=1440 //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BPXBATCH SH + export LIBPATH=/usr/lpp/db2/d9fg/db2910_jdbc/lib:$LIBPATH; + cd /wasv61config/mdcell/mdnodea/AppServer/bin; + ./manageprofiles.sh + -augment + -templatePath /wasv61config/mdcell/mdnodea/AppServer+ /profileTemplates/managed.wbsfabric + -profileName default + -cellName mdcell + -nodeName mdnodea + -serverName mdsr01b_WPS + -configureRESTSecurity true + -dmgrAdminUserName mdadmin + -dmgrAdminPassword mdadmin + -dmgrHost wtsc42.itso.ibm.com + -dmgrPort 20010 + -fabricDbCreateNew false + -fabricDbType DB2_UNIVERSAL + -fabricDbDriverType 2 + -fabricDbName DB9F + -fabricDbUser MDDBU + -fabricDbPassword MDDBU + -fabricDbServerPort 37893 + 330 z/OS: WebSphere Business Process Management V6.2 Production Topologies -fabricDbHostName wtsc42.itso.ibm.com + -fabricDbJDBCClasspath /usr/lpp/db2/d9fg/db2910_jdbc/classes + 1> /tmp/mdnodebFabricConfig_0A11.out + 2> /tmp/mdnodebFabricConfig_0A11.err; /* //***********************************************************/ //* STEPS TO COPY THE OUTPUT THE JOB LOG */ //***********************************************************/ //CPOUT EXEC PGM=IKJEFT01,REGION=0M //SYSEXEC DD DISP=SHR,DSN=BBO6142.MDNODEA.SBBOEXEC //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BBOHFSWR '/tmp/mdnodebFabricConfig_0A11.out' BBOHFSWR '/tmp/mdnodebFabricConfig_0A11.err' /* // Example 6-7 Sample job MDAGNODB to execute the manageprofiles.sh command for node B //MDAGNODB JOB (0),CLASS=A,MSGCLASS=A,REGION=0M, // USER=MDADMIN,PASSWORD=MDADMIN, // NOTIFY=&SYSUID /*JOBPARM SYSAFF=SC53 //***********************************************************/ //* Configure Fabric in empty node node b */ //***********************************************************/ //INSTO EXEC PGM=IKJEFT01,REGION=0M,TIME=1440 //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BPXBATCH SH + export LIBPATH=/usr/lpp/db2/d9fg/db2910_jdbc/lib:$LIBPATH; + cd /wasv61config/mdcell/mdnodeb/AppServer/bin; + ./manageprofiles.sh + -augment + -templatePath /wasv61config/mdcell/mdnodeb/AppServer+ /profileTemplates/managed.wbsfabric + -profileName default + -cellName mdcell + -nodeName mdnodeb + -serverName mdsr01b_WPS + -configureRESTSecurity true + -dmgrAdminUserName mdadmin + -dmgrAdminPassword mdadmin + -dmgrHost wtsc42.itso.ibm.com + Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 331 -dmgrPort 20010 + -fabricDbCreateNew false + -fabricDbType DB2_UNIVERSAL + -fabricDbDriverType 2 + -fabricDbName DB9F + -fabricDbUser MDDBU + -fabricDbPassword MDDBU + -fabricDbServerPort 37893 + -fabricDbHostName wtsc53.itso.ibm.com + -fabricDbJDBCClasspath /usr/lpp/db2/d9fg/db2910_jdbc/classes + 1> /tmp/mdnodebFabricConfig_0A11.out + 2> /tmp/mdnodebFabricConfig_0A11.err; /* //***********************************************************/ //* STEPS TO COPY THE OUTPUT THE JOB LOG */ //***********************************************************/ //CPOUT EXEC PGM=IKJEFT01,REGION=0M //SYSEXEC DD DISP=SHR,DSN=BBO6142.MDNODEA.SBBOEXEC //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BBOHFSWR '/tmp/mdnodebFabricConfig_0A11.out' BBOHFSWR '/tmp/mdnodebFabricConfig_0A11.err' /* // After you run the MDAGNODA and MDAGNODB jobs, check the joblog for the job. You should see the following message in the output for each job: INSTCONSUCCESS: Profile augmentation succeeded. To ensure the augmentation process did actually do what it was supposed to, continue with 6.3.2, “Verify the augmentation process succeeded” on page 332 below. 6.3.2 Verify the augmentation process succeeded The profile augmentation process creates several artifacts with the WebSphere Application Server for z/OS managed node profile. We can verify what was created by using the WebSphere Application Server for z/OS administrative console. 332 z/OS: WebSphere Business Process Management V6.2 Production Topologies Start the deployment manager Using the MVS console, or SDSF TSO panels, issue the start command for the deployment manager. On our system, the command was as follows: START MDMGCR,JOBNAME=MDDMGR,ENV=MDCELL.MDDMNODE.MDDMGR Make sure the three address spaces start up. See Example 6-8. Example 6-8 SDSF DA panel for a deployment manager SDSF DA SC42 SC* COMMAND INPUT ===> PREFIX=MD* DEST=(ALL) NP JOBNAME StepName MDDMGR MDDMGR MDDEMN MDDEMN MDDMGRS MDDMGRS PAG 0 OWNER=* ProcStep BBOCTL BBODAEMN BBOSR CPU/L/Z 6/ 5/ SORT=SysName/A JobID Owner STC01031 MDACRU STC01032 MDACRU STC01033 MDASRU Review any error messages and repair When the server finishes initialization, look in the server’s log files for any error messages, and examine the Integrated Solutions Console for the settings of various components. This is quite easy if you know what you are looking for. Examine the server logs on JES spool Using SDSF Display Active panel, select your application server’s started tasks with the PREfix or Select MD* commands and look in the control region (MDDMGR), and servant region (MDDMGRS). Select each one of them with the S action character, and look for any sever error messages with the FIND SEVERE ALL command. Hopefully, there will be none. If there are, note the number displayed by SDSF, and step through each one and start your problem determination process. We also find it useful to review any Java exceptions in the servant region joblogs. These exceptions do not always get flagged as severe errors, but that does not mean they are not important. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 333 Back up the cell’s ZFS data sets At each major configuration milestone, and before proceeding, we find it a good idea to back up our configuration HFS. Now that we have augmented nodes A and B, we need to dump all three HFS’s: For the deployment manager: WASCFG.MDCELL.MDDMNODE.CONFIG.ZFS For node A: WASCFG.MDCELL.MDNODEA.CONFIG.ZFS For node B: WASCFG.MDCELL.MDNODEB.CONFIG.ZFS We used the JCL shown in Example 6-9 to perform the back up. Example 6-9 JCL to back up the deployment manager and nodes A and B //MDDUMP3S JOB NOTIFY=&SYSUID,CLASS=A,MSGCLASS=A, // REGION=0M,TIME=1440 /*JOBPARM S=SC42 //* DUMP CONFIG HFS FOR DMGR AND BOTH NODES //DUMP EXEC PGM=ADRDSSU //DMGR DD DISP=(NEW,CATLG),DSN=WASCFG.MDCELL.DUMPDMGR.MAR7, // UNIT=3390,SPACE=(CYL,(1000,100),RLSE),VOL=SER=TSTO61 //NODEA DD DISP=(NEW,CATLG),DSN=WASCFG.MDCELL.DUMPNODA.MAR7, // UNIT=3390,SPACE=(CYL,(1000,100),RLSE),VOL=SER=TSTO63 //NODEB DD DISP=(NEW,CATLG),DSN=WASCFG.MDCELL.DUMPNODB.MAR7, // UNIT=3390,SPACE=(CYL,(1000,100),RLSE),VOL=SER=TSTO64 //SYSPRINT DD SYSOUT=* //SYSIN DD * DUMP DATASET( INCLUDE( WASCFG.MDCELL.MDDMNODE.CONFIG.ZFS ) ) OUTDD(DMGR) TOL(ENQF) OPTIMIZE(4) COMPRESS DUMP DATASET( INCLUDE( WASCFG.MDCELL.MDNODEA.CONFIG.ZFS ) ) OUTDD(NODEA) TOL(ENQF) OPTIMIZE(4) COMPRESS WASCFG.MDCELL.MDNODEB.CONFIG.ZFS - DUMP DATASET( INCLUDE( ) ) OUTDD(NODEB) TOL(ENQF) OPTIMIZE(4) COMPRESS - /* //OTHER DD * 334 z/OS: WebSphere Business Process Management V6.2 Production Topologies Log in to administrative console Next, log in to the admin server for the deployment manager. See Figure 6-21. Figure 6-21 Login to deployment manager admin console Verify that the Fabric resources were created properly by the profile augmentation process. Verify the Fabric JDBC path variable was set The JDBC provider specified in the manageprofiles.sh command is listed as a JDBC driver for the server being augmented. This JDBC provider is present on z/OS servers by default and is not created as a result of the profile augmentation. To verify the JDBC path variable, perform the following steps: 1. In the Integrated Solutions Console, navigate to Resources → JDBC. 2. Click JDBC Providers. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 335 A list of JDBC Providers is displayed. The datasource we specified on the manageprofiles.sh command, DB2 Universal JDBC Driver Provider, is in the list. See Figure 6-22. Figure 6-22 JDBC Providers on managed node B 336 z/OS: WebSphere Business Process Management V6.2 Production Topologies Verify the Fabric .jar files were copied The following two .jar files should be copied to the plugins directory of the WebSphere Process Server for z/OS managed node B. com.ibm.ws.repository_6.2xxx.jar com.ibm.ws.fabric.catalog_6.2.0.jar See Figure 6-23. Figure 6-23 Fabric jar files in plugins directory of managed node B Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 337 Verify the Fabric SCA jars were copied Verify the following jars were copied to the lib/ext directory of the WebSphere Process Server for z/OS managed node B. fabric-da-api.jar fabric-da-model.jar fabric-da-sca.jar fabric-da-scdl.jar fabric-types.jar See Figure 6-24. Figure 6-24 Fabric SCA jars in the /lib/ext directory of the custom node 6.3.3 Post augmentation tasks There are several tasks that must be completed after profile augmentation to complete the configuration of Fabric on the stand-alone server. These steps are outlined below. 338 z/OS: WebSphere Business Process Management V6.2 Production Topologies Update xml file for Business Space As part of the configuration for Business Space powered by WebSphere, you must specify endpoints for all of the configured services. The configuration for WebSphere Business Services Fabric for z/OS is contained in the wbsfEndpoints XML file contained in the profile template used to configure Fabric. On our system, after profile augmentation for the managed node B, this file is located in /wasv61config/mdcell/mdnodeb/AppServer/profiles/default/BusinessSpace/regist ryData/wbsfEndpoints.xml. You must modify this file to contain the URL of the server where WebSphere Business Services Fabric for z/OS is deployed. Refer to Example 6-10. Note that this is an ASCII file and must be edited through an ASCII editor. Example 6-10 wbsfEndpoints XML file for Business Space powered by WebSphere configuration <?xml version="1.0" encoding="UTF-8" ?> - <!-START NON-TRANSLATABLE --> - <tns:BusinessSpaceRegistry xmlns:tns="http://com.ibm.bspace/BusinessSpaceRegistry" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://com.ibm.bspace/BusinessSpaceRegistry BusinessSpaceRegistry.xsd"> - <tns:Endpoint> <tns:id>{com.ibm.bspace}bspaceBusinessVariableServiceRootId</tns:id> <tns:version>1.0.0.0</tns:version> <tns:url>https://wtsc53.itso.ibm.com:20049/fabricrest</tns:url> <tns:description>Location of Business Space Business Policy server</tns:description> </tns:Endpoint> </tns:BusinessSpaceRegistry> 6.4 Verify the WebSphere Business Services Fabric for z/OS basic clustered node operation Before we can proceed with the remainder of the cluster configuration, we must first start the cluster and the servers in the clustered nodes and verify their basic operation. We outline this process below in the sections that follow. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 339 6.4.1 Start cluster node agents Before the clustered servers can be started, start the node agents on each node. Start the node agents Using the MVS console, or SDSF TSO panels, issue the start command for both node agents. On our system, the command was as follows: For node A START MDACRA,JOBNAME=MDAGNTA,ENV=MDCELL.MDNODEA.MAGNTA For node B SC53 START MDACRB,JOBNAME=MDAGNTB,ENV=MDCELL.MDNODEB.MAGNTB Make sure the deployment manager address space is up, and that three additional address spaces start up. See Example 6-11. Example 6-11 SDSF DA panel for a deployment manager SDSF DA SC42 SC* COMMAND INPUT ===> PREFIX=MD* DEST=(ALL) NP JOBNAME StepName MDDMGR MDDMGR MDDEMN MDDEMN MDDMGRS MDDMGRS MDAGNTA MDAGNTA MDDEMN MDDEMN MDAGNTB MDAGNTB PAG 0 OWNER=* ProcStep BBOCTL BBODAEMN BBOSR BBOCTL BBODAEMN BBOCTL CPU/L/Z 3/ 3/ SORT=SysName/A JobID Owner STC01031 MDACRU STC01032 MDACRU STC01033 MDASRU STC01534 MDACRU STC01549 MDACRU STC01548 MDACRU Review any error messages and repair When the server finishes initialization, look in the server’s log files for any error messages, and examine the Integrated Solutions Console for the settings of various components. This is quite easy if you know what you are looking for. Examine the server logs on JES spool Using the SDSF Display Active panel, select your application server’s started tasks with the PREfix or Select MD* commands and look in the node agents control region (MDAGNTA), and (MDAGNTB). Notice there is no servant region for the node agents. Node agents only have a control region. 340 z/OS: WebSphere Business Process Management V6.2 Production Topologies Select each one of them with the S action character, and look for any sever error messages with the FIND SEVERE ALL command. Hopefully, there will be none. If there are, note the number displayed by SDSF, and step through each one and start your problem determination process. We also find it useful to review any Java exceptions in the servant region joblogs. These exceptions do not always get flagged as severe errors, but that does not mean they are not important. Back up the cell’s ZFS data sets At each major configuration milestone, and before proceeding, we find it a good idea to back up our configuration HFS. Now that we have augmented nodes A and B, we need to dump all three HFS’s: For the deployment manager: WASCFG.MDCELL.MDDMNODE.CONFIG.ZFS For node A: WASCFG.MDCELL.MDNODEA.CONFIG.ZFS For node B: WASCFG.MDCELL.MDNODEB.CONFIG.ZFS We used the JCL shown in Example 6-12 to perform the back up. Example 6-12 JCL to back up the deployment manager and nodes A and B //MDDUMP3S JOB NOTIFY=&SYSUID,CLASS=A,MSGCLASS=A, // REGION=0M,TIME=1440 /*JOBPARM S=SC42 //* DUMP CONFIG HFS FOR DMGR AND BOTH NODES //DUMP EXEC PGM=ADRDSSU //DMGR DD DISP=(NEW,CATLG),DSN=WASCFG.MDCELL.DUMPDMGR.MAR7, // UNIT=3390,SPACE=(CYL,(1000,100),RLSE),VOL=SER=TSTO61 //NODEA DD DISP=(NEW,CATLG),DSN=WASCFG.MDCELL.DUMPNODA.MAR7, // UNIT=3390,SPACE=(CYL,(1000,100),RLSE),VOL=SER=TSTO63 //NODEB DD DISP=(NEW,CATLG),DSN=WASCFG.MDCELL.DUMPNODB.MAR7, // UNIT=3390,SPACE=(CYL,(1000,100),RLSE),VOL=SER=TSTO64 //SYSPRINT DD SYSOUT=* //SYSIN DD * DUMP DATASET( INCLUDE( WASCFG.MDCELL.MDDMNODE.CONFIG.ZFS ) ) OUTDD(DMGR) TOL(ENQF) OPTIMIZE(4) COMPRESS DUMP DATASET( INCLUDE( WASCFG.MDCELL.MDNODEA.CONFIG.ZFS - Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 341 ) ) OUTDD(NODEA) TOL(ENQF) OPTIMIZE(4) COMPRESS WASCFG.MDCELL.MDNODEB.CONFIG.ZFS - DUMP DATASET( INCLUDE( ) ) OUTDD(NODEB) TOL(ENQF) OPTIMIZE(4) COMPRESS - /* //OTHER DD * 6.4.2 Start clustered servers Once the node agents have been started, we log in into the deployment manager administrative console and start the servers in the cluster. We outline this process below. 1. Log in to the Integrated Solutions Console. 2. In the deployment manager console, navigate to Servers → Application servers. 3. Click Application servers. 4. Select the msr01a_WPS and mdsr01b_WPS check boxes. 5. Click Start. See Figure 6-25. Figure 6-25 Start managed servers mdsr01a, and mdsr01b 342 z/OS: WebSphere Business Process Management V6.2 Production Topologies Wait for the servers to come up. See Figure 6-26 Figure 6-26 Successful start of servers mdsr01a and mdsr01b We are now ready to proceed to the last part of the configuration of WebSphere Business Services Fabric for z/OS (Figure 6.5). 6.5 Complete cluster configuration for WebSphere Business Services Fabric for z/OS Additional configuration is required to set up WebSphere Business Services Fabric for z/OS in a cluster. Prior to executing the steps below, ensure the following circumstances: 1. You have completed configuration of a WebSphere Process Server for z/OS network deployment cell with a deployment manager and two managed nodes. 2. You have created a cluster with two managed nodes, one on system wtsc42, and one on system wtsc53. Both nodes have one server configured. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 343 3. You have augmented the deployment manager and managed nodes with WebSphere Business Services Fabric for z/OS components as referenced in the following sections: – 5.2, “Install WebSphere Business Services Fabric for z/OS in a standalone node” on page 206 – 6.2, “Install WebSphere Business Services Fabric for z/OS in the deployment manager” on page 301 – 6.3, “Install WebSphere Business Services Fabric for z/OS in a managed node” on page 327 4. The deployment manager, the managed nodes, and their node agents have been started. 5. The cluster and participating servers have been started. When configuring a WebSphere Process Server for z/OS cell for a clustered WebSphere Business Services Fabric for z/OS implementation, you have a choice to make: If you chose to create empty managed nodes and federate them into your cluster, you must create all of the WebSphere Business Services Fabric for z/OS resources for the cluster. We detail the steps required in 6.6, “Creating a WebSphere Business Services Fabric for z/OS cluster configuration with empty managed nodes” on page 345. If you chose to first create a stand-alone server and federate it into your cluster, most of the critical WebSphere Business Services Fabric for z/OS resources were created for you at the stand-alone server level. In this case, the process for completing the cluster configuration involves changing these references to be at the cluster level so all nodes can access them. In addition, there are a few more resources that need to be defined for the cluster. For details on this process, see 6.7, “Configuring a WebSphere Business Services Fabric for z/OS cluster from a stand-alone node” on page 361. 344 z/OS: WebSphere Business Process Management V6.2 Production Topologies 6.6 Creating a WebSphere Business Services Fabric for z/OS cluster configuration with empty managed nodes We review our general approach for creating the cluster using empty nodes: 1. We created a deployment manager (Chapter 2, “Configuring a WebSphere Process Server for z/OS standalone server” on page 39) and augmented it to include WebSphere Process Server for z/OS (5.2.4, “Augment WebSphere Process Server for z/OS standalone node profile” on page 220.) 2. We augmented the deployment manager with WebSphere Business Services Fabric for z/OS as described in 6.2, “Install WebSphere Business Services Fabric for z/OS in the deployment manager” on page 301. 3. We created two nodes and augmented them with WebSphere Process Server for z/OS as described in Chapter 4, “Creating additional WebSphere Process Server nodes” on page 179. 4. We federated both nodes into the cell as described in 4.4, “Federating the new node into the network deployment cell” on page 188. 5. We created a cluster with both empty nodes as described in 4.5, “Creating an additional cluster member” on page 189. 6. We augmented both nodes with WebSphere Business Services Fabric for z/OS as described in 6.3, “Install WebSphere Business Services Fabric for z/OS in a managed node” on page 327. 7. We executed the WebSphere Business Services Fabric for z/OS python scripts as described in 6.6.1, “Running the post augmentation WebSphere Business Services Fabric for z/OS python configuration scripts” on page 345. 8. We observed the fallout and proceeded with verification steps. 6.6.1 Running the post augmentation WebSphere Business Services Fabric for z/OS python configuration scripts There are two scripts that must be run to complete configuration for the WebSphere Business Services Fabric for z/OS cluster: fabricSIBConfig.py (6.6.2, “Run fabricSIBConfig.py” on page 346) fabricSIBConfig.py sets up the messaging engine resources for Fabric in the cluster. fabricAppDeploy.py (6.6.4, “Run fabricAppDeploy.py” on page 354) fabricAppDeploy.py deploys the required Fabric applications and support functions to the nodes in the cluster. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 345 Both scripts are executed through the wsadmin.sh interface against the deployment manager. 6.6.2 Run fabricSIBConfig.py We created sample job MDCSIBPY (Example 6-13) to execute the fabricSIBConfig.py script through the BPXBATCH interface. We chose to use JCL rather than execute the script from a UNIX command shell so we could capture the output of the script in the joblog. Note: The values we specified on the fabricSIBConfig.py invocation are specific for our environment. Make sure you change the parameters to reflect your WebSphere Business Services Fabric for z/OS installation. Example 6-13 MDCSIBPY JCL job invocation //MDCSIBPY JOB (0),CLASS=A,MSGCLASS=A,REGION=0M, // USER=MDADMIN,PASSWORD=MDADMIN, // NOTIFY=GATES /*JOBPARM SYSAFF=SC42 //***********************************************************/ //* Configure Fabric SIB buses in the cluster */ //***********************************************************/ //INSTO EXEC PGM=IKJEFT01,REGION=0M,TIME=1440 //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BPXBATCH SH + cd /wasv61config/mdcell/mddmnode/DeploymentManager/bin; + ./wsadmin.sh + -user MDADMIN + -password MDADMIN + -f /wasv61config/mdcell/mddmnode/DeploymentManager+ /profileTemplates/dmgr.wbsfabric/actions/scripts/cluster+ /fabricSIBConfig.py + -cluster MDSR1 + -useSCASystemBusProperties Yes + -schemaName MD01F + 1> /tmp/mddmnodeSIBpy_0A11.out + 2> /tmp/mddmnodeSIBpy_0A11.err; /* //***********************************************************/ //* STEPS TO COPY THE OUTPUT THE JOB LOG */ //***********************************************************/ //CPOUT EXEC PGM=IKJEFT01,REGION=0M //SYSEXEC DD DISP=SHR,DSN=BBO6142.MDDMNODE.SBBOEXEC //SYSTSPRT DD SYSOUT=* 346 z/OS: WebSphere Business Process Management V6.2 Production Topologies //SYSTSIN DD * BBOHFSWR '/tmp/mddmnodeSIBpy_0A11.out' BBOHFSWR '/tmp/mddmnodeSIBpy_0A11.err' /* // 6.6.3 Verify fabricSIBConfig.py completed successfully We verify the following resources were correctly created for the cluster. Verify fabric.mdcell.bus is created 1. In the deployment manager console, navigate to Service integration → Buses. 2. Click fabric.mdcell.bus. See Figure 6-27. Figure 6-27 Verify fabric.mdcell.bus Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 347 Verify Bus members are targeted to cluster 1. In the deployment manager console, navigate to Service integration → Buses. 2. Click fabric.mdcell.bus. 3. Click Bus members. See Figure 6-28. Figure 6-28 Verify bus members 348 z/OS: WebSphere Business Process Management V6.2 Production Topologies Verify Bus topics 1. In the deployment manager console, navigate to Service integration → Buses. 2. Click fabric.mdcell.Bus. 3. Click Destinations. 4. Verify the following topics are available: – DA.Event.Topic – Hub.Request.Queue See Figure 6-29. Figure 6-29 Verify destinations Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 349 Verify Security for fabric.mdcell.bus 1. In the deployment manager console, navigate to Service integration → Buses. 2. Click fabric.mdcell.bus. 3. Click Security. 4. Verify the Enable bus security check box is selected. 5. Verify the authentication alias is that same as that used for the SCA system bus. See Figure 6-30. Figure 6-30 Verify bus security 350 z/OS: WebSphere Business Process Management V6.2 Production Topologies Verify Bus connector role 1. In the deployment manager console, navigate to Service integration → Buses. 2. Click fabric.mdcell.bus. 3. Click Security. 4. Click Users and groups in the bus security role. 5. Verify role definitions are the same as those defined for the SCA system bus. See Figure 6-31. Figure 6-31 Verify bus connector roles Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 351 Verify messaging engine is created 1. In the deployment manager console, navigate to Service integration → Buses. 2. Click fabric.mdcell.bus. 3. Click Messaging engines. See Figure 6-32. Figure 6-32 Verify messaging engine 352 z/OS: WebSphere Business Process Management V6.2 Production Topologies Verify Fabric messaging engine message store 1. In the deployment manager console, navigate to Service integration → Buses. 2. Click fabric.mdcell.bus. 3. Click Messaging engines. 4. Click MDRS1.000-fabric.mdcell.bus. 5. Click Message store. 6. Verify the following: a. Verify the schema name is set to MD01F b. The Create tables check box is not selected. c. The authentication alias is that same as that used for the SCA system bus. See Figure 6-33. Figure 6-33 Verify message store Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 353 6.6.4 Run fabricAppDeploy.py We created sample job MDCAPPPY (Example 6-14) to execute the fabricAppDeploy.py script through the BPXBATCH interface. We chose to use JCL rather than execute the script from a UNIX command shell so we could capture the output of the script in the joblog. Note: The values we specified on the fabricAppDeploy.py invocation are specific for our environment. Make sure you change the parameters to reflect your WebSphere Business Services Fabric for z/OS installation. Example 6-14 MCCAPPPY JCL job invocation //MDCAPPPY JOB (0),CLASS=A,MSGCLASS=A,REGION=0M, // USER=MDADMIN,PASSWORD=MDADMIN, // NOTIFY=GATES /*JOBPARM SYSAFF=SC42 //***********************************************************/ //* Configure Fabric applications in the cluster */ //***********************************************************/ //INSTO EXEC PGM=IKJEFT01,REGION=0M,TIME=1440 //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BPXBATCH SH + cd /wasv61config/mdcell/mddmnode/DeploymentManager/bin; + ./wsadmin.sh + -user MDADMIN + -password MDADMIN + -f /wasv61config/mdcell/mddmnode/DeploymentManager+ /profileTemplates/dmgr.wbsfabric/actions/scripts/cluster+ /fabricAppDeploy.py -cluster MDSR1 + -toolsUserid MDBSPADM + -dmgrPath /wasv61config/mdcell/mddmnode/DeploymentManager+ /profileTemplates/dmgr.wbsfabric + 1> /tmp/mddmnodeApppy_0A11.out + 2> /tmp/mddmnodeApppy_0A11.err; /* //***********************************************************/ //* STEPS TO COPY THE OUTPUT THE JOB LOG */ //***********************************************************/ //CPOUT EXEC PGM=IKJEFT01,REGION=0M //SYSEXEC DD DISP=SHR,DSN=BBO6142.MDDMNODE.SBBOEXEC //SYSTSPRT DD SYSOUT=* 354 z/OS: WebSphere Business Process Management V6.2 Production Topologies //SYSTSIN DD * BBOHFSWR '/tmp/mddmnodeApppy_0A11.out' BBOHFSWR '/tmp/mddmnodeApppy_0A11.err' /* // 6.6.5 Verify fabricAppDeploy.py completed successfully We verify the following resources were correctly created for the cluster. Verify enterprise applications are deployed to the dmgr This section describes the steps to verify the installation and configuration of WebSphere Business Services Fabric. 1. Log in to the Integrated Solutions Console. 2. In the deployment manager console, navigate to Applications → Enterprise applications. 3. Click Enterprise applications. 4. Verify the Fabric applications are started as shown in Figure 6-34. Figure 6-34 Fabric application .ears deployed on managed node Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 355 Verify application status is mapped to cluster Perform the following steps to verify the application status is mapped to cluster: 1. Log in to the Integrated Solutions Console. 2. In the deployment manager console navigate to Applications → Enterprise applications. 3. Click Enterprise applications. Perform the following for each of the Fabric EAR files: a. Click the ear file. b. Click Target specific application status. c. Verify target is the cluster MDSR1. See Figure 6-35. Figure 6-35 Verify application status target is cluster 356 z/OS: WebSphere Business Process Management V6.2 Production Topologies Verify JMS resources defined at the cluster level Perform the following steps to verify JMS resources defined at the cluster level: 1. In the deployment manager console, navigate to Resources → JMS → JMS providers. 2. Click Default messaging provider for the cluster MDSR1. 3. Verify the JMS resources were correct defined as follows: a. Click Connection factories. i. Click DA Event Connection Factory. ii. Ensure resource is scoped to the cluster, MDSR1 See Figure 6-36. Figure 6-36 Verify connection factory resource is defined at cluster level b. Click Queue. c. Click Hub Request Queue. d. Ensure resource is scoped to the cluster, MDSR1. See Figure 6-37 on page 358. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 357 Figure 6-37 Verify queue resource is defined at the cluster level 358 z/OS: WebSphere Business Process Management V6.2 Production Topologies Verify name space bindings are created at the cluster level The following steps verify name space bindings are created at the cluster level: 1. In the Integrated Solutions Console, navigate to Environment naming. 2. Click Name Space Bindings. See Figure 6-38. Figure 6-38 Verify namespace bindings Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 359 Verify Replication domain is created 1. In the deployment manager console, navigate to Environment → Replication domains. 2. Click Replication domains. See Figure 6-39. Figure 6-39 Verify replication domain 360 z/OS: WebSphere Business Process Management V6.2 Production Topologies Verify Fabric Object Cache instances are created 1. In the deployment manager console, navigate to Resources → Cache instances. 2. Click Object cache instances. 3. Verify all Object cache instances are created at the cluster level (Figure 6-40). Figure 6-40 Verify object cache instances 6.7 Configuring a WebSphere Business Services Fabric for z/OS cluster from a stand-alone node We found the process for federating the stand-alone node into the cell to be complicated and not entirely intuitive. After augmenting the deployment manager and federating the stand-alone node and node B into the cluster, we found that the augmentation process did not create all of the Fabric cluster resources correctly. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 361 Our general approach was as follows: 1. We created a stand-alone node and augmented it to include WebSphere Process Server for z/OS. 2. We augmented the stand-alone node with WebSphere Business Services Fabric for z/OS. 3. We created a deployment manager (Chapter 2, “Configuring a WebSphere Process Server for z/OS standalone server” on page 39) and augmented it to include WebSphere Process Server for z/OS (5.2.4, “Augment WebSphere Process Server for z/OS standalone node profile” on page 220). 4. We augmented the deployment manager with WebSphere Business Services Fabric for z/OS as described in 6.2, “Install WebSphere Business Services Fabric for z/OS in the deployment manager” on page 301. 5. We federated the stand-alone node into the cell as described in 3.4, “Federating the standalone server into this ND cell” on page 165. 6. We created an additional node and augmented it with WebSphere Process Server for z/OS as described in Chapter 4, “Creating additional WebSphere Process Server nodes” on page 179. 7. We federated the additional node into the cell as described in 4.4, “Federating the new node into the network deployment cell” on page 188. 8. We augmented the additional node with WebSphere Business Services Fabric for z/OS as described in 6.3, “Install WebSphere Business Services Fabric for z/OS in a managed node” on page 327. 9. We clustered the stand-alone node and the empty node as described in 4.5, “Creating an additional cluster member” on page 189. 10.We observed the fallout from our approach and proceeded to fix the configuration. We describe below what we found and what we had to do to the configuration to fix the Fabric definitions. Because you will be changing the cell configuration, make sure you stop the node agents and the clustered servers before you begin this process. 362 z/OS: WebSphere Business Process Management V6.2 Production Topologies Verify fabric.mdbasea.bus is created 1. In the deployment manager console, navigate to Service integration → Buses. 2. Click Fabric.mdbasea.Bus. See Figure 6-41. Figure 6-41 Bus created with incorrect name Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 363 Verify Bus members are targeted to cluster 1. In the deployment manager console, navigate to Service integration → Buses. 2. Click Fabric.mdbasea.Bus. 3. Click Bus members. See Figure 6-42. Figure 6-42 Bus member not targeted to cluster Notice that the mdnodeA:mdsr01a_WPS bus member is defined at the server scope. This resource was created as a result of federating the stand-alone node. It must be recreated at the cluster level so it can be used by all nodes in the cluster. We will create this bus member with the same parameters as the existing server level bus. 4. Click Add. 5. Click Cluster. 364 z/OS: WebSphere Business Process Management V6.2 Production Topologies 6. Take the default of cluster MDSR1, from the drop-down menu. See Figure 6-43. Figure 6-43 Specify a new bus member at the cluster level 7. Click Next. 8. Click Data store. See Figure 6-44 Figure 6-44 Specify message bus data store 9. Click Next. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 365 10.Specify the following values: – Data source JNDI Name: jdbc/fabricme/mdnodea.mdsr01a_WPS.Fabric.mdbasea.Bus – Schema name: MD01F – Authentication alias: SCASYSME00_Auth_Alias 11.Clear the Create Tables check box. See Figure 6-45. Figure 6-45 Specify message store properties 12.Click Next. Review values (Figure 6-46). Figure 6-46 Review bus member properties 13.Click Finish and click Save. 366 z/OS: WebSphere Business Process Management V6.2 Production Topologies 14.Verify the new cluster level bus member is created (Figure 6-47). Figure 6-47 New cluster level bus engine Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 367 Verify bus topics 1. In the deployment manager console, navigate to Service integration → Buses. 2. Click Fabric.mdbasea.Bus. 3. Click Destinations. 4. Verify the following topics are available: – DA.Event.Topic – Hub.Request.Queue See Figure 6-48. Figure 6-48 Verify Fabric Bus topics 368 z/OS: WebSphere Business Process Management V6.2 Production Topologies Verify security for Fabric.mdbasea.Bus 1. In the deployment manager console, navigate to Service integration → Buses. 2. Click Fabric.mdbasea.Bus. 3. Click Security. 4. Verify the following: a. The Enable bus security check box is selected. b. The authentication alias is that same as that used for the SCA system bus. See Figure 6-49. Figure 6-49 Verify Bus security Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 369 Verify bus connector role 1. In the deployment manager console, navigate to Service integration → Buses. 2. Click Fabric.mdbasea.Bus. 3. Click Security. 4. Click Users and groups in the bus security role. 5. Verify role definitions are the same as those defined for the SCA system bus. See Figure 6-50. Figure 6-50 Verify Bus security roles 370 z/OS: WebSphere Business Process Management V6.2 Production Topologies Verify messaging engine is created 1. In the deployment manager console, navigate to Service integration → Buses. 2. Click Fabric.mdbasea.Bus. 3. Click Messaging engines. See Figure 6-51. Figure 6-51 Verify message engine created Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 371 Verify Fabric messaging engine message store 1. In the deployment manager console, navigate to Service integration → Buses. 2. Click Fabric.mdbasea.Bus. 3. Click Messaging engines. 4. Click mdnodea.mdsr01a WPS-Fabric.mdbase.Bus. 5. Click Message store. 6. Verify the following: a. Verify the schema name is set to MD01F. b. The Create tables check box is cleared. c. The authentication alias is that same as that used for the SCA system bus. See Figure 6-52. Figure 6-52 Verify message engine data store 372 z/OS: WebSphere Business Process Management V6.2 Production Topologies Verify enterprise applications are deployed to the dmgr This section describes the steps to verify the installation and configuration of WebSphere Business Services Fabric. 1. Log in to the Integrated Solutions Console 2. In the deployment manager console, navigate to Applications → Enterprise applications. 3. Click Enterprise applications. 4. Verify the Fabric applications are started as shown in Figure 6-53. Figure 6-53 Fabric application .ears deployed on managed node Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 373 Verify application status is mapped to cluster 1. Log in to the Integrated Solutions Console. 2. In the deployment manager console, navigate to Applications → Enterprise applications. 3. Click Enterprise applications. Perform the following for each of the Fabric EAR files: a. Click the ear file. b. Click Target specific application status. c. Verify target is the cluster MDSR1. See Figure 6-54. Figure 6-54 Verify application status target is cluster 374 z/OS: WebSphere Business Process Management V6.2 Production Topologies Verify JMS resources defined at the cluster level 1. In the deployment manager console, navigate to Resources → JMS → JMS providers. 2. Click Default messaging provider for the cluster MDSR1. 3. Verify the JMS resources were correct defined as follows: a. Click Connection factories i. Click DA Event Connection Factory. ii. Ensure resource is scoped to the cluster, MDSR1. See Figure 6-55. Figure 6-55 Verify connection factory resource is defined at cluster level Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 375 b. Click Queue. i. Click Hub Request Queue. ii. Ensure resource is scoped to the cluster, MDSR1. See Figure 6-56. Figure 6-56 Verify queue resource is defined at the cluster level 376 z/OS: WebSphere Business Process Management V6.2 Production Topologies c. Click Topic. i. Click DA Event Topic. ii. Ensure resource is scoped to the cluster, MDSR1. See Figure 6-57. Figure 6-57 Verify topic resource is defined at the cluster level Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 377 d. Click Activation specification. i. Click DA Perfmon Activation. ii. Ensure resource is scoped to the cluster, MDSR1. See Figure 6-58. Figure 6-58 Verify activation specification 378 z/OS: WebSphere Business Process Management V6.2 Production Topologies iii. Click Hub Event Activation. iv. Ensure resource is scoped to the cluster, MDSR1. See Figure 6-59. Figure 6-59 Verify activation specification Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 379 v. Click Hub Request Activation. vi. Ensure resource is scoped to the cluster, MDSR1. See Figure 6-60. Figure 6-60 Verify activation specification 380 z/OS: WebSphere Business Process Management V6.2 Production Topologies Verify name space bindings are created at the cluster level 1. In the deployment manager console, navigate to Environment naming. 2. Click Name Space Bindings. 3. See Figure 6-61. Figure 6-61 Namespace bindings Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 381 There are no Name Space Bindings created. Perform the following steps to create them: 1. Ensure scope is set to cluster MDSR1. 2. Click New. 3. Select the String check box. See Figure 6-62. Figure 6-62 Add a new namespace binding 4. Click Next. 382 z/OS: WebSphere Business Process Management V6.2 Production Topologies 5. Specify the following parameters: – Binding identifier: wbsf.da.executionTrace.viewer.enable – Name in namespace: wbsf.da.executionTrace.viewer.enable – String value: True See Figure 6-63. Figure 6-63 Specify namespace binding properties 6. Click Next. See Figure 6-65 on page 384. Figure 6-64 Summary panel 7. Verify namespace properties. 8. Click Finish. 9. Click Save. 10.Click New. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 383 11.Select the String check box. See Figure 6-65. Figure 6-65 Add a new namespace binding 12.Click Next. 13.Specify the following parameters: – Binding identifier: wbsf.da.executionTrace.maxcount – Name in namespace: wbsf.da.executionTrace.maxcount – String value: 100 See Figure 6-66. Figure 6-66 Specify namespace binding properties 384 z/OS: WebSphere Business Process Management V6.2 Production Topologies 14.Click Next. See Figure 6-67. Figure 6-67 Verify namespace properties 15.Click Finish. 16.Click Save. 17.Verify the namespaces have been created (Figure 6-68). Figure 6-68 Verify namespace bindings Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 385 Verify Replication domain is created for the context replication In the deployment manager console, perform the following steps: 1. Navigate to Environment → Replication domains. 2. Click Replication domains. See Figure 6-69. Figure 6-69 No replication domains defined 386 z/OS: WebSphere Business Process Management V6.2 Production Topologies 3. Notice there are no Replication domains created. We will create them now: a. Click New. b. Specify the following parameters: • • • • Name: WBSF DA Replication Request timeout: 5 Encryption type: None Number of replicas: Select Entire Domain See Figure 6-70. Figure 6-70 Create new replication domain Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 387 4. Click OK. 5. Click Save. See Figure 6-71. Figure 6-71 Verify new replication domain Verify Fabric Object Cache instances are created In the deployment manager console, perform the following steps: 1. Navigate to Resources → Cache instances. 2. Click Object cache instances. See Figure 6-72. Figure 6-72 Verify object cache instances are created at the node level 388 z/OS: WebSphere Business Process Management V6.2 Production Topologies Notice that the object cache instances are all defined at the node scope. These resources were created as a result of federating the stand-alone node. They must be recreated at the cell level so they can be used by all nodes in the cell. We will create these resources with the same parameters as the existing node level resources. 3. Change Scope to cell=mdcell. See Figure 6-73. Figure 6-73 Create new cell level cache instances 4. Create cell level Object instance for Fabric Context Cache a. Click New. b. Specify the following parameters: • • • • • • • • • Name: Fabric Context Cache JNDI name: services/cache/wbsf.contexts Cache size: 2000 Default priority: 1 Disk Cache settings: Enable disk offload Consistency settings: Enable cache replication Full group replication domain: WBSF DA Replication Replication type: Both push and pull Push frequency: 0 seconds See Figure 6-74 on page 390 and Figure 6-75 on page 391. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 389 Figure 6-74 Create cell level Fabric Context Cache Page 1 390 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 6-75 Create cell level Fabric Context Cache Page 2 c. Click OK. d. Click Save. 5. Create cell level Object instance for Fabric Mediation Endpoint Selection Cache. a. Click New. b. Specify the following parameters. • • • • Name: Fabric Mediation Endpoint Selection JNDI name: services/cache/fabric/mediation/selection Cache size: 1000 Default priority: 1 See Figure 6-76 on page 392. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 391 Figure 6-76 Create cell level Fabric Mediation Endpoint Selection Cache c. Click OK. d. Click Save. 392 z/OS: WebSphere Business Process Management V6.2 Production Topologies 6. Create cell level Object instance for Fabric Mediation Interface Endpoints Cache. a. Click New. b. Specify the following parameters • • • • Name: Fabric Mediation Interface Endpoints Cache JNDI name: services/cache/fabric/mediation/endpoint Cache size: 100 Default priority: 1 See Figure 6-77 on page 394. Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 393 Figure 6-77 Create cell level Fabric Mediation Interface Endpoints Cache c. Click OK. d. Click Save. 394 z/OS: WebSphere Business Process Management V6.2 Production Topologies 7. Create cell level Object instance for Fabric Mediation Subscription Cache a. Click New. b. Specify the following parameters • • • • Name: Fabric Mediation Subscription Cache JNDI name: services/cache/fabric/mediation/subscription Cache size: 2000 Default priority: 1 See Figure 6-78. Figure 6-78 Create cell level Fabric Mediation Subscription Cache Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 395 c. Click OK. d. Click Save. 8. Verify all Object cache instances (Figure 6-79). Figure 6-79 All Object cache instances 396 z/OS: WebSphere Business Process Management V6.2 Production Topologies 6.8 Verify basic server operation This section contains verification steps. Clear the OSGI cache The OSGI cache is used by Java components to cache frequently used objects, particularly those that are used by eclipse componentry on the browser. Whenever we change the configuration we must clear this cache to avoid unexpected results. To clear the OSGI cache, perform the following steps: 1. Make sure the deployment manager, node agents, and clustered servers are all stopped. 2. For the Deployment Manager, from a telnet or OMVS session, go to /wasv61config/mdcell/mddmnode/DeploymentManager/profiles/default/ configuration and delete the contents of this directory. 3. For node A, from a telnet or OMVS session, go to /wasv61config/mdcell/mdnodea/AppServer/profiles/default/ configuration and delete the contents of this directory. 4. For node B, from a telnet or OMVS session, go to /wasv61config/mdcell/mdnodeb/AppServer/profiles/default/ configuration and delete the contents of this directory. Enable Business Space powered by WebSphere To enable WebSphere Business Services Fabric for z/OS functions in Business Space powered by WebSphere, you need to edit the wbsfEndpoints.xml files for the Deployment Manager and nodes A and B. Note these are ASCII files so you will need an ASCII editor to make the updates: For the deployment manager: /wasv61config/mdcell/mddmnode/DeploymentManager/profiles/default/ BusinessSpace/registryData/wbsfEndpoints.xml We made the changes to the deployment manager as shown in Example 6-15. Example 6-15 wbsfEndpoints.xml file for deployment manager <?xml version="1.0" encoding="UTF-8"?> <!-- START NON-TRANSLATABLE --> <tns:BusinessSpaceRegistry xmlns:tns="http://com.ibm.bspace/BusinessSpaceRegistry" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 397 xsi:schemaLocation="http://com.ibm.bspace/BusinessSpaceRegistry BusinessSpaceRegistry.xsd "> <tns:Endpoint> <tns:id>{com.ibm.bspace}bspaceBusinessVariableServiceRootId</tns:id> <tns:version>1.0.0.0</tns:version> <tns:url>https://wtsc42.itso.ibm.com:20049/fabricrest</tns:url> <tns:description>Location of Business Space Business Policy server</tns:description> </tns:Endpoint> </tns:BusinessSpaceRegistry> For node A: /wasv61config/mdcell/mdnodea/AppServer/profiles/default/BusinessSpace/ registryData/wbsfEndpoints.xml We made changes for node A as shown in Example 6-16. Example 6-16 wbsfEndpoints.xml file for node a <?xml version="1.0" encoding="UTF-8"?> <!-- START NON-TRANSLATABLE --> <tns:BusinessSpaceRegistry xmlns:tns="http://com.ibm.bspace/BusinessSpaceRegistry" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://com.ibm.bspace/BusinessSpaceRegistry BusinessSpaceRegistry.xsd "> <tns:Endpoint> <tns:id>{com.ibm.bspace}bspaceBusinessVariableServiceRootId</tns:id> <tns:version>1.0.0.0</tns:version> <tns:url>https://wtsc42.itso.ibm.com:20049/fabricrest</tns:url> <tns:description>Location of Business Space Business Policy server</tns:description> </tns:Endpoint> </tns:BusinessSpaceRegistry> 398 z/OS: WebSphere Business Process Management V6.2 Production Topologies For node B: /wasv61config/mdcell/mdnodeb/AppServer/profiles/default/BusinessSpace/ registryData/wbsfEndpoints.xml We made the changes for node B as shown in Example 6-17. Example 6-17 wbsfEndpoints.xml file for node B <?xml version="1.0" encoding="UTF-8"?> <!-- START NON-TRANSLATABLE --> <tns:BusinessSpaceRegistry xmlns:tns="http://com.ibm.bspace/BusinessSpaceRegistry" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://com.ibm.bspace/BusinessSpaceRegistry BusinessSpaceRegistry.xsd "> <tns:Endpoint> <tns:id>{com.ibm.bspace}bspaceBusinessVariableServiceRootId</tns:id> <tns:version>1.0.0.0</tns:version> <tns:url>https://wtsc53.itso.ibm.com:20049/fabricrest</tns:url> <tns:description>Location of Business Space Business Policy server</tns:description> </tns:Endpoint> </tns:BusinessSpaceRegistry> Back up the cell’s ZFS data sets Before starting the verification steps we decided to back up our final configuration HFS. We continue to dump all three HFS’s: For the deployment manager: WASCFG.MDCELL.MDDMNODE.CONFIG.ZFS For node A: WASCFG.MDCELL.MDNODEA.CONFIG.ZFS For node B: WASCFG.MDCELL.MDNODEB.CONFIG.ZFS We used the JCL shown in Example 6-12 on page 341 to perform the back up. Example 6-18 JCL to back up the deployment manager and nodes A and B //MDDUMP3S JOB NOTIFY=&SYSUID,CLASS=A,MSGCLASS=A, // REGION=0M,TIME=1440 /*JOBPARM S=SC42 //* DUMP CONFIG HFS FOR DMGR AND BOTH NODES //DUMP EXEC PGM=ADRDSSU //DMGR DD DISP=(NEW,CATLG),DSN=WASCFG.MDCELL.DUMPDMGR.MAR7, Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 399 // UNIT=3390,SPACE=(CYL,(1000,100),RLSE),VOL=SER=TSTO61 //NODEA DD DISP=(NEW,CATLG),DSN=WASCFG.MDCELL.DUMPNODA.MAR7, // UNIT=3390,SPACE=(CYL,(1000,100),RLSE),VOL=SER=TSTO63 //NODEB DD DISP=(NEW,CATLG),DSN=WASCFG.MDCELL.DUMPNODB.MAR7, // UNIT=3390,SPACE=(CYL,(1000,100),RLSE),VOL=SER=TSTO64 //SYSPRINT DD SYSOUT=* //SYSIN DD * DUMP DATASET( INCLUDE( WASCFG.MDCELL.MDDMNODE.CONFIG.ZFS ) ) OUTDD(DMGR) TOL(ENQF) OPTIMIZE(4) COMPRESS DUMP DATASET( INCLUDE( WASCFG.MDCELL.MDNODEA.CONFIG.ZFS ) ) OUTDD(NODEA) TOL(ENQF) OPTIMIZE(4) COMPRESS WASCFG.MDCELL.MDNODEB.CONFIG.ZFS - DUMP DATASET( INCLUDE( ) ) OUTDD(NODEB) TOL(ENQF) OPTIMIZE(4) COMPRESS - /* //OTHER DD * Start Deployment manager Start the deployment manager. See “Start the deployment manager” on page 305 Start cluster node agents Start cluster node agents. See 6.4.1, “Start cluster node agents” on page 340 Start clustered servers Start clustered servers. See 6.4.2, “Start clustered servers” on page 342 Login to fabric console We will verify Fabric is working on both of our server nodes. Repeat the steps below for both of the urls: Node A, server 1a: http://wtsc42.itso.ibm.com:20048/fabric/app. Node B, server 1b: http://wtsc53.itso.ibm.com:20048/fabric/app. 400 z/OS: WebSphere Business Process Management V6.2 Production Topologies On our system we have sysplex distributer setup with virtual ip addresses defined for the cluster and for each node. We also tried the following URL for the cluster: http://wpsplex.itso.ibm.com:20048/fabric/app We are only verifying that we can access the Fabric console on the deployed server nodes. For a more complete verification of WebSphere Business Services Fabric for z/OS and WebSphere Process Server for z/OS function, see Chapter 7, “Installing and testing WebSphere Business Services Fabric for z/OS” on page 403. You will see the login window as shown in Figure 6-80. Log in with MDFABADM, the user ID you added to RACF as the Fabric Administrator user ID. Figure 6-80 WebSphere Business Services Fabric for z/OS login panel Chapter 6. Incorporating WebSphere Business Services Fabric for z/OS into the cluster 401 After login, the main WebSphere Business Services Fabric for z/OS welcome panel is displayed (Figure 6-81). Figure 6-81 WebSphere Business Services Fabric for z/OS successful login 402 z/OS: WebSphere Business Process Management V6.2 Production Topologies 7 Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS This chapter contains the following sections: 7.1, “Install and test the sample application” on page 405 7.2, “Errors we encounters while testing the Fabric sample application” on page 427 7.3, “Verify WebSphere Business Services Fabric for z/OS functionality using Business Space powered by WebSphere” on page 433 7.4, “Configuring the WebSphere Business Services Fabric for z/OS Content Packs” on page 451 © Copyright IBM Corp. 2009. All rights reserved. 403 Note: A word about service levels. We experienced a significant amount of pain during the writing of this book because of incorrect and misleading service levels of the code in support of WebSphere Business Services Fabric for z/OS. We recommend that before you attempt any of the verification steps below, you ask your IBM representative to do a thorough search of current product databases to ensure you have the most current code levels. At the time of this writing, the most current level of software for WebSphere Business Services Fabric for z/OS was UK45433, UK45434, UK45435, and UK45437. In addition, a support PTF for WebSphere Process Server for z/OS is also required for proper operation of WebSphere Business Services Fabric for z/OS. This PTF number is UK42437 Additional service was expected for WebSphere Business Services Fabric for z/OS and WebSphere Process Server for z/OS, but the exact PTF numbers were not available before this book went to press. 404 z/OS: WebSphere Business Process Management V6.2 Production Topologies 7.1 Install and test the sample application We tested the WebSphere Business Services Fabric environment by using the sample enterprise application described in Redbooks publication WebSphere Business Process Management V6.2 Production Topologies, SG24-7732. The enterprise application installed in this section is supplied with the additional materials provided with this book. Refer to Appendix A, “Additional material” on page 615. Perform the following steps to install and test the sample application: 1. Update the endpoint URLS of ITSOBankCBAPrj. See 7.1.1, “Update the endpoint URLs of ITSOBankCBAPrj” on page 406. 2. Change endpoint URLs by editing the .owl file. See 7.1.2, “Change endpoint URLs by editing the .owl file” on page 406. 3. Import the FCA files. See 7.1.3, “Import the FCA files” on page 407. 4. Assign MDFABADM to the ITSOBankOrg team. See 7.1.4, “Assign MDFABADM to the ITSOBankOrg team” on page 408 5. Create an external LDAP federation project. See 7.1.5, “Create an external LDAP federation project” on page 409. 6. Set enrollments. See 7.1.6, “Set enrollments” on page 416. 7. Install the sample application’s .ear files. See 7.1.7, “Install the sample application’s .ear files” on page 420. 8. Map modules to the web server. See 7.1.8, “Map modules to the Webserver” on page 422. 9. Update the SCA import URLs. See 7.1.9, “Update the SCA import URLs” on page 423. 10.Test the sample application. See 7.1.10, “Test the sample application” on page 424. Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 405 Note: When you are using the Fabric console, each operation you select from the menu on the left is displayed as a series of tabbed pages. Each tab usually has its own Save button. It is important to click Save on each tab. Clicking Save on one tab does not save changes made on the other tabs. We found that forgetting to save changes on each tab was the source of a lot of confusing errors. 7.1.1 Update the endpoint URLs of ITSOBankCBAPrj The sample application provided with the Additional Materials in code\Scenarios\Fabric was originally tested and developed for the non-z/OS part of this Redbooks publication. All the endpoints defined in the application point to host name itsodmgr, using the default HTTP port of 80. When testing the sample on your z/OS system, you will need to set a different host name and port from the ones hard-coded in the sample application. This update must be made before you import the application into Fabric. There are two ways to update the application Endpoint URLs: Update the file http_www.ibm.com_vehicleloan_inst.owl in ITSOBankCBAPrj20080804-owl.zip Update the endpoints using WID The best way to do this is using WID. The process for updating the application’s endpoint URLs using WID is described in WebSphere Business Process Management V6.2 Production Topologies, SG24-7732. A z/OS systems programmer configuring the ITSO Fabric sample application for the first time is unlikely to have WID on their workstation. Therefore, we describe how to edit the .owl file in 7.1.2, “Change endpoint URLs by editing the .owl file” on page 406 so you can install the ITSO sample application into your WebSphere Business Services Fabric for z/OS cluster without the need for WID. 7.1.2 Change endpoint URLs by editing the .owl file This section describes how to update the endpoint URLs that are coded within the http_www.ibm.com_vehicleloan_inst.owl file, which is in the ITSOBankCBAPrj20080804-owl.zip. 1. Extract the ITSOBankCBAPrj20080804-owl.zip file on your workstation. 2. Open http_www.ibm.com_vehicleloan_inst.owl in Textpad or WordPad. 406 z/OS: WebSphere Business Process Management V6.2 Production Topologies 3. Search for String http://itsodmgr. Replace the host name itsodmgr with your <host>:<port> (where <host> is the WebSphere Business Services Fabric for z/OS host and <port> is the HTTP port). There is no port following the host name itsodmgr in the original file because it was defaulting to port 80. We used wpsplex.itso.ibm.com:20048 for the host name. 4. Save the .owl file. 5. Compress the updated owl file back into ITSOBankCBAPrj20080804-owl.zip. Note: Make sure the file is directly in the zip file and not added under a folder within the zip file. 7.1.3 Import the FCA files Perform the following steps to import the FCA files. Note: Before you import the FCA files, make sure you have updated the host name:port in the ITSOBankCBAPrj20080804-owl.zip as described in 7.1.2, “Change endpoint URLs by editing the .owl file” on page 406. 1. Log in to the WebSphere Business Services Fabric console by using the following URL: http://wpsplex.itso.ibm.com:20048/fabric/app 2. Navigate to Governance Manager → Import/Export. 3. Click Browse and locate the code\Scenarios\Fabric\FCA folder in the additional materials supplied with this book. Choose OrganizationsUsersandRoles20080804-owl.zip, and click Import file. 4. Repeat these steps to install the FCA files in the following order: – – – – FabricGovernance20080804-owl.zip ITSOBankOntPrj20080804-owl OrganizationsUserandRoles20080804-owl.zip ITSOBankCBAPrj20080804-owl.zip Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 407 7.1.4 Assign MDFABADM to the ITSOBankOrg team Perform the following steps to assign MDFABADM to the ITSOBankOrg team: 1. In the Fabric console, expand Governance Manager and click Manage User Accounts. The “Manage User Accounts” window (Figure 7-1) opens. Figure 7-1 Click a user ID to assign to the ITSOBankOrg 2. Click the user ID you want to add to the ITSOBankOrg project. This will probably be the user ID with which you just logged in. In our case this was mpfabadm. 3. Click the team ITSOBankOrg and click the right-arrow to move the ITSOBankOrg team into the Selected Teams box on the right, as shown in Figure 7-2. Figure 7-2 Select the ITSOBankOrg team and move to Selected Teams 408 z/OS: WebSphere Business Process Management V6.2 Production Topologies 4. When the ITSOBankOrg team is in the Selected Teams box (Figure 7-3), click Save. Figure 7-3 Save the Team assignment change 7.1.5 Create an external LDAP federation project When WebSphere Business Services Fabric for z/OS is configured into a WebSphere Process Server for z/OS cell that is using a LocalOS user registry (RACF in our case), authentication to Fabric and verification of Fabric J2EE roles like FabricAdministrator is performed against RACF. When you are configuring projects inside Fabric (assigning roles, managing enrollments, and managing subscriptions for example), Fabric will need to use some other user registry (such as LDAP). We configured a Tivoli Directory Server for z/OS V6 LDAP server to use in conjunction with WebSphere Business Services Fabric for z/OS. You need to configure a new External LDAP federation project that will allow Fabric to work with the external LDAP server. To do so, perform the following steps: 1. In the Fabric console, navigate to Governance Manager → Configure Projects. 2. Click Create Project. Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 409 3. Complete the new project details (Figure 7-4) in the “Create a Project” window as follows: a. In the Project Details section, enter a name for the project (we chose TDSzOS) and a description with the same name: TDSzOS. b. In the Project Specification section, choose External LDAP in the Project Type field. c. In the Source Name text box, enter a name to uniquely identify the LDAP repository. We set the started task name for our LDAP server, MDLDAP. d. Enter a value in the User NSI URI text box. The format of this is different than is stated in the Information Center. The required format is as follows: <http url>/dir/NSx# In the above example, x is some number. The URI follows web semantics but a real hostname:port is not required. Nevertheless, we entered the following URI: http://wpsplex.itso.ibm.com:20048/TDSzOS/NS1# e. In the Project Team section select ITSOBankOrg from the drop-down menu for the Project Team. f. Click Create Project. See Figure 7-4. Figure 7-4 Create a new project 410 z/OS: WebSphere Business Process Management V6.2 Production Topologies 4. Click Save to save the new project (Figure 7-5). Figure 7-5 Save the new project Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 411 5. Click the Project Team tab. Notice in Figure 7-6 that user ID mdfabadm is a Team Member because it was assigned to ITSOBankOrg team and the ITSOBankOrg team was assigned to the TDSzOS project. Figure 7-6 The Project Team showing Team Members 412 z/OS: WebSphere Business Process Management V6.2 Production Topologies 6. Click the Namespaces tab. Notice in Figure 7-7 that the Namespace name is a concatenation of the project name and the string External NS 1. Figure 7-7 The Namespaces tab showing the Namespace name 7. Click the Federation Settings tab. 8. Enter the details of the LDAP server as shown in Figure 7-8 on page 414. When running Tivoli Directory Server for z/OS in a sysplex, the host name must be a Distributed VIPA. – – – – – – host name: wpsplex.itso.ibm.com Port: 2389 Username: uid=LDAPADM,o=ibm Password: LDAPADM Source Name: MDLDAP User Namespace: TDSzOS External NS1 Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 413 Figure 7-8 Setting LDAP server details on the Federation Settings tab 9. Click Test Connection. If this does not succeed as shown in Figure 7-9, recheck the details you entered. Figure 7-9 Test connection was successful 10.Click Save to save the changed on the Federation Settings tab once you have tested the connection successfully. 414 z/OS: WebSphere Business Process Management V6.2 Production Topologies 11.Click the Federation Mapping tab. When using Tivoli Directory server, it is necessary to code the BaseDN attributes that apply to the users in the LDAP server. For our LDAP server the users were defined under cn=People. The suffix was o=ibm, so we coded BaseDN:cn=People,o=ibm, as shown in Figure 7-10. Figure 7-10 Mapping LDAP attributes on the Federation Mapping tab 12.Click Save. The TDSzOS project is now complete. 13.Stop and restart the four fabric applications. 14.During the restart of the applications look in the servant logs for messages like those in Example 7-1, which show a successful federation of the LDAP server. If there are problems with the external LDAP project or with the LDAP server, you will see errors in the logs at this point. Any errors must be resolved before preceding. Example 7-1 Messages showing successful federation of external LDAP server Trace: 2008/09/04 16:00:52.799 01 t=7B1988 c=UNK key=P8 (13007002) ThreadId: 0000005b FunctionName: internalInfo SourceId: com.webify.fabric.catalog.federation.host.FederationHost Category: INFO ExtendedMessage: Registering federated source 'ldap(wpsplex.itso.ibm.com:2389)' as 'MDLDAP' Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 415 7.1.6 Set enrollments The next step is to grant roles, enrollments, and subscriptions. See the following relevant sections: “Grant Administrator role within ITSOBankOrg” on page 416 “Enroll ITSOBankOrg to ITSOBankLoadApp” on page 419 “Subscribe the user ID to ITSOBankLoan App” on page 419 Grant Administrator role within ITSOBankOrg Perform the following steps to Grant Administrator role within ITSOBankOrg. 1. In the Fabric console, navigate to Subscriber Manager → Manage Subscribers. Click ITSOBankOrg → Users → Grant User Roles. If Fabric can contact the external LDAP Server and search for users, you will see all the users Fabric finds in the Available users table on the left, as shown in Figure 7-11. Note: If the users in your external LDAP Server are not listed in the Available Users list, check that you stopped and started all the fabric applications after creating the external LDAP server project. Also, go back and check that you clicked Save on every tab in external LDAP project. If this still does not work, try recycling the servers. Figure 7-11 Users from the LDAP server listed in Available Users field 416 z/OS: WebSphere Business Process Management V6.2 Production Topologies 2. Select the FabricAdministrator (mpfabadm) user and click the right arrow to move the selection into the Selected Users box (Figure 7-12). Figure 7-12 FabricAdministrator moved to Selected Users field 3. Select the Administrator role under Available Roles and click the right arrow to move the selection into the Selected Roles box as shown in Figure 7-13. Figure 7-13 Administrator moved to Selected Roles field Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 417 4. Click Grant Users Roles as shown in Figure 7-14. Figure 7-14 Click Grant Users Roles On the Users tab you will now see the selected User ID and its role under the User Selection section, as shown in Figure 7-15. Figure 7-15 User mdfabadm listed with its role under User Selection 418 z/OS: WebSphere Business Process Management V6.2 Production Topologies Enroll ITSOBankOrg to ITSOBankLoadApp Perform the following steps to Enroll ITSOBankOrg to ITSOBankLoadApp. 1. Navigate to Subscriber Manager → Manage Subscribers. Click ITSOBankOrg, and click the Enrollments tab. 2. Select ITSOBankLoanApp under the Enrollment Selection section. Click Save Enrollments. When you select ITSOBankLoanAPP, the LoanProcessBS Business Service is also selected, as shown in Figure 7-16. Figure 7-16 Selecting and saving the enrollments Subscribe the user ID to ITSOBankLoan App Perform the following steps to Subscribe the user ID to ITSOBankLoan App. 1. Navigate to Subscriber Manager → Manage Subscribers. Click ITSOBankOrg and click the Users tab. 2. Click the user mdfabadm (Figure 7-17), and click the Subscriptions tab. Figure 7-17 Select the User Id mdfabadm 3. Select ITSOBankLoanAPP in the Subscription Selection section and click Save Subscriptions. Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 419 7.1.7 Install the sample application’s .ear files Perform the following steps to Install the sample application’s EAR Files Important: Ensure that at least one cluster member in the WebSphere Process Server for z/OS cluster is started before attempting to deploy these applications. 1. Log in to the Integrated Solutions Console and navigate to Applications → Install New Application. 2. Choose Local file system and click Browse. 3. In the Additional Material, navigate to the folder code\Scenarios\Fabric\EAR, choose ITSO_implApp.ear and click Next. 4. In the “Step 1: Select installation options” window, click Next. 5. In the “Step 2: Map modules to servers” window (Figure 7-18), select the boxes next to all the modules, then select the cluster and any web servers you may have configured from the Clusters and Servers list box. Click Apply. In our single-cluster topology the application was already mapped to the correct cluster. Figure 7-18 Mapping modules to the cluster Important: Notice in Figure 7-18 the small circle with a cross to the left of Step 3. This is to draw your attention to this step. Do not skip past Step 3. 420 z/OS: WebSphere Business Process Management V6.2 Production Topologies Check the mapping shown in the Server column on the right of the table as shown in Figure 7-19. Figure 7-19 Checking that the modules are mapped to the correct cluster 6. Click Next. 7. On the “Step 3: Map virtual hosts for Web modules” window (Figure 7-20), in the Virtual host list box, select default_host, and click Next. When you have done this, the circular icon with a cross next to Step 3 (mentioned in the note after Figure 7-18 on page 420) disappears. Figure 7-20 Be sure to check the box to select the default_host 8. On the “Step 4: Summary” window, click Finish. 9. Click Save. 10.Repeat these steps to install WebSphereEnvUtilApp.ear and ITSOApp.ear from the folder code\Scenarios\Fabric\EAR directory of the Additional Materials. Note: If you receive the following error when deploying the ITSOApp, see “CWWBF0029E: Process components of ITSOApp cannot be configured in WebSphere configuration repository” on page 427: CWWBF0029E: Process components of ITSOApp cannot be configured in WebSphere configuration repository 11.Navigate to Applications → Enterprise Applications. Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 421 12.Choose ITSOAPP, ITSO_implApp, and WebSphereEnvUtilApp and click Start. Note: The ITSOAPP application is particular about how it is started. We had to reatart our servers to get it to start. Also make sure your /tmp directory has space in it and that your HFS’s for your nodes can expand. 7.1.8 Map modules to the Webserver Note: This step may not be necessary if you selected the box to map all the applications to the Web server at the time you configured the Web servers. When using a Web server to balance work across the servers and maintain affinity, it is necessary to map some of the applications to the web server. This is allows them to be in the plugin-cfg.xml file used by the WebSphere plugin in the HTTP Server. Perform the following steps to map modules to the Web server. 1. In the Integrated Solutions Console, navigate to Applications → Enterprise Applications → BPCExplorer_mdcl01_WPS → Manage Modules. 2. Select all the modules. 3. Choose your Web server (in our environment this is itsowebserver) and the cluster where the application is already mapped under Clusters and Servers 4. Click Apply and click OK. 5. Save your changes. 6. Repeat these steps for the following applications: – – – – Fabric_Catalog Fabric_Engine Fabric_Tools Fabric_Tools_Help (if installed) 7. Navigate to Servers → Web servers → <webserver> → Generate Plug-in. 8. Select <webserver> → Propagate Plug-in. 422 z/OS: WebSphere Business Process Management V6.2 Production Topologies 7.1.9 Update the SCA import URLs In the sample application, all the Web service SCA Imports are bound to URLs that point to itsodmgr. There is no port explicitly defined in the URLs, so the default HTTP port of 80 is used. You will likely need to change these to match your hostname:port. To do so, perform the following steps: Note: Ensure that at least one WebSphere Process Server for z/OS cluster member is started and that the three ITSO sample applications are started. 1. In the Integrated Solutions Console, navigate to Applications → SCA Modules and click ITSO. 2. Navigate to Imports → CreditCheckImport → Bindings, as shown in Figure 7-21. Figure 7-21 Expanding the CreditCheckExport bindings 3. Click CreditCheckExport_CreditCheckHttpService. 4. Change the host name and port number in the Endpoint field (Figure 7-22 on page 424). Click OK. When you have a cluster with web servers running the WebSphere plug-in, you would normally specify the Distributed VIPA and port of the web server. Here we specified the HTTP port of the WebSphere Process Server for z/OS servers because we had not finished testing the web servers at the time we deployed the sample Fabric applications. Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 423 Figure 7-22 Specify the appropriate host name and port 5. Repeat these steps for RatingImport, VinLookImport, and VerifyCustomerImport in the SCA module ITSO. 6. Exit the ITSO SCA module. 7. Navigate to Applications → SCA Modules. Click the WebSphereEnvUtil module and repeat these steps for the SubscriberManagerImport present in the module WebSphereEnvUtil (Figure 7-23). Figure 7-23 Updating SubscriberManagerService in WebSphereEnvUtil 8. Save the configuration changes. 7.1.10 Test the sample application Perform the following steps to test the sample application: 1. Ensure that the user MDFABADM has the WebClientUser role and the eventCreator role. We connected user ID MDFABADM to the RACF groups MDWCUG and MDEVTCRG, which were on the access list for the RACF EJBROLE profiles called MD.WebClientUser and MD.eventCreator. Note: We created sample job MDEJBBSR (Example 7-14) to define all the EJBROLEs required for use with Business Space powered by WebSphere and WebSphere Business Services Fabric for z/OS as well as those needed for Fabric access to the BPC Explorer. If you have already run this job, you should not encounter the errors listed below. 424 z/OS: WebSphere Business Process Management V6.2 Production Topologies Note: If you try to log in to the BPC Explorer with a user ID that does not have access to the WebClientUser role, then login will fail, returning the following message: You are not authorized to view this page There will also be the following messages in the z/OS SYSLOG: ICH408I USER(MDFABADM) GROUP(MDFADMG ) NAME(MD FABRIC ADMIN MD.WebClientUser CL(EJBROLE ) INSUFFICIENT ACCESS AUTHORITY ACCESS INTENT(READ ) ACCESS ALLOWED(NONE ) ) If you try to run the TestLoanProcess from BPC Explorer while logged in as MDFABADM, and that user is not authorized to create events, the following errors are issued: ICH408I USER(MDFABADM) GROUP(MDFADMG ) NAME(MD FABRIC ADMIN MD.eventcreator CL(EJBROLE ) INSUFFICIENT ACCESS AUTHORITY ACCESS INTENT(READ ) ACCESS ALLOWED(NONE ) ) 2. Log in to BPC Explorer with user MDFABADM. In our environment we used the following URL: http://wpsplex.itso.ibm.com:20048/bpc 3. Click Process Templates, select the TestLoanProcess check box, and click Start Instance, as shown in Figure 7-24. Figure 7-24 Start an instance of TestLoadProcess Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 425 Note: If you do not see the New Loan Process and TestLoadProcess listed in BPC Explorer, check that you have started ITSOApp. 4. Specify the following values: – – – – – CustomerIdentificationNumber: 100 CustomerAddress: <Any string value> VIN: 12345678901234567 LoanAmountRequested: 1000.0 BankID: ITSO 5. Click Submit. The results look like Figure 7-25. Figure 7-25 Sample application test result 426 z/OS: WebSphere Business Process Management V6.2 Production Topologies Note: In the sample application, the TestLoanProcess acts as a proxy to the New Loan Process. The TestLoanProcess calls the New Loan Process through a context injector, which injects the required context for WebSphere Business Services Fabric. If you receive the following error when running TestLoanProcess, see “CWWBU0001E: in BPC Explorer withTestLoanProcess” on page 429: CWWBU0001E: A communication error occurred when the 'class com.ibm.bpe.clientmodel.command.CallCommand' function was called 7.2 Errors we encounters while testing the Fabric sample application CWWBF0029E: Process components of ITSOApp cannot be configured in WebSphere configuration repository Problem After successfully deploying two of the three sample applications, the deployment of ITSOApp failed with a CWWBF0029E error as shown in Example 7-2. Example 7-2 CWWBF0029E error CWWBF0029E: Process components of ITSOApp cannot be configured in WebSphere configuration repository Cause 1 There can be several possible causes of this symptom. In the Deployment Manager’s servant file, there were the errors shown in Example 7-3 during the deployment. Example 7-3 Deployment Manager’s servant file errors Trace: 2008/09/04 16:26:00.807 01 t=7C4270 c=UNK key=P8 (0000000A) Description: Log Java Message Message: CWWBV0001I: Validated process model 'NewLoanProcess' successfully: 0 warnings, 0 information. java.lang.NullPointerException at com.ibm.bpe.management.application.process.ProcessSCAArchive.\ addLinkReferencePropertiesToContext(ProcessSCAArchive.java:608) at com.ibm.bpe.management.application.process.ProcessSCAArchive.\ createBPELContext(ProcessSCAArchive.java:312) Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 427 at com.ibm.bpe.management.application.process.ProcessSCAArchive.\ initBPELProcessContexts(ProcessSCAArchive.java:222) at com.ibm.bpe.management.application.process.ProcessSCAArchive.\ <init>(ProcessSCAArchive.java:134) at com.ibm.bpe.management.application.process.ProcessSCAArchive.\ <init>(ProcessSCAArchive.java:169) at com.ibm.bpe.management.application.process.ProcessSCAArchive.\ <init>(ProcessSCAArchive.java:145) at com.ibm.bpe.processarchive.SCDLProcessComponentConfigureTask.performTask\ (SCDLProcessComponentConfigureTask.java:129) at com.ibm.ws.management.application.SchedulerImpl.run(SchedulerImpl.java:262) at java.lang.Thread.run(Thread.java:810) Cause 2 In another cell that suffered the same symptom, the fabricbus had not started in the adjunct because of the error shown in Example 7-4. Example 7-4 Fabricbus not starting error Exception: com.ibm.ws.sib.msgstore.MessageStoreRuntimeException: CWSIS1524E: Data source, /jdbc/fabric/me, not found. com.ibm.ws.sib.utils.ras.SibMessage Notice the JNDI name in the message is /jdbc/fabric/me (with the leading ‘/’) instead of jdbc/fabric/me. This was a typing error when configuring the fabricbus. Cause 3 On another occasion the cause was a problem in the deployment manager. The deployment manager’s ZFS could not extend because the volume it was on was full. Solution 1 After generating a personal certificate for the servant region user ID and restarting the cluster, it was possible to deploy ITSOApp without errors. Solution 2 Check that the fabricbus is started. If not, resolve any problems and restart the Fabric cluster (mdcl01_WPS in our scenario). Ensure that the fabricbus starts before trying to install ITSOApp. Solution 3 Ensure there is space on the volumes holding the configuration ZFS datasets to allow the datasets to extend. We cleaned out all FFDC logs and restarted the deployment manager before installing ITSOApp successfully. 428 z/OS: WebSphere Business Process Management V6.2 Production Topologies CWWBU0001E: in BPC Explorer withTestLoanProcess The error shown in Example 7-5 was received when submitting a TestLoanProcess through the BPC Explorer. Example 7-5 Error when submitting TestLoanProcess through the BPC Explorer CWWBU0001E: A communication error occurred when the 'class com.ibm.bpe.clientmodel.command.CallCommand' function was called. This symptom can have many causes. It is important to look in the logs for additional errors and in any FFDC logs that are produced. Here we describe some of the issues we experienced. Problem 1 The error shown in Example 7-6 was seen in the servant log after receiving CWWBU0001E when testing the TestLoanprocess. Example 7-6 Servant log error after receiving CWWBU0001E when testing the TestLoanprocess Trace: 2008/09/05 03:41:39.995 01 t=7BF188 c=UNK key=P8 (13007002) ThreadId: 0000003c FunctionName: com.ibm.ws.sca.internal.webservice.handler.PortHandler.processMessage#01 SourceId: SCA.FFDC Category: ALL ExtendedMessage: Exception: WebServicesFault faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.generalException faultString: WSWS3713E: Connection to the remote host itsodmgr failed. Received the following error: null faultActor: null faultDetail: Cause 1 Notice that the remote host in the message is itsodmgr, which is the host name coded in the application. The SCA bindings had not been updated in the ITSOApp to match the host name of our WebSphere Process Server for z/OS cluster. Solution 1 Update the SCA bindings as described in 7.1.9, “Update the SCA import URLs” on page 423. Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 429 Problem 2 On another occasion the error shown in Example 7-7 was seen in the servant log after receiving CWWBU0001E when testing the TestLoanprocess. Example 7-7 Servant log error after receiving CWWBU0001E when testing the TestLoanprocess Trace: 2008/08/29 12:36:10.541 01 t=7B87B8 c=UNK key=P8 (13007002) ThreadId: 00000045 FunctionName: com.ibm.ws.sca.internal.webservice.handler.PortHandler.processMessage#01 SourceId: SCA.FFDC Category: ALL ExtendedMessage: Exception: WebServicesFault faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.generalException faultString: WSWS3713E: Connection to the remote host wpsplex.itso.ibm.com failed. Received the following error: EDC8128I Connection refused. (errno2=0x74940000) The message contained the correct host name, so the problem must have been with the port, but the message did not report the port number. At the same time as this error there was an FFDC log written. In this FFDC log, in the transport section, there was more information (Example 7-8) about the remote host that was causing the problem. Example 7-8 Remote host of transport section in FFDC log transport = DEFAULT_TRANSPORT_NAME = http sessionContext = null transportName = http url = http://wpsplex.itso.ibm.com:22048/fabric-catalog/services/subscriberManager Cause 2 In this case we had updated the SCA bindings as described in 7.1.9, “Update the SCA import URLs” on page 423 but had coded the HTTP port number of the wrong cluster. The HTTP port number should have been 20048 for the cluster we were testing in, not 22048. Solution 2 Check all the SCA binds for the correct host name and port. Check especially for any blank characters that might have been accidentally inserted into the URL when using copy/paste. 430 z/OS: WebSphere Business Process Management V6.2 Production Topologies If you find an incorrect URL here, it would be a good idea to re-check the host name and port you specified in the wpsfEndpoint.xml when you customized it in 5.2.1, “Copy Fabric artifacts” on page 207. Problem 3 After correcting the SCA binding there was an error (Example 7-9) in the logs. Example 7-9 Log error after correcting the SCA binding ExtendedMessage: Assembly Failure: <soapenv:Body xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <soapenv:Fault><faultcode>soapenv:Server.generalException</faultcode> <faultstring>WSWS3713E: Connection to the remote host wpsplex.itso.ibm.com failed. Received the following error: EDC8128I Connection refused. (errno2=0x74940000)</faultstring></soapenv:Fault></soapenv:Body> (http://wpsplex.itso.ibm.com:22048/ITSO_implWeb/sca/PremiumLoanProviderExport) Cause 3 Notice the port 22048 in this message. This URL is coming from the application file http_www.ibm.com_vehicleloan_inst.owl, which is part of the ITSOBankCBAPrj20080804-owl.zip. The correct port was 20048 for the cluster we were testing. Solution 3 When testing with the sample ITSOApp, there are some hard-coded URLs in the application which must be updated to match the hostname:port of the fabric cluster with which you are testing. After updating the URLs in file http_www.ibm.com_vehicleloan_inst.owl, re-insert the file into the zip file ITSOBankCBAPrj20080804-owl.zip, and re-import that zip file into Fabric. Then re-do the Subscriber Manager subscriptions. Otherwise, you will encounter the problem described in Problem 5. Problem 4 The logs contained the ‘EDC8128I Connection refused’ message but in addition there was the error shown in Example 7-10. Example 7-10 Incorrect security settings error Trace: 2008/09/05 04:57:32.773 01 t=7BF188 c=UNK key=P8 (0000000A) Description: Log Java Message Message: Problem sending invocation event(s) org.springframework.jms.JmsSecurityException: CWSIA0006E: The authorization for the supplied user name was not successful.; Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 431 Cause 4 This error is caused by incorrect security settings for the fabricbus. In our case, when setting the user ID/group in the role authorized to connect to the bus, we had chosen “group” but specified the user ID MPJMSU instead of the group MPJMSG. Solution 4 Double-check the security settings on the fabric bus by navigating to Buses → fabricbus → Security for bus fabricbus → Users and groups in the bus connector role. Problem 5 The error shown in Example 7-11 was seen in the servant log when the CWWBU0001E error was issued in the BPC Explorer. Example 7-11 Servant log error when the CWWBU0001E error was issued in the BPC Explorer Trace: 2008/08/29 14:26:42.671 01 t=7BE7D0 c=UNK key=P8 (13007002) ThreadId: 00000044 FunctionName: error SourceId: com.ibm.ws.fabric.da.impl.DaContextManagerImpl Category: SEVERE ExtendedMessage: com.webify.wsf.engine.context.InvalidContextException: The subscription ''NONE'' is not fully qualified. Subscription ID must the fully qualified URI. The easiest way to get valid IDs is with the copy to clipboard feature in Subscription Manager Cause 5 Notice that the message is saying that the subscription is NONE, which suggests that the user has no subscriptions. We had coded an incorrect port number in the file http_www.ibm.com_vehicleloan_inst.owl the first time we customized it for the HTTP hostname:port of our WebSphere Business Services Fabric for z/OS cluster. After correcting this and re-importing the FCA zip file, ITSOBankCBAPrj20080804-owl.zip, we found that the user ID MPFABADM had lost its subscriptions. 432 z/OS: WebSphere Business Process Management V6.2 Production Topologies Solution 5 After re-importing FCA files, re-assign the user ID’s subscriptions again as described in “Enroll ITSOBankOrg to ITSOBankLoadApp” on page 419. Note: After correcting any incorrect SCA bindings or re-importing the ITSOBankCBAPrj20080804-owl.zip, stop and restart the related applications, ITSOApp, ITSO_implApp and WebSphereEnvUtilApp. 7.3 Verify WebSphere Business Services Fabric for z/OS functionality using Business Space powered by WebSphere Business Space powered by WebSphere functions are enabled by default for a stand-alone WebSphere Process Server for z/OS server. For a clustered implementation, you must complete the configuration manually. 7.3.1 Enable Business Space widgets for WebSphere Business Services Fabric for z/OS To enable WebSphere Business Services Fabric for z/OS widgets in Business Space powered by WebSphere you must perform the following tasks: Complete all configuration steps for WebSphere Business Services Fabric for z/OS including security and database. See sections Chapter 5, “WebSphere Business Services Fabric for z/OS” on page 201 and Chapter 6, “Incorporating WebSphere Business Services Fabric for z/OS into the cluster” on page 299. Complete the base Business Space powered by WebSphere configuration as specified in Chapter 9, “Creating an integrated user experience using Business Space powered by WebSphere” on page 547. Edit the wbsf.Endpoints.xml file and make sure it is placed in the proper location in the HFS for each of your cluster notes (7.3.2, “Edit required endpoints files” on page 434). Configure the Fabric Rest Services Gateway (7.3.3, “Configure the Fabric Rest Services gateway” on page 435). Once these configuration steps have been completed, you can verify that Business Space powered by WebSphere functions for WebSphere Business Services Fabric for z/OS work as expected. Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 433 7.3.2 Edit required endpoints files This section describes how to edit the endpoint files and place these files in the correct directory. wbsfEndpoints.xml We made the following modifications to wbsf.Endpoints (Example 7-12). Example 7-12 Modifications to wbsf.Endpoints.xml file <?xml version="1.0" encoding="UTF-8"?> <!-- START NON-TRANSLATABLE --> <tns:BusinessSpaceRegistry xmlns:tns="http://com.ibm.bspace/BusinessSpaceRegistry" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://com.ibm.bspace/BusinessSpaceRegistry BusinessSpaceRegistry.xsd "> <tns:Endpoint> <tns:id>{com.ibm.bspace}bspaceBusinessVariableServiceRootId</tns:id> <tns:version>1.0.0.0</tns:version> <tns:url>http://wpsplex.itso.ibm.com:20048/fabricrest</tns:url> <tns:description>Location of Business Space Business Policy server</tns:description> </tns:Endpoint> </tns:BusinessSpaceRegistry> fabricEndpoints.xml We made the following modifications to fabric.Endpoints (Example 7-12). Example 7-13 Modifications to fabric.Endpoints.xml file <?xml version="1.0" encoding="UTF-8"?> <!-- START NON-TRANSLATABLE --> <tns:BusinessSpaceRegistry xmlns:tns="http://com.ibm.bspace/BusinessSpaceRegistry" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://com.ibm.bspace/BusinessSpaceRegistry BusinessSpaceRegistry.xsd "> <tns:Endpoint> <tns:id>{com.ibm.bspace}bspaceBusinessVariableServiceRootId</tns:id> 434 z/OS: WebSphere Business Process Management V6.2 Production Topologies <tns:version>1.0.0.0</tns:version> <tns:url>http://wpsplex.itso.ibm.com:20048/fabricrest</tns:url> <tns:description>Location of Business Space Business Policy server</tns:description> </tns:Endpoint> </tns:BusinessSpaceRegistry> Place the Endpoints files in the correct HFS directory Once you have completed the edit of the wbsfEndpoitns.xml and fabricEndpoints files, you must copy them to the following directories: Node A /wasv61config/mdcell/mdnodea/AppServer/profiles/default/BusinessSpace/ registry/Data Node B /wasv61config/mdcell/mdnodeb/AppServer/profiles/default/BusinessSpace/ registry/Data 7.3.3 Configure the Fabric Rest Services gateway WebSphere Business Services Fabric for z/OS makes extensive use of REST services in its Business Space powered by WebSphere widgets. Unfortunately it does not use the REST Services gateway functions in the base WebSphere Process Server for z/OS product. Instead, it implements its own gateway functions. We must therefore configure an additional EJBROLE for the role used by this gateway function. We issued the following RACF commands to authorize our Fabric administrator ID, MDFABADM, to the EJBROLE required by the Fabric REST Services Gateway and to the base WebSphere Process Server for z/OS REST services gateway. We created sample job MDEJBBSR (Example 7-14 on page 436) to define all the EJBROLEs required for use with Business Space powered by WebSphere and WebSphere Business Services Fabric for z/OS, as well as those needed for Fabric access to the BPC Explorer. Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 435 Note: Carefully check the output of this job. Because we had multiple team members defining roles for various BPM suite products and functions, we found that some IDs and EJBROLEs that needed to be defined had already been defined within the environment. In this case, only the additional PERMITs for the Fabric admin ID were neccesary. Example 7-14 EJBROLEs definitions for MDFABADM, Business Spaces, and BPC //MDEJBBSR JOB (0),'RACFADM',CLASS=A,MSGCLASS=A,REGION=0M, // NOTIFY=&SYSUID //***********************************************************/ //* Make sure you edit this file with CAPS OFF. */ //* Run this job using a userid with RACF SPECIAL. */ //* The RACF userids and groups used here must exist. */ //* Run job XXRACF first. */ //* Change MD to your two character cell prefix. */ //* For example, MD. (Use upper case.) */ //* Set the OWNER as you want. */ //***********************************************************/ //* //* Create EJB role profiles for Fabric access to Business Space //* //RACFDBA EXEC PGM=IKJEFT01,DYNAMNBR=20 //SYSTSPRT DD SYSOUT=* //SYSUDUMP DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSTSIN DD * /* Define Fabric REST Services Gateway role */ RDEFINE EJBROLE MD.RESTServiceUser + UACC(NONE) + OWNER(MDCFG) + DATA('MD RESTService User role') PERMIT MD.RESTServiceUser + CLASS(EJBROLE) + RESET PERMIT MD.RESTServiceUser + CLASS(EJBROLE) + ID(MDFABADM,MDFADMG) + ACCESS(READ) /* Define WPS REST Services Gateway role */ RDEFINE EJBROLE MD.RestServicesUser + UACC(NONE) + OWNER(MDCFG) + DATA('MD RestServicesUser role') 436 z/OS: WebSphere Business Process Management V6.2 Production Topologies PERMIT PERMIT MD.RestServicesUser CLASS(EJBROLE) RESET MD.RestServicesUser CLASS(EJBROLE) ID(MDFABADM,MDFADMG) ACCESS(READ) /* Define WebClientUser role */ RDEFINE EJBROLE MD.WebClientUser UACC(NONE) OWNER(MDCFG) DATA('MD WebClientUser role') PERMIT MD.WebClientUser CLASS(EJBROLE) RESET PERMIT MD.WebClientUser CLASS(EJBROLE) ID(MDFABADM,MDFADMG) ACCESS(READ) /* Define eventCreator role */ RDEFINE EJBROLE MD.eventCreator UACC(NONE) OWNER(MDCFG) DATA('MD eventCreator role') PERMIT MD.eventCreator CLASS(EJBROLE) RESET PERMIT MD.eventCreator CLASS(EJBROLE) ID(MDFABADM,MDFADMG) ACCESS(READ) //* //* Create EJB role profiles for Fabric access to BPC //* /* Define TaskSystemAdministator role */ RDEFINE EJBROLE MD.TaskSystemAdministrator UACC(NONE) OWNER(MDCFG) DATA('MD TaskSystemAdministrator role') PERMIT MD.TaskSystemAdministrator CLASS(EJBROLE) RESET PERMIT MD.TasksystemAdministrator CLASS(EJBROLE) ID(MDFABADM,MDFADMG) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 437 ACCESS(READ) /* Define TaskSystemAdministator role */ RDEFINE EJBROLE MD.TaskSystemMonitor UACC(NONE) OWNER(MDCFG) PERMIT MD.TaskSystemMonitor CLASS(EJBROLE) RESET PERMIT MD.TasksystemMonitor CLASS(EJBROLE) ID(MDFABADM,MDFADMG) ACCESS(READ) /* Define BPESystemAdministrator role */ RDEFINE EJBROLE MD.BPESystemAadministrator UACC(NONE) OWNER(MDCFG) DATA('MD BPESystemAdministrator') PERMIT MD.BPESystemAdministrator CLASS(EJBROLE) RESET PERMIT MD.BPESystemAdministrator CLASS(EJBROLE) ID(MDFABADM,MDFADMG) ACCESS(READ) /* Define BPESystemMonitor role */ RDEFINE EJBROLE MD.BPESystemMonitor UACC(NONE) OWNER(MDCFG) DATA('MD BPE System Monitor') PERMIT MD.BPESystemMonitor CLASS(EJBROLE) RESET PERMIT MD.BPESystemMonitor + CLASS(EJBROLE) ID(MDFABADM,MDFADMG) ACCESS(READ) SETR RACLIST(EJBROLE) REFRESH SEARCH MASK(MD) CLASS(EJBROLE) /* 438 z/OS: WebSphere Business Process Management V6.2 Production Topologies + + + + + + + + + + + + + + + + + + + + + + + 7.3.4 Clear the OSGI cache Once you have completed the previous configuration/reconfiguration you must clear out the OSGI cache. To do this, perform the following steps: 1. Make sure the clustered servers are all stopped. 2. For node A, from a telnet or OMVS session, go to /wasv61config/mdcell/mdnodea/AppServer/profiles/default/ configuration. Delete the contents of this directory. 3. For node B, from a telnet or OMVS session, go to /wasv61config/mdcell/mdnodeb/AppServer/profiles/default/ configuration. Delete the contents of this directory. 4. Retart the cluster and servers. 7.3.5 Verify WebSphere Business Services Fabric for z/OS Business Space functions We verified Business Space powered by WebSphere functions for WebSphere Business Services Fabric for z/OS by performing the following steps: 1. Verify that the deployment manager, node agents, cluster, and servers are all started by performing the following steps a. Login to the Integrated Solutions Console. b. Navigate to Servers → Application Servers. c. Click Application Servers. See Figure 7-26 on page 440. Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 439 Figure 7-26 Servers are up on both nodes A and B 2. Verify that thhe required applications are started by performing the following steps: a. Login to the Integrated Solutions Console. b. Navigate to Applications → Enterprise Applications. c. Click Applications. d. The following applications should be started for Fabric: • • • • • Fabric Catalog Fabric Engine Fabric REST Services Fabric Tools Fabric Tools Help e. The following applications should be started for Business Space: • • Business Space Manager IBM BSPACE WIDGETS f. REST Services Gateway should be started for WPS REST Services Gateway. See Figure 7-27 on page 441. 440 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 7-27 Verify all required applications are started and available Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 441 3. Log in to the Business Spaces console (Figure 7-28). Figure 7-28 Login to Business Spaces console 4. Navigate to the Business Space Manager. A user can have many business spaces with each one having a different purpose, but Business Space can display the contents of one space at a time. That space is known as the open Business Space. 5. Create a new Business Space by navigating to the Business Space Manager. Select Manage Business Spaces from the Welcome drop-down menu. See Figure 7-29 on page 443. 442 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 7-29 Navigate to the Business Space Manager 6. Click the icon on the top left in the tool bar, as shown in Figure 7-30. Figure 7-30 Click the icon to create a new Business Space Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 443 7. Enter a name for the Business Space (such as FabricTest1). Choose the Fabric Authoring template, and click OK, as shown in Figure 7-31. Figure 7-31 Create a new Fabric space A new Business Space named FabricTest1 is created. Because we used the Fabric Authoring template, our space was pre-populated with several pages dealing with the authoring components of WebSphere Business Services Fabric for z/OS. See Figure 7-32 on page 445. 444 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 7-32 Business Space with Fabric Authoring pages deployed 8. Click Start working with Applications to bring up the application browser. See Figure 7-33. Figure 7-33 Manage Fabric Authoring applications 9. Click Actions and select Add Application to Space from the drop-down menu. See Figure 7-34 on page 446. Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 445 Figure 7-34 Add an application 10.Select an application from the left pane and click Add to move it to your active applications. Click OK. The new application is added. See Figure 7-35. Figure 7-35 New application added 11.Select the new application by clicking on it. 12.Click the Governance page (Figure 7-36 on page 447). 446 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 7-36 the Fabric Governance page is displayed 13.Click Actions and select Create New Change Set. 14.Create a new change set with the following values: – Name: Test change set 1 – Description: This is a test change set Click OK. See Figure 7-37. Figure 7-37 Create a change set 15.Click the new change set Test change set 1 The details of the change set we just created are displayed. Notice the change set is in draft status. See Figure 7-38 on page 448. Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 447 Figure 7-38 Change set in draft status 16.Click Submit Change Set. 17.Enter Change Set Comments: Change set 1 submit for approval. See Figure 7-39. Figure 7-39 Submit draft change set for approval 18.Click OK. See Figure 7-40 on page 449. 448 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 7-40 Change set approval pending Notice the status has changed to Pending. 19.Click Approve Change Set. 20.Enter Change Set Comments: Approve change set 1. See Figure 7-41. Figure 7-41 Approve change set 1 21.Click OK. See Figure 7-42 on page 450. Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 449 Figure 7-42 Change set 1 is approved Notice Change set 1 is now approved. 22.Click Publish Change Set. 23.Enter Change Set Comments: Publish change set 1. See Figure 7-43. Figure 7-43 Publish change set 1 24.Click OK. See Figure 7-44 on page 451. 450 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 7-44 Change set is published You have created a sample change set and executed a simple workflow fo rit. The workflow follows the steps: Create → Submit → Approve → Publish. You have verified basic Business Space functionality for WebSphere Business Services Fabric for z/OS. 7.4 Configuring the WebSphere Business Services Fabric for z/OS Content Packs Content packs are industry-specific solutions provided as a starting point for common pattterns and implementations. The following Industry Content Packs are available: IBM Banking Payments Pack for WebSphere Business Services Fabric focuses on payments capabilities of financial services enterprises. See 7.4.1, “The Banking Payments Content Pack” on page 452. IBM Healthcare Payor Pack for WebSphere Business Services Fabric focuses on the payor processes in health care enterprises. See 7.4.2, “The Healthcare Payer Content Pack” on page 454 IBM Insurance P & C Pack for WebSphere Business Services Fabric focuses on the property and casualty systems for insurance enterprises. See 7.4.3, “The Insurance Property and Casualty Content Pack” on page 457 Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 451 IBM Product Lifecycle Management Pack for WebSphere Business Services Fabric focuses on the ecosystem of the manufacturers in automotive, aerospace and defense, and electronics verticals. This ecosystem includes suppliers, service providers, and other manufacturers. See 7.4.4, “The Product Lifecycle Management Content Pack” on page 459 IBM Telecom Operations Pack for WebSphere Business Services Fabric focuses on the operational support systems for telecommunications enterprises. See 7.4.5, “The Telecom Operations Content Pack” on page 461 7.4.1 The Banking Payments Content Pack Note: In the following steps we have specified our installations values as follows. Fabric install directory: /usr/lpp/webifyMP/V6R2 host name: wpsplex.itso.ibm.com host port: 20048 Specify your own installation values. The following sections detail the procedures to install and verify the installation of the Banking Payments Content Pack on z/OS. Installation Perform the following steps to install the Banking Payments Content Pack on z/OS: 1. Unzip/untar the install package for z/OS. 2. After ensuring that server is stopped, copy the following plugins from /usr/lpp/WebifyMP/V6R20/plugins to WPS_HOME\plugins directory: – com.ibm.websphere.repository.fabric.industry_6.2.0.jar – com.ibm.websphere.repository.fabric.banking.payments_6.2.0.jar 3. Start the server and install the Banking Payments Content Pack runtime artifacts for the WebSphere Process Server. 4. Locate the runtime directory where you specified the install location of the content pack files. 5. Install the EAR files listed in Table 7-1 on page 453 into the WebSphere Process Server. Use the Integration Solutions Console or the WebSphere Process Server’s administration client utility to install the EAR files using the JACL scripts located under the configuration folder within the Banking Payments Content Pack installation root directory. 452 z/OS: WebSphere Business Process Management V6.2 Production Topologies Table 7-1 Installing the runtime artifacts for WebSphere Process Server Component with which the Artifact is associated Artifact Name Location Banking Payments Content Pack Common Services banking-fabric.ear /usr/lpp/webifyMP/V6R20/ runtime Banking Payments Content Pack Reference Implementation banking-reference-app.ear /usr/lpp/WebifyMP/V6R20/ reference/vendor-payments/ runtime Banking Payments Content Pack Reference Implementation bk-app.ear /usr/lpp/webifyMP/V6R20/ reference/vendor-payments/ runtime 6. Import the Banking Payments Content Pack FCA files into the WebSphere Business Services Fabric by performing the following steps: a. Log on to the IBM WebSphere Business Service Fabric console at http://wpsplex.itso.ibm.com:20048/fabric/app b. From the left navigation pane, click My Services → Governance Manager → Import/Export. c. Load each .jar file in the specified order as listed in Table 7-2. Table 7-2 Importing Reference Implementation FCA files into Fabric Component the Artifact is associated with Artifact Name Banking Payments Content Pack Reference implementation banking-reference-implontology-model.jar banking-reference-implontology-lp.jar Location /usr/lpp/webifyMP/V6R20/ reference/vendor-payments/fabri c Verification To verify successful installation, confirm that the following applications are present in the WebSphere Process Server Integrated Solutions console: BPCP_Common_Services BPCP_Vendor_Payment_Mock_Services BPCP_Vendor_Payment_Process Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 453 To verify a successful installation of the Banking Payments Content Pack, perform the following steps: 1. Use the browser to access the IBM Business Services Foundation Pack at http://wpsplex.itso.ibm.com:20048/fabric/app 2. Log on to the WebSphere Business Services Fabric portal using your credentials. 3. In the My Services menu, click Governance Manager → Configure Projects. 4. If installation is successful, the following WebSphere Business Services Fabric projects are displayed: – Industry - ISO Codes Model (Project Type: Ontology) – Industry – Best Practices Fabric Model (Project Type: Ontology) – Industry – Best Practices Glossary Model (Project Type: Ontology) – Banking Payments Glossary Model (Project Type: Ontology) – Banking Payments Fabric Model (Project Type: Ontology) – Banking Payments Business Capability and Process Map (Project Type: Ontology) – Banking Payments Content Pack (Project Type: Legacy Business Service) – Vendor Payments Reference Implementation (Project Type: Legacy Business Service): This project must also be there in the listing, if the reference implementation FCA are imported. 7.4.2 The Healthcare Payer Content Pack Note: In the following steps we have specified our installations values as follows. Fabric install directory: /usr/lpp/webifyMP/V6R2 host name: wpsplex.itso.ibm.com host port: 20048 Make sure you specify your installation values. The following sections detail the procedures to install and verify the installation of the Healthcare Payer Content Pack on z/OS: 454 z/OS: WebSphere Business Process Management V6.2 Production Topologies Installation 1. Unzip/untar the install package for z/OS. 2. After ensuring that server is stopped, copy the following plugins from /usr/lpp/WebifyMP/V6R20/\plugins to WPS_HOME\plugins directory. – com.ibm.websphere.repository.fabric.industry_6.2.0.jar – com.ibm.websphere.repository.fabric.healthcare.payer_6.2.0.jar 3. Start the server and install the Healthcare Payer Content Pack runtime artifacts for the WebSphere Process Server. – Locate the runtime directory where you specified the install location of the content pack files. – Install the EAR files listed in Table 7-3 into the WebSphere Process Server. Use the Integration Solutions Console or the WebSphere Process Server’s administration client utility to install the EAR files using the JACL scripts located under the configuration folder within the Healthcare Payer Content Pack installation root directory. Table 7-3 Installing the runtime artifacts for the WebSphere Process Server Component Name Artifact Name Location Healthcare Payor Content Pack Common Services healthcare-fabric.ear /usr/lpp/webifyMP/V6R20/ runtime Healthcare Payor Content Pack Reference Implementation Services hc-app.ear /usr/lpp/webifyMP/V6R20/ reference/benefits-and-eli gibility/runtime Healthcare Payor Content Pack Reference Implementation healthcare-reference-app. ear /usr/lpp/webifyMP/V6R20/ reference/benefits-and-eli gibility/runtime 4. Import the Healthcare Payer Content Pack FCA files into the WebSphere Business Services Fabric by performing the following steps: a. Log on to the IBM WebSphere Business Service Fabric console at http://wpsplex.itso.ibm.com:20048/fabric/app b. From the left navigation pane, click My Services → Governance Manager → Import/Export. c. Load each .jar file in the specified order as listed in Table 7-4 on page 456. Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 455 Table 7-4 Importing Reference Implementation FCA files into Fabric Component Name Artifact Name Healthcare Payer Content Pack Reference Implementation healthcare-payor-reference-imp l.jar healthcare-payor-reference-imp l-lp.jar Location /usr/lpp/webifyMP/V6R20/refere nce/benefits-and-eligibility/fabric Verification To verify successful installation, confirm that the following applications are present in the WebSphere Process Server Integrated Solutions console: HPCP_Common_Services HPCP_Benefits_and_Eligibility_Mock_Services HPCP_Benefits_and_Eligibility_Process To verify a successful installation of the Healthcare Payer Content Pack perform the following steps: 1. Use the browser to access the IBM Business Services Foundation Pack at http://wpsplex.itso.ibm.com:20048/fabric/app 2. Log on to the WebSphere Business Services Fabric portal using your credentials. 3. In the My Services menu, click Governance Manager → Configure Projects. 4. If installation is successful, the following WebSphere Business Services Fabric projects are displayed: – Industry - ISO Codes Model (Project Type: Ontology) – Industry – Best Practices Fabric Model (Project Type: Ontology) – Industry – Best Practices Glossary Model (Project Type: Ontology) – Healthcare Payor Reference Implementation (Project Type: Legacy Business Service) : After importing the FCA files, this project gets displayed with the reference implementation listing. – Healthcare Payor Content Pack (Project Type: Legacy Business Service) – Healthcare Payor Business Capability and Process Map Model (Project Type: Ontology) – Healthcare Payor Fabric Model (Project Type: Ontology) – Healthcare Payor Glossary Model (Project Type: Ontology) 456 z/OS: WebSphere Business Process Management V6.2 Production Topologies 7.4.3 The Insurance Property and Casualty Content Pack Note: In the following steps we have specified our installations values as follows. Fabric install directory: /usr/lpp/webifyMP/V6R2 host name: wpsplex.itso.ibm.com host port: 20048 Make sure you specify your installation values. The following sections detail the procedures to install and verify the installation of the Insurance Property and Casualty Content Pack on z/OS: Installation 1. Unzip/untar the install package for z/OS. 2. After ensuring that server is stopped, copy the following plugins from /usr/lpp/WebifyMP/V6R20/\plugins to WPS_HOME\plugins directory. – com.ibm.websphere.repository.fabric.industry_6.2.0.jar – com.ibm.websphere.repository.fabric.insurance.pc_6.2.0.jar 3. Start the server and install the Insurance Property and Casualty Content Pack runtime artifacts for the WebSphere Process Server. – Locate the runtime directory where you specified the install location of the content pack files. – Install the EAR files listed in Table 7-5 into the WebSphere Process Server. Use the Integration Solutions Console or the WebSphere Process Server’s administration client utility to install the EAR files using the JACL scripts located under the configuration folder within the Insurance Property and Casualty Content Pack installation root directory. Table 7-5 Installing the runtime artifacts for WebSphere Process Server Component Name Artifact Name Location Insurance Property and Casualty Content Pack Common Services insurance-fabric-service.e ar /usr/lpp/webifyMP/V6R20/ runtime Insurance Property and Casualty Content Pack Reference Implementation Services ins-app.ear /usr/lpp/webifyMP/V6R20/ reference/claim-summaryacord-gateway/runtime Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 457 Component Name Artifact Name Location Insurance Property and Casualty Content Pack Reference Implementation ins-reference-app.ear /usr/lpp/webifyMP/V6R20/ reference/claim-summaryacord-gateway/runtime 4. Import the Insurance Property and Casualty Content Pack FCA files into the WebSphere Business Services Fabric. – Log on to the IBM WebSphere Business Service Fabric console at http://wpsplex.itso.ibm.com:20048/fabric/app – From the left navigation pane, click My Services → Governance Manager → Import/Export. – Load each .jar file in the specified order as listed in Table 7-6. Table 7-6 Importing the Reference Implementation FCA Component Name Artifact Name Insurance Property and Casualty Content Pack Reference implementation insurance-pc-reference-i mpl-fca.jar insurance-pc-reference-i mpl-fca-lp.jar Location /usr/lpp/webifyMP/V6R20/ reference/claim-summary/f abric Verification To verify successful installation, confirm that the following applications are present in the WebSphere Process Server Integrated Solutions console: IPCP_Common_Services IPCP_Claim_Summary_Mock_Services IPCP_Claim_Summary_Process To verify a successful installation of the Insurance Property and Casualty Content Pack, perform the following steps: 1. Use the browser to access the IBM Business Services Foundation Pack at http://wpsplex.itso.ibm.com:20048/fabric/app 2. Log on to the WebSphere Business Services Fabric portal using your credentials. 3. In the My Services menu, click Governance Manager → Configure Projects. 458 z/OS: WebSphere Business Process Management V6.2 Production Topologies 4. If installation is successful, the following WebSphere Business Services Fabric projects are displayed: – Industry - ISO Codes Model (Project Type: Ontology) – Industry – Best Practices Glossary Model (Project Type: Ontology) – Industry – Best Practices Fabric Model (Project Type: Ontology) – Insurance Glossary Model for Property and Casualty (Project Type: Ontology) – Insurance Fabric Model for Property and Casualty (Project Type: Ontology) – Insurance Business Capability and Process Map Model (Project Type: Ontology) – Insurance P&C Content Pack (Project Type: Legacy Business Service) – Insurance P&C Reference Implementation (Project Type: Legacy Business Service) : This project must be there in the listing if the reference implementation FCA files are imported. 7.4.4 The Product Lifecycle Management Content Pack Note: In the following steps we have specified our installations values as follows. Fabric install directory: /usr/lpp/webifyMP/V6R2 host name: wpsplex.itso.ibm.com host port: 20048 Make sure you specify your installation values. The following sections detail the procedures to install and verify the installation of the Product Lifecycle Management Content Pack on z/OS: Installation 1. Unzip/untar the install package for z/OS. 2. After ensuring that server is stopped, copy the following plugins from /usr/lpp/WebifyMP/V6R20/\plugins to WPS_HOME\plugins directory. – com.ibm.websphere.repository.fabric.industry_6.2.0.jar – com.ibm.websphere.repository.fabric.plm_6.2.0.jar Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 459 3. Start the server and install the Product Lifecycle Management Content Pack runtime artifacts for the WebSphere Process Server by performing the following steps. a. Locate the runtime directory where you specified the install location of the content pack files. b. Install the EAR files listed in Table 7-7 into the WebSphere Process Server. Use the Integration Solutions Console or the WebSphere Process Server’s administration client utility to install the EAR files using the JACL scripts located under the configuration folder within the Product Lifecycle Management Content Pack installation root directory. Table 7-7 Installing the runtime artifacts for WebSphere Process Server Component with which the Artifact is associated Artifact Name Product Lifecycle Management Content Pack Reference implementation plm-create-ecr-app.ear plm-analyse-ecr-app.ear plm-comment-ecr-app.ear plm-approve-ecr-app.ear plm-specfy-and-decide-ecr-app .ear plm-transform-ecr-app.ear Location /usr/lpp/webifyMP/V6R20/ reference/runtime 4. Import the Product Lifecycle Management Content Pack FCA files into the WebSphere Business Services Fabric. – Log on to the IBM WebSphere Business Service Fabric console at http://wpsplex.itso.ibm.com:20048/fabric/app – From the left navigation pane, click My Services → Governance Manager → Import/Export. – Load each .jar file in the specified order as listed in Table 7-8. Table 7-8 Importing the Reference Implementation FCA into Fabric Component Name Artifact Name Product Lifecycle Management Content Pack Reference implementation 460 plm-reference-impl.jar plm-reference-impl-lp.jar Location /usr/lpp/webifyMP/V6R20/ reference/fabric z/OS: WebSphere Business Process Management V6.2 Production Topologies Verification To verify a successful installation of the Product Lifecycle Management Content Pack, perform the following steps: 1. Use the browser to access the IBM Business Services Foundation Pack at http://wpsplex.itso.ibm.com:20048/fabric/app 2. Log on to the WebSphere Business Services Fabric portal using your credentials. 3. In the My Services menu, click Governance Manager → Configure Projects. 4. If installation is successful, the following WebSphere Business Services Fabric projects are displayed: – Industry - ISO Codes Model (Project Type: Ontology) – Industry – Best Practices Fabric Model (Project Type: Ontology) – Industry – Best Practices Glossary Model (Project Type: Ontology) – Product Lifecycle Management Glossary Model (Project Type: Ontology) – Product Lifecycle Management Fabric Model (Project Type: Ontology) – Product Lifecycle Management Business Capability and Process Map Model (Project Type: Ontology) – Product Lifecycle Management Content Pack (Project Type: Legacy Business Service) – Specify And Decide On Engineering Change Reference Implementation (Project Type: Legacy Business Service) : This project must be there in the listing if the reference implementation FCA files are imported. 5. To verify successful installation of the reference implementation, use the Web browser and access the Java Server Pages (JSP™) at http://wpsplex.itso.ibm.com:20048/SpecifyWeb/MROSystem.jsp. 7.4.5 The Telecom Operations Content Pack Note: In the following steps we have specified our installations values as follows. Fabric install directory: /usr/lpp/webifyMP/V6R2 host name: wpsplex.itso.ibm.com host port: 20048 Make sure you specify your installation values. Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 461 The following sections detail the procedures to install and verify the installation of the Telecom Operations Content Pack on z/OS: Installation 1. Unzip/untar the install package for z/OS. 2. After ensuring that server is stopped, copy the following plugins from /usr/lpp/WebifyMP/V6R20/\plugins to WPS_HOME\plugins directory. – com.ibm.websphere.repository.fabric.industry_6.2.0.jar – com.ibm.websphere.repository.fabric.telecom.operations_6.2.0.jar 3. Start the server and install the Telecom Operations Content Pack runtime artifacts for the WebSphere Process Server. – Locate the runtime directory where you specified the install location of the content pack files. – Install the EAR files listed in Table 7-9 into the WebSphere Process Server. Use the Integration Solutions Console or the WebSphere Process Server’s administration client utility to install the EAR files using the JACL scripts located under the configuration folder within the Telecom Operations Content Pack installation root directory. Table 7-9 Install the runtime artifacts for the WebSphere Process Server Component Name Artifact Name Location Telecom Operations Content Pack Common Services telecom-fabric.ear /usr/lpp/webifyMP/V6R20/r untime Telecom Operations Content Pack Reference Implementation Services tc-app.ear /usr/lpp/webifyMP/V6R20/r eference/check-order-feasi bility/runtime Telecom Operations Content Pack Reference Implementation telecom-reference-app.ear /usr/lpp/webifyMP/V6R20/r eference/check-order-feasi bility/runtime 4. Import the Telecom Operations Content Pack FCA files into the WebSphere Business Services Fabric by performing the following steps: a. Log on to the IBM WebSphere Business Service Fabric console at http://wpsplex.itso.ibm.com:20048/fabric/app b. From the left navigation pane, click My Services → Governance Manager → Import/Export. c. Load each .jar file in the specified order as listed in Table 7-10 on page 463. 462 z/OS: WebSphere Business Process Management V6.2 Production Topologies Table 7-10 Import the Reference Implementation FCA Component Name Artifact Name Telecom Operations Content Pack Reference Implementation telecom-operations-reference-i mpl.jar telecom-operations-reference-i mpl-lp.jar Location /usr/lpp/webifyMP/V6R20/refere nce/check-order-feasibility/fabric Verification To verify successful installation, confirm that the following applications are present in the WebSphere Process Server Integrated Solutions console: TOCP_Common_Services TOCP_Order_Feasibility_Mock_Services TOCP_Order_Feasibility_Process To verify a successful installation of the Telecom Operations Content Pack, perform the following steps: 1. Use the browser to access the IBM Business Services Foundation Pack at http://wpsplex.itso.ibm.com:20048/fabric/app 2. Log on to the WebSphere Business Services Fabric portal using your credentials. 3. In the My Services menu, click Governance Manager → Configure Projects. 4. If installation is successful, the following WebSphere Business Services Fabric projects are displayed: – Industry - ISO Codes Model (Project Type: Ontology) – Industry – Best Practices Glossary Model (Project Type: Ontology) – Industry – Best Practices Fabric Model (Project Type: Ontology) – Telecom Operations Reference Implementation (Project Type: Legacy Business Service): After importing the FCA files, this project gets displayed to the reference implementation listing. – Telecom Operations Content Pack (Project Type: Legacy Business Service) – Telecom Operations Business Capability and Process Map Model (Project Type: Ontology) – Telecom Operations Fabric Model (Project Type: Ontology) – Telecom Operations Glossary Model (Project Type: Ontology) Chapter 7. Installing and testing WebSphere Business Services Fabric for z/OS 463 464 z/OS: WebSphere Business Process Management V6.2 Production Topologies 8 Chapter 8. Monitoring the Business Process Management environment In this chapter we introduce WebSphere Business Monitor product as one of the service-oriented architecture (SOA) products stack key. Incorporating WebSphere Business Monitor into a WebSphere Process Server production topology assists business users to monitor runtime business processes. allowing them to make decisions or take actions based on the monitored runtime data. This chapter guides you through the necessary steps for installing WebSphere Business Monitor in a clustered topology on Linux on system Z environment in conjunction with WebSphere Process Server configured on z/OS with the BPM reference topology for z/OS as an event source. We cover the following subjects in this chapter: 8.1, “Overview of WebSphere Business Monitor” on page 466 8.2, “Incorporating WebSphere Business Monitor on Linux on System z with WebSphere Process Server for z/OS and the BPM reference topology for z/OS” on page 467 8.3, “Considerations for WebSphere Business Monitor installation” on page 468 © Copyright IBM Corp. 2009. All rights reserved. 465 8.4, “Installation of WebSphere Business Monitor distributed topology” on page 470 8.5, “Building WebSphere Business Monitor profiles” on page 476 8.6, “Creating the WebSphere Business Monitor clusters” on page 479 8.7, “Configuring the WebSphere Business Monitor infrastructure” on page 483 8.8, “Installing WebSphere Business Monitor support applications” on page 490 8.9, “Monitoring the BPM reference topology for z/OS using WebSphere Business Monitor” on page 505 8.10, “Additional Monitor SI Bus configuration for the WebSphere Process Server for z/OS cell” on page 527 8.11, “Verifying WebSphere Business Monitor functions with the BPM reference topology on z/OS” on page 537 8.1 Overview of WebSphere Business Monitor WebSphere Business Monitor is a comprehensive business-activity monitoring solution that provides a near-real-time view of your business performance. It measures business performance, monitors runtime and completed processes. It reports on business operations by processing events, calculating business metrics and presenting key performance indicators (KPIs) through business dashboards. Because WebSphere Business Monitor is not supported directly on z/OS, we must use a hybrid approach for this configuration. The basic installation is to install required CEI buses out of the WebSphere Process Server production environment to send event information to the WebSphere Business Monitor server running remotely in a Linux for System z LPAR. This chapter guides you through the necessary steps for installing the required CEI components in our production BPM topology. You must also configure the WebSphere Business Monitor server in a Linux for System z LPAR. Refer to WebSphere Business Process Management V6.2 Production Topologies, SG24-7732 for details on how to complete this configuration. 466 z/OS: WebSphere Business Process Management V6.2 Production Topologies 8.2 Incorporating WebSphere Business Monitor on Linux on System z with WebSphere Process Server for z/OS and the BPM reference topology for z/OS WebSphere Business Monitor is not currently available for z/OS. While we cannot therefore install WebSphere Business Monitor on z/OS, we can install it in a Linux on System z LPAR and use it to monitor WebSphere Process Server for z/OS configured in the BPM reference topology for z/OS. We chose to install WebSphere Business Monitor in a Linux on System z LPAR to take advantage of the high speed interconnect available to us on System z. By configuring our network to take advantage of hypersockets, we get the best of both worlds. WebSphere Process Server for z/OS running in a z/OS LPAR and WebSphere Business Monitor running in Linux for System z connected with hypersockets utilizes proven technology. This System z hybrid configuration is depicted in Figure 8-1 on page 468. Chapter 8. Monitoring the Business Process Management environment 467 Controller MDSR1A Controller MDSR1B Servant MDSR1AS (WPS) Servant MDSR1BS (WPS) BPC (HTM and BFM) CEI BPC Explorer BRM Business Space Fabric Monitor BPC (HTM and BFM) CEI BPC Explorer BRM Business Space Fabric Monitor SCA APP SCA SYS ME ME CEI ME Adjunct MDSR1AA BPC Mon ME ME Adjunct MDSR1BA mdsr1 Cluster Member 1 Business Spaces AlphaBlox Member 2 Business Spaces AlphaBlox Member 1 Model Logic Module Member 2 Moderator Module Member 1 Model Logic Module Member 2 Model Logic Module Member 1 (Mon) Action Services DM Service CEI Rest Services Member 2 (Mon) Action Services DM Service CEI Rest Services Monitor Model Logic Cluster Monitor Support Cluster (Mon Support) Member 2 (WAS) monNode01 monNode02 Monitor Bus Member 1 (WAS) CEI Bus BPC Mon ME ME SCA.APP Bus CEI ME SCA.SYSTEM Bus mdnodeb Monitor Model Moderator Cluster BPC Bus Monitor Bus CEI Bus SCA.APP Bus SCA.SYSTEM Bus BPC Bus SCA APP SCA SYS ME ME mdnodea Business Space Center Cluster Messaging Cluster (Messaging) Figure 8-1 WebSphere Business Monitor and WebSphere Process Server for z/OS hybrid topology 8.3 Considerations for WebSphere Business Monitor installation This section outlines the installation considerations for WebSphere Business Monitor. 8.3.1 Prerequisite software This chapter describes how to install WebSphere Business Monitor network deployment. This topology requires the following components: 468 z/OS: WebSphere Business Process Management V6.2 Production Topologies WebSphere Application Server V6.1.0.21 WebSphere Process Server V6.2 DB2 UDB ESE V8.2 fix pack 8 on z/OS IBM Tivoli Directory Server V6.0 or V6.1 For more information regarding the supported WebSphere Business Monitor products, refer to the following Web page: http://www-01.ibm.com/software/integration/wbimonitor/requirements/ 8.3.2 Installation overview This chapter describes how to create and establish the topology shown in previous sections. The installation steps in this chapter were performed in Linux for System z. The topology contains the following clusters: Message engine cluster – Message engine for the WebSphere Business Monitor bus – Message engine for the CEI bus WebSphere Business Monitor support cluster – – – – CEI event service Monitor action services Data services scheduler Monitor event emitter service (EmitterRestServices.ear) Monitor model moderator cluster Monitor model moderator module Monitor model logic cluster Monitor model logic module Business Space cluster – – – – – Business Space application Business Space widgets REST services application AlphaBlox® Mobile dashboards (optional) Note: In this topology, Monitor database is created on a DB2 instance on a z/OS machine separately from the WebSphere Business Monitor installation machines. In this chapter, the WebSphere Business Monitor cluster is installed over two machines. Machine one contains the WebSphere Business Monitor deployment manager and monNode01. Machine two contains monNode02. Chapter 8. Monitoring the Business Process Management environment 469 8.4 Installation of WebSphere Business Monitor distributed topology In the following sections we describe in details the required steps to install the WebSphere Business Monitor cluster on Linux for System z. 8.4.1 Installing WebSphere Business Monitor deployment manager Install WebSphere Business Monitor in the /opt/ibm/WebSphere/MonServer directory on the Monitor Deployment manger machine. You can use the WebSphere Business Monitor installation wizard GUI by using X Windows® configuration. Note: If you will use a remote database, the monitor database should be created on the remote server before proceeding to the following steps. The database could be created without tables and after installation you can run the database scripts to create the tables as in next section. 1. As root use the Install Wizard to install WebSphere Business Monitor media to the /opt/ibm/WebSphere/MonServer directory. 2. Run <media install root >/WBM/install a. Click Next. b. Accept the license agreement and select Advanced Installation. c. Be sure to check Business Monitor Server including Business Space in the installation component list. Click Next. d. Select Install a new copy of Business Monitor Server. e. Select Alphablox features for Business Space from the list. In a network deployment environment, Alphablox server should be manually installed on the support cluster after completing the WebSphere Business Monitor installation. f. Specify /usr/IBM/WebSphere/MonServer as the installation directory. g. Select Deployment manager from the list. It will create Monitor deployment manager profile after completing the WebSphere Business Monitor installation. h. Select Enable administrative security. Type a sample username and password. 470 z/OS: WebSphere Business Process Management V6.2 Production Topologies i. Select DB2 Universal Database™ for z/OS v8 and the Use existing database option. After completing installation, run the generated DDL script on the DB2 server manually to create the monitor database. j. Select type 4 as the JDBC driver type. k. Let the launchpad finish the installation. 8.4.2 Creating the WebSphere Business Monitor databases WebSphere Business Monitor uses a single database for persistence. The default name is MONITOR. We create this database on the same DB2 server used by the existing WebSphere Process Server topology on z/OS. Configuring the Monitor database using DB2 connect Perform the following steps to create the WebSphere Business Monitor database tables. 1. Perform the following steps to prepare the database creation scripts for execution. a. Locate the script createDatabaseDb2.ddl, found at /opt/ibm/WebSphere/MonServer/scripts.wbm/database. b. Edit the following variables in the createDatabaseDb2Zos.dll script: • $DBNAME$: This variable represents the name of the Monitor database (Use MONITOR). • $SCHEMA$: This variable represents the name of the Monitor schema (Use MONITOR). c. Save and close the file as createDatabaseDb2Zos.alter.ddl d. Make the new file executable: chmod 755 createDatabaseDb2Zos.alter.ddl 2. Open the DB2 command line interface and run the createDatabaseDb2Zos.ddl script using the following command: db2 -tf createDatabaseDb2Zos.alter.ddl 3. Bind the command line interface to the Monitor database using the following commands: db2 connect to MDMONDB db2 bind /home/db2inst1/sqllib/bnd/@db2cli.lst blocking all grant public db2 connect reset 4. The result of bind command should be as shown in Example 8-1. Chapter 8. Monitoring the Business Process Management environment 471 Example 8-1 Bind command result LINE MESSAGES FOR db2cli.lst -------------------------------------------------------------------SQL0061W The binder is in progress. SQL0091N Binding was ended with "0" errors and "0" warnings. Configuring the Monitor database using local z/OS utilities If you prefer to execute the SQL statements on the local z/OS system instead of using DB2 connect, there are several options: Batch program DSNTEP2 SPUFI DBUtility.sh script provided with WebSphere Process Server for z/OS Note: The SQL statements are not well-formed for execution using the DB2 DSNTEP2 batch program or SPUFI, because many statements are much longer than 72 columns. The DBUtility.sh is supplied with WebSphere Process Server for z/OS in the {app_server_root}/bin/ directory of the configuration HFS (For example: /wasv61config/mdcell/mddmnode/DeploymentManager/bin/). You can use this to execute some SQL statements that you receive from another installation, such as the WebSphere Business Monitor installation on another system. Here is an example of using it to execute the SQL statements that you may have on your workstation: 1. Edit the file with WordPad and save it on your local workstation. This adds the cr/lf characters so the record delimiters will be proper when transferred to the z/OS system. 2. Use FTP to transfer the file in binary mode to your working UNIX System Services directory on z/OS. 3. Edit the SQL file, changing the following values for your installation: a. $STOGRP$: This variable represents the name of the Monitor database. (We used MDMONSG.) b. $DBNAME$: This variable represents the name of the Monitor database. (We used MDMONDB.) c. $SCHEMA$: This variable represents the name of the Monitor schema prefix. (We used MDMON.) 472 z/OS: WebSphere Business Process Management V6.2 Production Topologies Note: You must use a separate schema prefix from the BPC and WebSphere Process Server Common database tables because some of the Monitor table names are identical table names in those databases. 4. Make sure the file permissions allow it to be read by the following script. 5. Create a shell script to invoke the DBUtility.sh script using Example 8-2 as a template: Example 8-2 Sample DBUtility.sh script #!/bin/sh # # This shell script will run the DButility.sh script # pointing to the sql statements for # WebSphere Business Monitor. # Make sure there are no trailing blanks after the back-slashes! # $WAS_HOME/bin/DBUtility.sh createTable \ -DdbStorageGroup=MDMONSG \ -DdbSchemaName=MDMON \ -DsqlScriptName.default=BusMon_createTables.sql \ -DsqlScriptPath.default=/var/WebSphere/home/MDCFG/wpswork/BusMon_cre ateDatabaseDb2Zos.sql \ -DdbType=DB2UDBOS390_V8_1 \ -DdbName=MDMONDB \ -DprofileName=default \ -DprofilePath=$WAS_HOME/profiles/default \ -DdbJDBCProperties=/etc/d9fg \ -DdbConnectionLocation=DB9F \ -DdbJDBCClasspath=/usr/lpp/db2/d9fg/db2910_jdbc/classes \ -DdbUserId=MDADMIN -DdbPassword=MDADMIN \ -DdbDelayConfig=false \ -DdbCreateNew=false \ -DdbHostName=wtsc42.itso.ibm.com \ -DdbServerPort=37893 \ >/tmp/mdcell/BusMonDdl.out 2>/tmp/mdcell/BusMonDdl.err 6. Export the LIBPATH for the DB2 libraries: EXPORT LIBPATH=/usr/lpp/db2/d9fg/db2910_jdbc/lib:$LIBPATH 7. Export the $WAS_HOME and $STEPLIB variables by changing to the <WAS_HOME>/bin/ directory and running the setupCmdLine.sh script as shown in Example 8-3 on page 474: Chapter 8. Monitoring the Business Process Management environment 473 Example 8-3 Running the setupCmdLine.sh script cd /wasv61config/mdcell/mdnodea/AppServer/bin . setupCmdLine.sh Notice that the command . setupCmdLine.sh is preceded by a period and a space (“.”). Running the command in this fashion retains the export of WAS_HOME that is performed by setupCmdLine.sh and makes WAS_HOME available to the createDB.sh script. Verify the variables that were set by running the setupCmdLine.sh script and the user ID with which you are running, using the UNIX commands in Example 8-4. Example 8-4 Verifying the variables set for running DBUtility.sh >id uid=209(MDADMIN) gid=16(MDCFG) >echo $WAS_HOME /wasv61config/mdcell/mdnodea/AppServer >echo $STEPLIB none:BBO6142.MDNODEA.SBBOLOAD:BBO6142.MDNODEA.SBBOLD2: BBO6142.MDNODEA.SBBOLPA: DB9F9.SDSNEXIT:DB9F9.SDSNLOAD:DB9F9.SDSNLOD2 >echo $LIBPATH usr/lpp/db2/d9fg/db2910_jdbc/lib:. . . 8. Return to your working directory, and run your modified script to execute the DBUtility.sh script. 9. Verify that the script ran successfully and the sql statements were run by looking in the following places: a. /tmp/mdcell/BusMonDdl.out a. /tmp/mdcell/BusMonDdl.err b. The DB2 Administrator’s Utility (ADBL) on the z/OS system (or SPUFI) to see that the 95 tables and 97 table spaces were created for the Monitor. 10.If you need to modify the SQL statements and re-run the script, you must delete the sql file in the <profile_root>/dbscripts/CommonDB/DB2zOSV8/MDMONDB directory or else the old sql statements will be used. 474 z/OS: WebSphere Business Process Management V6.2 Production Topologies If you want to see the detailed results of the shell script you are running, you can precede it with the “sh -x” string as in Example 8-5: Example 8-5 Invoking a shell script with tracing sh –x $WAS_HOME/bin/DBUtility.sh createTable \ -DdbStorageGroup=MDMONSG \ -DdbSchemaName=MDMON \ -DsqlScriptName.default=BusMon_createTables.sql \ . . . 8.4.3 Installing WebSphere Business Monitor nodes Repeat the below steps for each node that is required to be included in the WebSphere Business Monitor cluster. Install WebSphere Business Monitor on the /opt/ibm/WebSphere/MonServer directory. 1. As root use the Install Wizard to install WebSphere Business Monitor media to the /opt/ibm/WebSphere/ProcServer directory. This will add WebSphere Business Monitor to the WebSphere Process Server currently installed and configured. 2. Verify the Deployment Manger is stopped. 3. Run <media install root >/WBM/install a. Click Next. b. Select I accept both the IBM and the non IBM terms. c. Click Next, and on the new panel, click Next again. d. Select Advanced Installation, clear the Information Center check box, and click Next. e. Use the existing version of WebSphere Application Server Network Deployment. The value should be /opt/ibm/WebSphere/ProcServer. f. Click Next. g. Select None and click Next. h. Click Yes on the pop-up window warning about not creating a profile. i. Verify your Installation Summary. Click Next, then click Finish. Chapter 8. Monitoring the Business Process Management environment 475 8.5 Building WebSphere Business Monitor profiles This section describes how to prepare and build WebSphere Business Monitor custom profiles. 8.5.1 Preparing to build profiles In preparation to augment and create the profiles for this configuration, you will need to collect information in advance. Table 8-1 contains the information you will need prior to starting the process. Table 8-1 Information needed before starting profile creation process 476 Name Value Cell security ID admin Cell security ID password admin Database Name MDMONDB Database Schema name MDMONDB Database User Name db2inst1 Database User Password passw0rd Location (Directory) of JDBC driver classpath files on Dmgr host /opt/ibm/WebSphere/MonServer/universal .wmb/lib JDBC Driver type 4 Database host name wpsplex Database TCP service port 37893 Deployment Manager host name lnxsu1 DMGR SOAP Port 8879 Monitor Cluster Node host name lnxsu2 Location (Directory) of JDBC driver classpath files on monitor host /opt/ibm/WebSphere/MonServer/universal .wmb/lib z/OS: WebSphere Business Process Management V6.2 Production Topologies 8.5.2 Create WebSphere Business Monitor custom profiles After installing the cluster members binaries in you should create a profile for each cluster member node and federate it to the WebSphere Business Monitor deployment manager. The following section describes the required steps for creating node profiles and federating them into the deployment manager. Creating and federating clusters members: It is suggested to create separate profiles for WebSphere Business Monitor cluster members instead of augmenting them with the WebSphere Process Server profiles. It is a mandatory that the timing between any node machine and the deployment manager machine be less than 2 minutes. If the timing is greater than 2 minutes the federation of the profile to the deployment manager will fail. To create and federate cluster member nodes, perform the following steps: 1. Create the WebSphere Business Monitor custom profile: # cd /usr/IBM/WebSphere/MonServer/bin # ./manageprofiles.sh -create -templatePath /usr/IBM/WebSphere/MonServer/profileTemplates/wbmonitor/managed -profileName monNode01 -profilePath /usr/IBM/WebSphere/MonServer/profiles/monNode01 -nodeName lnxsu1 -hostName lnxsu1 -federateLater true Notes: It is strongly recommended to add to the hosts file (/etc/hosts) the IP/host name pairs for all nodes and deployment manager machines. If successful, you will see the following message: INSTCONFSUCCESS: Success: Profile monN01 now exists. Please consult /usr/IBM/WebSphere/MonServer/profiles/monNode01/logs/AboutThisProfil e.txt for more information about this profile. 2. Federate WebSphere Business Monitor custom profiles into the deployment manager. – On machine 1: # cd /usr/IBM/WebSphere/MonServer/profiles/monNode01/bin # ./addNode.sh lnxsu1 8879 –username user –password password Node lnxsu1 has been successfully federated. Chapter 8. Monitoring the Business Process Management environment 477 – On machine 2: # cd /usr/IBM/WebSphere/MonServer/profiles/monNode02/bin # ./addNode.sh lnxsu1 8879 –username user –password password Node lnxsu1 has been successfully federated. Notes: lnxsu1 is the WebSphere Business Monitor deployment manager machine host name and it contains one node. lnxsu2 is a second machine host name which contains one WebSphere Business Monitor node. 8879 is the soap port number In this book, we used two nodes on two different machines. Create a second custom profile. Repeat the previous steps, but change the profile name and node name. 1. From a terminal window you can issue -→ ps -elf | grep nodeagent. This should produce one Java process. This is the node that you just created and is federated to the deployment manager. 2. Open a web browser to access the Integrated Solutions Console. http://lnxsu1:9060/admin 3. Enter username and password and click login. 4. Expand System Administration and click Nodes. Figure 8-2 on page 479 should show three entries with a status of synchronized. The entries are as follows: – lnxsu1CellManager01 – lnxsu1 – lnxsu2 478 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 8-2 Nodes Panel Integrated Solutions Console 8.6 Creating the WebSphere Business Monitor clusters In this section, you are building clusters for the WebSphere Business Monitor applications to run. There will be five clusters added to the cell. WebSphere Business Monitor support cluster Monitor Model logic cluster Monitor Model Moderator cluster. Business Space cluster Messaging cluster The WebSphere Business Monitor support cluster will house the following applications and services: Action Services Data Movement Service Event Service The Monitor Model logic cluster will house the Monitor model logic part application. The Monitor Model Moderator cluster will house the Monitor model moderator part application. Chapter 8. Monitoring the Business Process Management environment 479 The Business Space cluster will house the following applications: AlphaBlox Business Space REST API Service The WebSphere Business Monitor messaging cluster will house the following applications and services: Monitor Bus Messaging Engine CEI Bus Messaging Engine To create these clusters, perform the following steps: 1. Log in to the Integrated Solutions Console. http://svsles:9060/admin 2. Expand Servers → Clusters. 3. Click New. 4. Enter MonSupCluster for the Cluster name. Click Next. 5. Create a cluster member named MonSup01 on the lnxsu1 using server template default_defaultWBM. Click Next (Figure 8-3 on page 481). 480 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 8-3 New Cluster wizard Step 2 6. Create the second cluster member using the values in Table 8-2, then click Add Member. After you have finished creating all of the cluster members click Next (Figure 8-4 on page 482). Table 8-2 WebSphere Business Monitor support cluster values Name Value SelectNode Server Template Cluster Name MonSupCluster Member Name MonSup01 lnxsu1 default_defaultWBM Member Name MonSup02 lnxsu2 default_defaultWBM Chapter 8. Monitoring the Business Process Management environment 481 Figure 8-4 New Cluster wizard Step 4 Note: If you do not click AddMember after you enter the second cluster member, your cluster will only have one cluster member. 7. Review the summary and click Finish. Save the changes to the master configuration repository. 8. Repeat these steps to create each WebSphere Business Monitor cluster. Once the five clusters are completed and started, the main window of the Integrated Solutions Console’s cluster panel should look like Figure 8-5 on page 483. 482 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 8-5 Server Clusters panel of Integrated Solutions Console Note: If you are going to enable security for the installation of the WebSphere Business Monitor clustered environment, it is strongly recommended to enable security at this stage of the installation and configuration process to avoid any reconfiguration problems. 8.7 Configuring the WebSphere Business Monitor infrastructure The communication between the WebSphere Process Server cell and the WebSphere Business Monitor cell will be through cross cell configuration. The WebSphere Business Monitor Infrastructure needs to have access to a CEI Server, Monitor and CEI Bus and Event Emitters to transmit outbound events. Chapter 8. Monitoring the Business Process Management environment 483 8.7.1 Deploy CEI Service on WebSphere Business Monitor support cluster Using the command line, deploy the CEI Server on the WebSphere Business Monitor Support Cluster. 1. Enable CEI service on the support cluster. On the deployment manager machine (machine 1), run the commands shown in Example 8-6. Example 8-6 Enable CEI service # cd /usr/IBM/WebSphere/MonServer/bin # ./wsadmin.sh -username admin -password admin WASX7209I: Connected to process "dmgr" on node lnxsu1CellManager01 using SOAP connector; The type of process is: DeploymentManager WASX7029I: For help, enter: "$Help help" wsadmin>$AdminTask deployEventService { -clusterName SupCluster } (This command will enable CEI on the SupCluster) wsadmin>$AdminConfig save wsadmin>$AdminTask configEventServiceDB2DB { -clusterName SupCluster -jdbcClassPath /usr/IBM/WebSphere/MonServer/universalDriver.wbm/lib -dbHostName lnxsu1 -dbName MONCEI -dbUser db2inst1 -dbPassword db2inst1 } (This command will configure the CEI to use a DB2 database as the event persistence) wsadmin>$AdminConfig save wsadmin>quit 2. Run the generated DB scripts to create the CEI database. The generated DB scripts are located in: /usr/IBM/WebSphere/MonServer/profiles/Dmgr01/databases/event/ MonSupCluster/dbscripts/db2 Copy the whole directory to the DB2 server machine and run the following script: # ./cr_event_db2.sh This script will create the CEI DB and related tables on DB2 server. 484 z/OS: WebSphere Business Process Management V6.2 Production Topologies 3. Create CEI message engine data source by performing the following steps: a. Create this data source in cell scope. In the Integrated Solutions Console click Console → Resources → JDBC → Data sources. Select Cell scope and click New. b. Specify the following values: • • • Data source name: CEI ME Database JNDI: jdbc/ceimedb Authentication alias: Monitor_JDBC_Alias Note: Record the JNDI of the data source. You will need it later. c. Select the existing JDBC Provider MonitorDBProvider. d. Specify the following values: • • • Database name: MONCEI Driver type: 4 Database server host name and port e. Click Finish to complete the wizard. 4. Configure the CEI message engine data store by performing the following steps: a. In the Integrated Solutions Console click Service integration → Buses → CommonEventInfrastructure_Bus → Messaging engines → MonSupCluster.000-CommonEventInfrastructure_Bus → Message store. b. Fill the JNDI of the data source you just created and select Monitor_JDBC_Alias as the authentication alias. c. Click OK and save the changes. 5. Configure the security settings for the CEI service: a. Add MonitorBusAuth as the Component-managed authentication alias for the following resources in the Integrated Solutions Console: • • • Queue connection factory: CommonEventInfrastructure_QueueCF Topic connection factory: CommonEventInfrastructure_AllEventsTopicCF Activiation specification: CommonEventInfrastructure_ActivationSpec b. Click Service integration → Buses → Security for bus CommonEventInfrastructure_Bus. Add MonitorBusAuth as the Inter-engine authentication alias of CEI Bus. c. Select Allow the use of all defined transport channel chains. d. Add the admin user as the bus connector role. Chapter 8. Monitoring the Business Process Management environment 485 Note: Restart all clusters, node agents, and deployment manager to let the changes take effect. 8.7.2 Create a Datasource for the Monitor Messaging Engine In the topology, all of the messaging engines will be defined on the Monitor Messaging cluster. You need to create a datasource for the Monitor Messaging Engine datastore. In this IBM Redbooks publication, you create a new schema in the MEDB. The best practice is to create a separate datasource for isolation. In the Integrated Solutions Console. create a new Datasource following these steps: 1. Create a JAAS - J2C authentication data alias a. Expand Security → Secure administration, applications, and infrastructure → authentication → Java Authentication and Authorization Service → J2C authentication data. b. Click New. c. Enter the values in the Table 8-3 into the new J2C Authentication. Click OK. Table 8-3 Monitor ME Authentication Alias values: Monitor ME Datastore Name Value Alias Monitor ME Auth Alias User ID db2inst1 Password passw0rd Description Monitor ME datastore d. Create a New DataSource. a. Expand Resources → JDBC _> Datasources. b. Change the scope to Cluster = RMS.Messaging. c. Click New. d. In the Create new Datasource wizard enter the values in Table 8-4 on page 487. Click Next. 486 z/OS: WebSphere Business Process Management V6.2 Production Topologies Table 8-4 DataSource values Name Value Data Source Name MonitorMEDatabase JNDI Name jdbc/monitorMEDB Component-managed authentication alias and XA recovery authentication alias CellManager01/Monitor ME AuthAlias e. Select the Select an existing JDBC Provider radio button and DB2 Universal JDBC Driver Provider from the drop-down menu. Click Next. f. Enter the values in Table 8-5. Click Next. Table 8-5 Database specific properties Name Value Database Name MEDB Driver type 4 Server name lnxsu1 Port Number 37893 g. Review the Summary panel shown and click Finish. h. Restart the Deployment Manager to have it pick up the new J2C Authentication Alias. i. Restart the WebSphere Process Server nodes to pick up the newly created datasource. j. Use the test connection button on the Datasource page to verify that the datasource is properly working. 8.7.3 Using the WebSphere Business Monitor configuration panel This is a new edition to WebSphere Business Monitor v6.2. It will allow you one location within the Integrated Solutions Console to view and change the WebSphere Business Monitor configuration. The wizard looks like Figure 8-6 on page 488. Chapter 8. Monitoring the Business Process Management environment 487 Figure 8-6 WebSphere Business Monitor configuration wizard panel Configure the Monitor Messaging Engine Perform the following steps to configure the Monitor Messaging Engine: 1. Launch the WebSphere Business Monitor configuration panel by performing the following steps. a. Log in to the Integrated Solutions Console. b. Expand Servers. c. Click WebSphere Business Monitor configuration. 2. Click the Message engine link under the component column of the panel. This will launch the messaging engine wizard shown in Figure 8-6. Click Configure the Messaging Engine. 3. Select the cluster radio button, select MonMECluster, then click Next. 4. Select the Data store radio button. Click Next. 5. Enter the values in Table 8-6 on page 489. Ensure the Create Tables check box is selected. Click Next. 488 z/OS: WebSphere Business Process Management V6.2 Production Topologies Table 8-6 Use existing Data source values Name Value Data Source JNDI Name jdbc/monitorMEDB Schema Name MONITOR Authentication Alias CellManager01/Monitor_JDBC_Alias 6. Review the summary and click Finish. Create Event Emitter factory Perform the following steps to create the Event Emitter factory. 1. Launch the WebSphere Business Monitor configuration panel. a. Login to the Integrated Solutions Console. b. Expand Servers. c. Click WebSphere Business Monitor configuration. 2. Click the Event Emitter Factory link. 3. Select WBM Support from the Configure an event emitter factory drop-down menu. Click Configure an event emitter factory. You should get a success message. 4. Navigate to Service Integration→ Common Event Infrastructure→ Event Emitter Factory. 5. Review Event Emitter Factories. You should see MonitorEmitterFactory. 6. Assign the event service transmission for the MonitorEmmiterfactory. Expand Service integration → Common Event Infrastructure → Event emitter factories → MonitorEmitterFactory → Additional Properties → Event service transmission. 7. Enter the values shown in Table 8-7 into the Event Service Transmission panel. Table 8-7 Table of values for default event transmission Name Value Name MonSupCluster default event service transmission JNDI com/ibm/events/configuration/bus-transmission/Default Event Service JNDI cells/clusters/MonSupCluster 8. In the Review panel, click OK. Click Save. Chapter 8. Monitoring the Business Process Management environment 489 8.8 Installing WebSphere Business Monitor support applications The section describes how to install the following support applications: Deploy Action Manager Deploy database service scheduler Deploy REST API service DB2 Alphablox installation Deploying Business Space Configure Business Space for dashboard widgets. Deploy monitor event emitter service (optional). Note: The monitor event emitter service ( EmitterRestServices.ear ) is not deployed using wizard. It is deployed manually. It is optional as it used to send XSD style events using Monitor API. The event emitter service is not used in the current installation. 8.8.1 Deploy Action Manager Perform the following steps to deploy Action Manager. 1. Verify the WBM clusters are in a started state. Expand Servers →Clusters. 2. Launch the WebSphere Business Monitor configuration panel by performing the following steps. a. Log in to the Integrated Solutions Console. b. Expand Servers. c. Click WebSphere Business Monitor configuration. 3. Click the Action Service link on the configuration panel. 4. Select MonSupCluster in the Deploy action services list box. Click Deploy Action Service. You should see the message shown in Figure 8-7. Figure 8-7 Action Manager success message 490 z/OS: WebSphere Business Process Management V6.2 Production Topologies Note: If deployment fails, verify that you have properly configured your /etc/security/limits.conf on your deployment manager machine. The nofile should be configured to * soft nofile 10240 and * hard nofile 10240. These should be set on all machines WebSphere Business Monitor is installed. Note: To enable the monitor to send events using action manager functionalities, you have to change CSIv2 inbound and outbound configuration. Perform the following steps for each member in Monitor model logic cluster. 1. Go to Application servers→ MMLogicMember01→ Server security→ CSIv2 outbound authentication. 2. In the Basic authentication area, select Supported. 3. In the Client certificate authentication area, select Supported. 4. Ensure that Identity assertion is selected 5. Ensure that Use server trusted identity is selected. Perform the following steps for each member in Monitor support cluster. 1. Go to Application servers→ WBMSupportMember01→ Server security→ CSIv2 inbound authentication. 2. In the Basic authentication area, select Supported. 3. In the Client certificate authentication area, select Supported. 4. Ensure that Identity assertion is selected. 5. Ensure that Stateful sessions is selected. 8.8.2 Deploy database service scheduler Perform the following steps to deploy the database service scheduler: 1. Launch the WebSphere Business Monitor configuration panel by performing the following steps: a. Log in to the Integrated Solutions Console. b. Expand Servers. c. Click WebSphere Business Monitor configuration. 2. Click the Data services scheduler link on the configuration panel. 3. Select MonSupCluster in the Deploy data services scheduler list box. Click the Deploy Data Services Scheduler button. Chapter 8. Monitoring the Business Process Management environment 491 8.8.3 Deploy REST API service Perform the following steps to deploy the REST API service 1. Launch the WebSphere Business Monitor configuration panel by performing the following steps: a. Log in to the Integrated Solutions Console. b. Expand Servers. c. Click WebSphere Business Monitor configuration. 2. Click the REST API service link on the configuration panel. 3. Select BusinessSpaceCluster in the Deploy REST API service list box. Click Deploy REST API service. Note: After completing services configuration, we should note that these services are deployed, but they are not running. You will have to expand Applications → Enterprise Applications and check the service. Click Start. If you are going to provide the option of viewing the dashboard through their mobile device, you will need to deploy the Dashboard application for Mobile devices on business space cluster. You will need to follow the same steps. 8.8.4 DB2 Alphablox installation You will install Alphablox software on all machines in the cluster where the Business Space Manager is to be deployed. Note: In this Redbooks publication, you are installing the Business Space Manager on the BusinessSpaceCluster cluster. Install AlphaBlox to the deployment manager Perform the following steps to install AlphaBlox to the deployment manager: 1. Stop all Monitor clusters before installing Alphablox. 2. Run the Alphablox installer located in the following directory: cd <WebSphere Business Monitor CD image>/ABX/Linux/ # ./install.bin 3. In Choose Locale, select English. 4. Accept the licence agreement. 5. Enter a Destination Directory of /opt/ibm/WebSphere/MonServer/ABX 6. Accept the default Server Instance Name of AlphabloxAnalytics. 492 z/OS: WebSphere Business Process Management V6.2 Production Topologies 7. Select an Installation Set of Typical. 8. Select the application server to use with Alphablox as WebSphere. 9. Enter a WebSphere Root Directory of /opt/ibm/WebSphere/MonServer 10.Select the node and server to install Alphablox on the required profile. Note: In this topology Alphablox application is installed on WBM.BusSpace cluster. 11.Accept the default values for HTTP Port and SOAP Connector Port. 12.Provide a WebSphere Administrator username and password. 13.Accept default values for Telnet Console Port and Server Log File Name. 14.Set the Console Message Level to 3. 15.Set the SMTP Server and Java Directory to default values. 16.Enter Y to enable additional drivers for IBM Alphablox 17.Enter the location of the driver as /opt/ibm/WebSphere/MonServer/universalDriver.wbm/lib and confirm that this drive is a DB2 Type 4 driver. 18.Select DB2 as the database repository. 19.Set the database Server to db2zos, the Port to 37893 and the Alias to MONITOR. This alias is the name of the database that will be used as an Alphablox repository, and can share the same name with the Monitor database. 20.Specify a User name and Password of db2inst1. 21.The installer will run a database connection test. Ensure the test runs successfully. A successful test will note that a series of tables were not found (Example 8-7). They will be created when the Alphablox server is started for the first time. Example 8-7 Missing alphablox system tables Alphablox Alphablox Alphablox Alphablox Alphablox Alphablox system system system system system system table table table table table table ABX_OBJECTS not found ABX_TYPES not found ABX_VERSION not found ABX_LOOKUP not found ABX_PROPERTY_MAP not found ABX_LOOKUP_VALUES not found 22.Enter 1 to configure clustering and accept the default values for Cluster Port Number and Cluster Subnet Mask. The Cluster Port Number should be identical for all Alphablox server instances in the cluster. Chapter 8. Monitoring the Business Process Management environment 493 23.Select Copy as the Conversion Operation. 24.Set Move Server Properties to All. 25.Select No for the User-defined DDL schema file. 26.Review the summary shown in Example 8-8 and let the installer complete the remainder of the installation. Example 8-8 Summary Installation Directory: /opt/IBM/WebSphere/MonServer/ABX Instance Name: AlphabloxAnalytics Application Server: WebSphere WebSphere Home: /opt/ibm/WebSphere/MonServer WebSphere Product: IBM WebSphere Application Server - ND WebSphere Version: 6.1.0.21 WebSphere Start File: setupCmdLine.sh WebSphere Cluster Install: true WebSphere Profile: BusinessSpace01 WebSphere Cell: WPSCell01 WebSphere Node: monNode01 WebSphere Server: dmgr HTTP Request Port: 9080 SOAP Connector Port: 8879 SOAP Admin User: wps Telnet Console Port: 20023 Server Log File Name: Server.log Console Message Level: INFO Java Directory: /opt/ibm/WebSphere/MonServer/java Additional Driver Directory: /opt/ibm/WebSphere/MonServer/universalDriver.wbm/libDB2 Driver Type: Drivers: Enabled: DB2 Type 4, Derby Repository Type: Database Database Type: DB2 Database Server: db2Zos Database Port: 37893 Database Alias: MONITOR Database User: db2inst1 CLUSTERING: Enabled: Port:: 7855 Subnet Mask:: 255.255.255.0 Repository Conversion Utility: Operation: Copy Existing Tables: 494 z/OS: WebSphere Business Process Management V6.2 Production Topologies 4 Deploy the AlphaBlox libraries to BusinessSpaceCluster Perform the following steps to deploy AlphaBlox libraries to BusinessSpaceCluster: 1. From the /opt/ibm/WebSphere/MonServer/ABX/bin directory, run the following command: ./DeployWebSphereLibraries.sh -conntype SOAP -username wps -password passw0rd 2. Install libraries to a cluster, by selecting the following: a. Select option 1 to install libraries. b. Select option 1 to install to a cluster. c. Select the BusinessSpaceCluster, because the Business Space Widgets will make use of the libraries. d. Verify the cluster members that should have the libraries deployed. e. Verify that the libraries were successfully installed. f. Exit the deploy script by entering 5 twice. Checking the successful deployment of AlphaBlox libraries Check the successful deployment of AlphaBlox libraries by performing the following steps: 1. After restarting the servers, log in to monitor deployment manager admin console. 2. Go to Environment → Shared Libraries. There should be a list of Alphablox shared libraries as shown in Figure 8-8 on page 496. Chapter 8. Monitoring the Business Process Management environment 495 Figure 8-8 Shared libraries deployment validation. If the Alphablox libraries deployment failed or after restarting servers you are not able to log in to Alphablox admin page, perform the deployment manually by copying all JAR files in /AlphabloxInstallationFolder/lib to the following folder in /MonitorInstallationFolder/lib/ext. Deploy the AlphaBlox applications Perform the following steps to deploy the AlphaBlox applications: 1. Start the WebSphere Business Monitor deployment manager and clusters to finalize the installation of IBM Alphablox. To finalize the IBM Alphablox installation, two applications should be installed on the Monitor Dashboard cluster: – AlphabloxPlatform.ear – ApplicationStudio.ear 2. Open the WebSphere Business Monitor Integrated Solutions Console. Navigate to Applications → Enterprise Applications and click Install. 3. Use the remote file system path setting to browse through the network to locate the AlphabloxPlatform.ear file. Browse to /opt/ibm/WebSphere/MonServer/ABX/installableApps/AlphabloxPlatform.ear. Accept defaults and click Next. 496 z/OS: WebSphere Business Process Management V6.2 Production Topologies 4. Accept defaults in the “Step 1: Select installation options” window. Click Next. 5. In the “Step 2: Map modules to servers” window, click Select all applications, select the BusinessSpaceCluster cluster and click Apply. Click Next. 6. In the “Step 3: Map Virtual hosts to web modules” window, keep defaults and click Next. 7. Click Finish. When you see the a message indicating success, click Save. Click OK. Repeat these steps to install the AlphaBloxStudio.ear application. Configure the AlphaBlox applications After installing Alphablox, you have to configure the applications to start up last. To do this you will have to change the starting weight to a high number. 1000 is a recommended start weight. Then, map roles to users for the applications. To perform both steps, perform the following steps: 1. From the Integrated Solutions Console, expand Applications → Enterprise Applications → AlphaBloxPlatform. Click StartUp Behavior (Figure 8-9). Figure 8-9 Startup behavior Chapter 8. Monitoring the Business Process Management environment 497 2. Change Startup Order from 1 to 1000 (Figure 8-10). Click OK. Figure 8-10 General Properties panel for Startup behavior 3. Click Save then OK. 4. Click the Security role to user/group mapping link. 5. For each role, select the role, and Look up users. Select the users to map to the selected role as shown in Figure 8-11. Figure 8-11 Select role to user/group mapping page 6. Click OK. 7. Repeat the steps for the AlphaBloxStudio application. 498 z/OS: WebSphere Business Process Management V6.2 Production Topologies Post-installation configuration For WebSphere vertical clusters, you need to perform the following post-installation configuration to configure properly the server-specific JVM parameter to identify the Java Management Extensions (JMX) communication port that Alphablox should use and set the server log name. Note: A vertical cluster has cluster members on the same node, or physical machine. A horizontal cluster has cluster members on multiple nodes across many machines in a cell. You can configure either type of cluster, or have a combination of vertical and horizontal clusters. From the Integrated Solutions Console. Expand Servers → Application Servers, for each vertical node in the cluster, perform the following steps: 1. Click the server's name (for example, server1). 2. In the Server Infrastructure section, select Java and Process Management → Process Definition. 3. In the Additional Properties section, select Java Virtual Machine. 4. In the Generic JVM arguments text box, enter the following arguments, leaving a space between the two: -Dabx.ws.admin.port.override=portNumber -Dabx.cluster.log.file.suffix=serverName The port number (portNumber) is usually generated when the server instance is created in the WebSphere server. To determine the port value, select the server's name under Servers −> Application Servers and click Ports. The value to use in portNumber above is the port number for the SOAP_CONNECTOR_ADDRESS port name. The server name (serverName) is the server name displayed under Servers → Application Servers. 5. Save your changes to the master configuration, and restart the servers in the cluster. After properly installing DB2 Alphablox, the WebSphere Business Monitor data sources must be using the DB2 Alphablox administration page. Complete the following steps to create the required data sources: 1. Open the DB2 Alphablox Admin Console. In our environment the URL for the DB2 Alphablox Admin Console was http://clustrm:9084/AlphabloxAdmin. 2. Go to the Administration tab and click Data Sources. Chapter 8. Monitoring the Business Process Management environment 499 3. Click Create and perform the following steps: a. Enter MONITOR in the Data Source Name text box. b. Select Application Server Data Source from the Adapter list. c. Enter jdbc/wbm/MonitorDatabase in the Application Server Data Source Name text box. d. Enter the database user name in the Default Username text box. e. Enter the database password in the Default Password text box. 4. Click Save. 5. Click Create and perform the following steps: a. Enter MONITOR_CUBE in the Data Source Name text box. b. Select Alphablox Cube Server Adapter from the Adapter list. 6. Click Save. 7. Restart the servers. After creating monitor data sources, you should configure monitor themes for alphablox. For more information refer to the following link in monitor information center http://publib.boulder.ibm.com/infocenter/dmndhelp/v6r2mx/topic/com.ibm.btools. help.monitor.install.doc/install/abx_theme_manual.html Note: You must restart the servers for the created data sources to be reflected on other cluster members. 8.8.5 Deploying Business Space Perform the following steps to deploy Business Space: 1. Using the Integrated Solutions Console, expand Servers → Clusters → BusinessSpaceCluster → Business Integration → Business Space Configuration. 2. Select the Install Business Space service check box in the panel shown in Figure 8-12 on page 501. 500 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 8-12 Configuration for Business Space service properties 3. Enter the values in Table 8-8. Table 8-8 Values for business space service properties Name Value Database Schema name COMMONDB Create Business Space data source using Monitor_Database 4. Click OK. Review the message that displays, and click Save. 5. Create the tables for the Business Space service. Use the script in the path /opt/IBM/WebSphere/MonServer/dbscripts/BusinessSpace/DB2zOSV8 6. Issue the commands as db2inst1. >db2 connect to MONITOR >db2 -tf createTable_BusinessSpace.sql >db2 reset 7. Start Business Space Manager by performing the following steps: a. Expand Applications → Enterprise Applications. b. Check Business Space Manager. c. Click Start. Chapter 8. Monitoring the Business Process Management environment 501 8.8.6 Configure Business Space for dashboard widgets After installing Business Space, you should configure monitor widget XML files to enable the dashboards views in Business Space. By default, the dashboard widgets are neither registered nor enabled in Business Space. Using the administrator user, you can add, remove, or update widget XML files. You can edit those XML files, make the required changes, and copy them to BusinessSpace/registry directory as indicated below. The Business Space registration files are located in the <WebSphere_Process_Server_installation>\BusinessSpace\registryData path in all nodes of Business Space cluster. Complete the following steps to configure WebSphere Business Monitor dashboards on Business Space for each cluster member: 1. Create the BusinessSpace/registryData directory on all the nodes of the Business Space cluster in the <WebSphere_Process_Server_installation>\profiles\<Monitor_profile _name>\BusinessSpace\registryData path. 2. To register WebSphere Business Monitor widgets, create a copy of the monitorWidgets.xml file and edit this file. Locate the element <tns:Widget> for all the widgets you would like to administer. Add the action attribute to the <tns:Widget> element as shown below: – – – – <tns:Widget <tns:Widget <tns:Widget <tns:Widget action=“addUpdate”> (This is the default) action=“add”> (Adds a new widget to the registry) action=“update”> (Updates the widget to the registry) action= “delete”> (Deletes the widget from the registry) An example of an edited monitorWidgets.xml file is shown in Example 8-9. Example 8-9 monitorWidgets.xml <!-- START NON-TRANSLATABLE --> <tns:Widget action=”update”> <tns:id>{com.ibm.wbimonitor}instances</tns:id> <tns:version>1.0.0.0</tns:version> <tns:name>Instances</tns:name> <tns:type>{com.ibm.bspace}mWidget</tns:type> <tns:description>IBM WebSphere Business Monitor</tns:description> <tns:tooltip>Instances</tns:tooltip> <tns:categoryId>{com.ibm.wbimonitor}monitor</tns:categoryId> <tns:widgetEndpointId>{com.ibm.wbimonitor}monitorWidgetRootId</tns:w idgetEndpointId> 502 z/OS: WebSphere Business Process Management V6.2 Production Topologies <tns:viewUrl>_Instances/jsp/html/InstancesView.jsp</tns:viewUrl> <tns:editUrl>_Instances/jsp/html/InstancesEdit.jsp</tns:editUrl> <tns:helpUrl>dash/help_instances.html</tns:helpUrl> <tns:iconUrl>img/Instances.gif</tns:iconUrl> <!-- <tns:previewUrl>TBD</tns:previewUrl> --> <tns:owner>IBM</tns:owner> <tns:email>TBD</tns:email> <tns:serviceEndpointRef> <tns:name>serviceUrlRoot</tns:name> <tns:refId>{com.ibm.wbimonitor}monitorServiceRootId</tns:refId> <tns:refVersion>1.0.0.0</tns:refVersion> </tns:serviceEndpointRef> <tns:localeInfo> <!-- END NON-TRANSLATABLE --> <tns:locale>en_US</tns:locale> <tns:name>Instances</tns:name> <tns:description>IBM WebSphere Business Monitor</tns:description> <tns:tooltip>Instances</tns:tooltip> <!-- START NON-TRANSLATABLE --> </tns:localeInfo> </tns:Widget> <!-- END NON-TRANSLATABLE --> 3. Save the monitorWidgets.xml file. 4. Copy the monitorWidgets.xml file to the <WebSphere_Monitor_Server_installation>\profiles\<Monitor_profile _name>\BusinessSpace\registryData directory, on all the nodes where Business Space is installed. 5. Restart the Business Space cluster. 6. To enable WebSphere Business Monitor widgets, create a copy of the following endpoint registration files: – monitorABXEndpoints.xml – monitorEndpoints.xml 7. Edit the two files as indicated below in bold. Example 8-10 on page 504 shows monitorEndpoints.xml. Example 8-11 on page 505 shows monitorABXEndpoints.xml. Chapter 8. Monitoring the Business Process Management environment 503 Example 8-10 monitorEndpoints.xml <?xml version="1.0" encoding="UTF-8"?> <!-- START NON-TRANSLATABLE --> <tns:BusinessSpaceRegistry xmlns:tns="http://com.ibm.bspace/BusinessSpaceRegistry" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://com.ibm.bspace/BusinessSpaceRegistry BusinessSpaceRegistry.xsd "> <tns:Endpoint action="addUpdate"> <tns:id>{com.ibm.wbimonitor}monitorServiceRootId</tns:id> <tns:version>1.0.0.0</tns:version> <tns:url>http://rest_services_hostname:port_number/rest/</tns:url> <tns:description>Location of backing services for Monitor widgets</tns:description> </tns:Endpoint> </tns:BusinessSpaceRegistry> <!-- END NON-TRANSLATABLE --> 504 z/OS: WebSphere Business Process Management V6.2 Production Topologies Example 8-11 monitorABXEndpoints.xml <?xml version="1.0" encoding="UTF-8"?> <!-- START NON-TRANSLATABLE --> <tns:BusinessSpaceRegistry xmlns:tns="http://com.ibm.bspace/BusinessSpaceRegistry" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://com.ibm.bspace/BusinessSpaceRegistry BusinessSpaceRegistry.xsd "> <tns:Endpoint action="addUpdate"> <tns:id>{com.ibm.wbimonitor}monitorABXServiceRootId</tns:id> <tns:version>1.0.0.0</tns:version> <tns:url>http://rest_services_hostname:port_number/rest/</tns:url> <tns:description>Location of backing services for Monitor widgets</tns:description> </tns:Endpoint> </tns:BusinessSpaceRegistry> <!-- END NON-TRANSLATABLE --> 8. Save the files. 9. Copy the monitorEndpoints.xml and monitorABXEndpoints.xml files to the <WebSphere_Process_Server_installation>\profiles\<Monitor_profile _name>\BusinessSpace\registryData directory, on all the nodes where Business Space is installed. 10.Restart the servers. 8.9 Monitoring the BPM reference topology for z/OS using WebSphere Business Monitor In this section, we introduce the steps to use the installed Monitor server cluster to monitor the business processes in the WebSphere Process Server topology on z/OS. 8.9.1 Configuring security for linking WebSphere Business Monitor to WebSphere Process Server for z/OS Because WebSphere Business Monitor and WebSphere Process Server for z/OS are configured in separate WebSphere Application Server Network Chapter 8. Monitoring the Business Process Management environment 505 Deployment cells, we must configure security in both cells so that they can talk to one another. We discuss these steps below: See 8.9.2, “Securing WebSphere Business Monitor” on page 506 for configuring WebSphere Business Monitor in the Linux for System z cell. See 8.9.3, “Securing the BPM reference topology for z/OS” on page 506 for configuring WebSphere Process Server for z/OS in the z/OS cell. 8.9.2 Securing WebSphere Business Monitor When you enable security for WebSphere Business Monitor, you are enabling administrative and application security settings. WebSphere Business Monitor uses many of the security mechanisms provided by the prerequisite products, including WebSphere Application Server. Configure access to the monitor model resources using Monitor Data Security in the Integrated Solutions Console. For WebSphere Application Server instances that run the WebSphere Business Monitor server, configure them to use the federated repository or custom user registry. They cannot use a local operating system, stand-alone LDAP registry, or stand-alone custom registry directly. For more information about enabling security of WebSphere Business Monitor refer to 8.11.1 Enabling federated repository security using LDAP. Note: In this book we are using one LDAP server to authenticate all cells. WebSphere Business Monitor v6.2 only supports federated repository and user customer registry. So for the WebSphere Business Monitor cell we are configuring LDAP under the federated repository option. 8.9.3 Securing the BPM reference topology for z/OS We want both the Linux for System z cell and the z/OS cell to use the same user registry. Both cells can run with a LDAP-based user registry. In the z/OS case, we can configure the cell to use a Standalone LDAP server. See 5.3.4, “Configure LDAP security” on page 280 for details on how to set up the z/OS cell to use a Standalone LDAP repository. 8.9.4 Security considerations Securing the environment should be the first point we should consider while integrating WebSphere Business Monitor with WebSphere Process Server on z/OS. In the current installation, we have both WebSphere Business Monitor and 506 z/OS: WebSphere Business Process Management V6.2 Production Topologies WebSphere Process Server security enabled using LDAP. As mentioned earlier, WebSphere Business Monitor v6.2 supports only federated repository or custom user registry and does not support standalone LDAP. We must therefore configure LDAP for WebSphere Business Monitor as the repository for the federated registry. Refer to “Enabling federated repository security using LDAP” on page 507 for details on setting up federated security. In addition to LDAP configuration for WebSphere Business Monitor and WebSphere Process Server, and to secure the communication between the two cells, we must exchange both SSL certificates and LTPA keys. Refer to “LTPA keys exchange” on page 513 for details on exchanging LTPA keys. Refer to “SSL configuration” on page 516 for details on exchanging SSL certificates. Note: It is strongly recommended to enable security for the WebSphere Business Monitor server environment just after installing the deployment manager and creating the clusters. Enabling federated repository security using LDAP It is strongly recommended that both WebSphere Business Monitor cell and WebSphere Process Server cell have to authenticate on the same LDAP server. This configuration will keep the environment security more consistent and reliable. The following steps will enable you to configure a federated repository using LDAP. 1. In the WebSphere Business Monitor deployment manager administrative console, navigate to Security → Secure administration, applications and infrastructure. 2. Click Security Configuration Wizard. 3. In the “Step1: Specify extent of protection” window, ensure the check box for Enable application security is selected and Click Next. 4. In the “Step 2: Select user repository” window, ensure the radio button for Federated repositories is selected and Click Next 5. In the “Step 3: Configure user repository” window, enter the primary administrative user name and password – Primary administrative user name: mdadmin – Password: mdadmin – Confirm password: mdadmin Click Next. 6. In the “Step 4: Summary” window, review the summary information, and click Finish. Chapter 8. Monitoring the Business Process Management environment 507 7. Select Federated repositories from the drop-down list for the Available realm definitions, as shown in Figure 8-13. Figure 8-13 Secure administration, applications and infrastructure page 8. Click Configure. 9. In the “Configuration” window, enter the following under the General Properties category, as shown in Figure 8-14 on page 509: – Realm name: <fully qualified LDAP server host name>: <LDAP Port> Example: wpsplex.itso.ibm.com:2389 – Primary administrative user name: mdadmin 508 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 8-14 Federated repositories page Note: It is important to define IP/host name of the LDAP server in /etc/hosts file to avoid any network problems. 10.Click Add Base entry to Realm. This opens the Repository reference panel. 11.Click Add Repository. 12.In the Repository reference→ New panel, enter the following information, as shown in Figure 8-15 on page 510: – Repository identifier: Ldap-conf (Any meaningful identifier) – LDAP Server • • • Directory Type: Custom Primary host name: wpsplex.itso.ibm.com Port: 2389 – Security • • • • Bind distinguished name: uid=ldapadm,o=ibm Bind password: ldapadm Login properties: uid Certificate mapping: EXACT_DN Click OK. Chapter 8. Monitoring the Business Process Management environment 509 Figure 8-15 New repository configuration page 13.In the repository configuration panel, enter the following as shown in Figure 8-16: – Repository: Select LDAP-idsldap from the drop-down list – Distinguished names of a base entry that uniquely identifies this set of entries in the realm: o=ibm – Distinguished name of a base entry in this repository: cn=People,o=ibm Click OK. Figure 8-16 Repository configuration page 510 z/OS: WebSphere Business Process Management V6.2 Production Topologies 14.Select the o=defaultWIMFIleBasedRealm check box if it exists (it exists if the default security is enabled), and click Remove. 15.Configure supported entity types in a federated repository configuration as shown in Figure 8-17. a. In the “Federated Repositories” panel, click Supported entity types under the Additional Properties section b. In the “Supported entity types” panel, update the Base entry for default parent values for Group, OrgContainer and PersonAccount with o=IBM and accept the defaults for the Relative Distinguished Name properties. c. Save to the master configuration with synchronization with other nodes. Figure 8-17 Supported entity types page 16.Configure group attribute definition a. In the “Federated repositories” panel, click the Repository identifier link, (Ldap-idsldap). b. In the next panel, click Group attribute definition under the Additional Properties section. c. In the “Group attribute definition” panel, enter the following information: • Name of group membership attribute: LDAP-AllGroups • For the Scope of group membership attribute, select All- Contains all direct, nested and dynamic members. Click Apply. d. In the “Group attribute definition” panel, click the Member attributes link under the Additional properties section, as shown in Figure 8-18 on page 512. Chapter 8. Monitoring the Business Process Management environment 511 Figure 8-18 Group attribute definition page e. In the next panel, click New to create a new member attribute. f. In the following panel, enter the following information: • Name of member attribute: uniqueMember • Object class: groupOfUniqueNames • For the Scope, select All- Contains all direct, nested and dynamic members. Click OK. g. Save to the master configuration with synchronization with other nodes. 17.Configure Single Sign-on as shown in Figure 8-19 on page 513. a. Go to the “Secure administration, applications and infrastructure” panel, expand Web Security under the Authentication category. b. Click single sign-on (SSO). c. In the single sign-on (SSO) panel, perform the following steps: i. i. i. i. i. Select the Enabled check box. Ensure the Requires SSL check box is cleared (Optional) Enter ibm.com as the Domain name. Select the Interoperability Mode check box. Clear the Web bound security attribute propagation check box d. Click OK. 512 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 8-19 Single sign on page 18.In the “Secure administration, applications and infrastructure” panel, make sure that Enable administrative security and Enable application security are selected. Make sure that Java 2 security is not selected. 19.Save configuration with synchronization with other nodes. 20.Restart all servers, node agents and deployment manager to enable security for all cluster members. LTPA keys exchange Perform the following steps to export LTPA keys from the WebSphere Business Monitor server and import them into WebSphere Process Server server. 1. Login to WebSphere Business Monitor deployment manager admin console. 2. Go to Security→ Secure administration, applications, and infrastructure→ Authentication mechanisms and expiration, as shown in Figure 8-20 on page 514. Chapter 8. Monitoring the Business Process Management environment 513 Figure 8-20 Secure administration, applications, and infrastructure page 3. Perform the following steps in the Cross-cell single sign-on area, as shown in Figure 8-21: a. Type a password. b. Confirm the same password. c. Type the path and file name required to store the key. The key file name should have the extension .keys. d. Click Export Keys. e. Click OK. Figure 8-21 Exporting LTPA keys Note: The used user permissions should allow to save and write files in the selected path to store the key files. 514 z/OS: WebSphere Business Process Management V6.2 Production Topologies 4. Copy the exported key file to be imported to WebSphere Process Server server. 5. Place the exported key file in the /wasv61config/mdcell/mddmnode/DeploymentManager/keys/ltpa.keys directory on the z/OS system: 6. Perform the following steps on the Integrated Solutions Console for the z/OS system (Figure 8-22 on page 516): a. Navigate to Security → Secure administration, applications, and infrastructure. b. Click Authentication mechanisms and expiration. c. In the Cross-cell single sign-on section enter the following information: • • • Password: mdadmin Confirm password: mdadmin Fully qualified key file name: /wasv61config/mdcell/mddmnode/DeploymentManager/keys/ltpa. keys d. Click Import Keys. Chapter 8. Monitoring the Business Process Management environment 515 Figure 8-22 Import LTPA keys e. Click OK, and Save. f. Stop and restart all servers including deployment manager, node agents, and servers. SSL configuration To secure the communication channel between WebSphere Business Monitor and WebSphere Process Server, you have to exchange SSL certificates between the two cells. You have to export the SSL certificate files from WebSphere Business Monitor server and import them into WebSphere Process Server. To exchange the SSL certificates, perform the following steps: 516 z/OS: WebSphere Business Process Management V6.2 Production Topologies 1. Log in to WebSphere Business Monitor deployment manager Integrated Solutions Console. 2. Go to Security→ SSL certificate and key management→ Key stores and certificates, as shown in Figure 8-23. Figure 8-23 SSL certificate and key management page 3. Click CellDefaultTrustStore, as shown in Figure 8-25 on page 518. Figure 8-24 Key stores and certificated page Chapter 8. Monitoring the Business Process Management environment 517 4. Click Signer certificates under Additional properties, as shown in Figure 8-25. Figure 8-25 CellDefaultTrustStore page 5. Select default in the Alias column and click Extract, as shown in Figure 8-26 on page 519. 518 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 8-26 Singer certificates page 6. Select Binary DER data as the data type. Type the path and file name as shown in Figure 8-27. The file should have extension .der. Click OK to save the file in the required location, as shown in Figure 8-27. Figure 8-27 Certificate extract page 7. Repeat steps 5 and 6 for both default 1 and default 2 with different file names. Chapter 8. Monitoring the Business Process Management environment 519 8. Copy the three extracted files to import them into WebSphere Process Server side. 9. Place the exported certificates in the following directories on the z/OS system: /wasv61config/mdcell/mddmnode/DeploymentManager/keys/default /wasv61config/mdcell/mddmnode/DeploymentManager/keys/default1 /wasv61config/mdcell/mddmnode/DeploymentManager/keys/default2 10.Perform the following steps on the Integrated Solutions Console for the z/OS system (Figure 8-28): a. Navigate to Security → SSL certificate and key management. b. Click Key stores and certificates. c. Select CellDefaultTrustStore. d. Click Signer certificates. e. For each of the three imported certificates, click Add and specify the following values: i. Alias: default, default1, default2 ii. File names: /wasv61config/mdcell/mddmnode/DeploymentManager/keys/default /wasv61config/mdcell/mddmnode/DeploymentManager/keys/default1 /wasv61config/mdcell/mddmnode/DeploymentManager/keys/default2 i. Data type: Binary DER data f. For each certificate, click OK and Save. Figure 8-28 Imported certificate 520 z/OS: WebSphere Business Process Management V6.2 Production Topologies 11.After you have imported the certificates, you see a window similar to Figure 8-29. Figure 8-29 Imported certificates For more information about enabling security of WebSphere Business Monitor and WebSphere Process Server refer to the End to end security lab available at the following Web page: http://publib.boulder.ibm.com/infocenter/ieduasst/v1r1m0/index.jsp?topi c=/com.ibm.iea.wpi_v6/wbmonitor/6.1/Dashboards.html 8.9.5 Configuring WebSphere Business Monitor to use a remote CEI server In this section, you will enable WebSphere Business Monitor server to receive events from a remote CEI. In the current installation the remote CEI is a WebSphere Process Server V6.2 server. Note: Security should be enabled or disabled on both the WebSphere Business Monitor cell and the WebSphere Process Server cell. Enabling security on only one cell in a cross cell configuration is not supported. Chapter 8. Monitoring the Business Process Management environment 521 In this installation we create an event transmission mechanism using the queue-based option. If you are using queue bypass option, you must install the crosscell files and queue bypass files, create the remote service integration bus, and create the link between the local and remote buses, configure and setup queue bypass, and update the node-metadata.properties Note: The queue bypass option is available only with WebSphere Process Server v6.2. To configure WebSphere Business Monitor to use remote CEI, perform the following steps: 1. From the WBM_root/scripts.wbm/crosscell folder of the local WebSphere Business Monitor server installation, copy the mon61to61CrossCell.zip file to the WPS_root/plugins folder of the remote CEI server and extract the contents (jar files). 2. From the WPS_root/bin folder on the remote CEI server, run the appropriate command to configure WebSphere Process Server to recognize the .jar from file: osgiCfgInit.sh 3. From the WBM_root\scripts.wbm\crosscell folder of the local WebSphere Business Monitor server V6.2 installation, run the service integration bus crosscell configuration utility as the following: a. Run the command configRemoteMonitorBus.sh as shown in Figure 8-30. The command will guide you through interactive steps. Figure 8-30 configRemoteMonitorBus command run Step 1 b. Determine if the security is enabled or not for this configuration. 522 z/OS: WebSphere Business Process Management V6.2 Production Topologies Note: If security is enabled, provide the requested credentials using the following information LOCAL WAS USERID and LOCAL WAS PASSWORD: An administrator or configurator user ID and password for WebSphere Business Monitor server. REMOTE WAS USERID and REMOTE WAS PASSWORD: An administrator or configurator user ID and password for the remote CEI server. REMOTE WAS BUS USERID and REMOTE WAS BUS PASSWORD: A user ID and password to be used for accessing the remote Monitor bus at remote CEI server. LINK USERID: A user ID that is valid in both cells that is used to authenticate the SIB link. LINK USERID REMOTE PASSWORD: The password for the LINK USERID in the remote CEI cell. LINK USERID LOCAL PASSWORD: The password for the LINK USERID in the local cell c. Type the full host name for WebSphere Business Monitor server. d. Type the soap port number for WebSphere Business Monitor server. the default value is 8880. e. Type the full host name for the remote CEI (WPS server). f. Type the soap port number for the remote CEI. g. Accept the default settings while configuring messaging engine. The configuration should be created and a successful message is displayed as shown in Figure 8-31 on page 524. Chapter 8. Monitoring the Business Process Management environment 523 Figure 8-31 configRemoteMonitorBus command Finish Message 4. Restart both the local WebSphere Business Monitor server and the remote CEI server. 5. Verify that the remote service integration bus exists and that the link between the local and remote buses is created successfully. a. From the administrative console on the remote WebSphere Process Server or the local WebSphere Business Monitor server, click Service Integration→ Buses. b. Click the MONITOR.cell_name.bus bus that you are verifying. c. Under Topology, click Messaging Engines. One messaging engine should be defined. The Status field displays a green arrow if the messaging engine is active. d. Click the messaging engine, and click Additional Properties → Service integration bus links. If you are connecting the remote cell to a single monitor installation and a monitor installation to a single remote cell, one link should be defined. You can, however, have more than one link. The Status field displays a green arrow if the link is active. 8.9.6 Configuring bus destination roles The last step in the configuration to enable the WebSphere Business Monitor server to receive events from a remote CEI is to configure bus destination roles. This is required so that users can send and receive events using the specified bus. 524 z/OS: WebSphere Business Process Management V6.2 Production Topologies Adding the administrative user to the sender role We must add the administrative user to the sender role for the foreign bus. The parameters are as follows: Monitor bus name: BusName Foreign bus name: ForeignBusName Administrative user name: username We created sample job, MDCAUFBR, in dataset WASCFG.MDCELL.WBM.CNTL to create the role for the foreign bus. The values we specified in the jobs are unique for our environment. See Example 8-12 for details. Example 8-12 Configure role for foreign bus //MDCAUFBR JOB (0),CLASS=A,MSGCLASS=A,REGION=0M, // USER=MDADMIN,PASSWORD=MDADMIN, // NOTIFY=GATES /*JOBPARM SYSAFF=SC42 //***********************************************************/ //* Configure foreign bus role for Monitor */ //***********************************************************/ //INSTO EXEC PGM=IKJEFT01,REGION=0M,TIME=1440 //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BPXBATCH SH + cd /wasv61config/mdcell/mddmnode/DeploymentManager/bin; + ./wsadmin.sh + -addUserToForeignBusRole + -role Sender + -bus MONITOR.mdcell.bus + -foreignBus MONITOR.lnxsu1.Cell01.bus + -user MDADMIN + 1> /tmp/mddmnodeUfbr_0A11.out + 2> /tmp/mddmnodeUfbr_0A11.err; /* //***********************************************************/ //* STEPS TO COPY THE OUTPUT THE JOB LOG */ //***********************************************************/ //CPOUT EXEC PGM=IKJEFT01,REGION=0M //SYSEXEC DD DISP=SHR,DSN=BBO6142.MDDMNODE.SBBOEXEC //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BBOHFSWR '/tmp/mddmnodeUfbr_0A11.out' BBOHFSWR '/tmp/mddmnodeUfbr_0A11.err' /* // Chapter 8. Monitoring the Business Process Management environment 525 Adding the administrative user to the sender role We must add the administrative user to the sender role for the foreign bus. The parameters are as follows: Monitor bus name: BusName Foreign bus name: ForeignBusName Queue destination on the Monitor host: destinationName (It starts with wbm_modelname) Administrative user name: username We created sample job, MDCAUDBR, in dataset WASCFG.MDCELL.WBM.CNTL to create the role for the remote destination. The values we specified in the jobs are unique for our environment. See Example 8-13 for details. Example 8-13 Configure role for remote destination //MDCAUDBR JOB (0),CLASS=A,MSGCLASS=A,REGION=0M, // USER=MDADMIN,PASSWORD=MDADMIN, // NOTIFY=GATES /*JOBPARM SYSAFF=SC42 //***********************************************************/ //* Configure destination role for Monitor */ //***********************************************************/ //INSTO EXEC PGM=IKJEFT01,REGION=0M,TIME=1440 //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BPXBATCH SH + cd /wasv61config/mdcell/mddmnode/DeploymentManager/bin; + ./wsadmin.sh + -addUserToDestinationRole + -type foreignDestination + -role Sender + -bus MONITOR.mdcell.bus + -foreignBus MONITOR.lnxsu1.Cell01.bus + -destinationName wbm_orderProcessingMM_20071009193454_Q+ Destination@MonMECluster.000-MONITOR.lnxsu1Cell01.Bus + 1> /tmp/mddmnodeUfbr_0A11.out + 2> /tmp/mddmnodeUfbr_0A11.err; /* //***********************************************************/ //* STEPS TO COPY THE OUTPUT THE JOB LOG */ //***********************************************************/ //CPOUT EXEC PGM=IKJEFT01,REGION=0M //SYSEXEC DD DISP=SHR,DSN=BBO6142.MDDMNODE.SBBOEXEC 526 z/OS: WebSphere Business Process Management V6.2 Production Topologies //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BBOHFSWR '/tmp/mddmnodeUfbr_0A11.out' BBOHFSWR '/tmp/mddmnodeUfbr_0A11.err' /* // For additional information for completing the installation of a monitor model in a secured environment refer to the following Web page: http://publib.boulder.ibm.com/infocenter/dmndhelp/v6r2mx/topic/com.ibm. btools.monitor.admin.doc/admin/model_install_secure.html 8.10 Additional Monitor SI Bus configuration for the WebSphere Process Server for z/OS cell We found additional configuration was required for the WebSphere Process Server for z/OS cell to enable the remote CEI bus. See the following for details. 8.10.1 Monitor SI Bus database configuration After configuring the Monitor SI bus we encountered errors in the server adjunct region that indicated the datastore for the Monitor messaging engine could not be connected to. We discovered a problem in the way the datastore was created by the SIB configuration assistant that resulted in the datastore for the Monitor SI Bus pointing at a database on a DB2 UDB image. To fix this problem, we followed the steps detailed in the following sections: Create a new database for the Monitor SIB To create the tables for the Monitor Service Integration Bus, we copied the DDL we had used for one of the other buses and changed the schema to MD01M (where the M stands for Monitor). You could also run the sibDDLGenerator.sh to generate new DDL. We created sample job MDDDLSI (Example 8-14 on page 528) to execute this DDL. Chapter 8. Monitoring the Business Process Management environment 527 Example 8-14 Sample job MDDDLSI to execute the Monitor bus DDL //MDDDLSI JOB (0),'MDADMIN',CLASS=A,REGION=0M,NOTIFY=GATES, // USER=SENIOKJ,PASSWORD=SENIOKJ /*JOBPARM S=SC42 //***********************************************************/ //* Run this job using a userid with DB2 SYSADM authority */ //* or grant DBADM to the <sqlid> on SET CURRENT SQLID. */ //* Set the SYSAFF to the LPAR for this node. */ //* Change <xx> to your two character cell prefix. */ //* For example, MD. (Use upper case.) */ //* The DB2 system administrator should review all the */ //* names and bufferpools used. */ //* Change all the dataset names and plan name to match */ //* those in use with your DB2 system. */ //* Set the SYSIN to the dataset holding the DDL. */ //***********************************************************/ //JOBLIB DD DISP=SHR,DSN=DB9F9.SDSNEXIT // DD DISP=SHR,DSN=DB9F9.SDSNLOAD //* //* Before running this: //* The userid on the SET CURRENT SQLID statement must //* have DB2 SYSADM authority and EXECUTE on PLAN DSNTEP2 //* //* Create database and tables for Fabric SIB //* //CREATE EXEC PGM=IKJEFT01,DYNAMNBR=20 //SYSTSPRT DD SYSOUT=* //SYSUDUMP DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SYSTSIN DD * DSN SYSTEM(D9FG) RUN PROGRAM(DSNTEP2) PLAN(DSNTEP91) LIB('DB9FU.RUNLIB.LOAD') //SYSIN DD * SET CURRENT SQLID = 'MD01M' ; // DD DISP=SHR,DSN=WASCFG.MDCELL.WBM.SQL(MD01M) Note: There are GRANT statements for the SIB tables at the end of the MD01M DDL member so it is not necessary to run a separate GRANT job for the SIB tables. 528 z/OS: WebSphere Business Process Management V6.2 Production Topologies Create a datasource that points to the new database Perform the following steps to create a datasource that points to the new database: 1. In the Integrated Solutions Console, navigate to Resources → JDBC. 2. Click Data sources. 3. Set scope to Cluster=MDSR1. 4. Click New. 5. Enter the following in the datasource information panel (Figure 8-32): a. Name: Monitor BUS ME Datasource b. JNDI name: jndi/monitor/MDSR1.Monitor.mdcell c. Component-managed authentication alias and XA recovery authentication alias: SCASYSME00_Auth_Alias Figure 8-32 Configure new datasource for Monitor SI Bus 6. Click Next. 7. Select the Select an existing JDBC provider check box. Chapter 8. Monitoring the Business Process Management environment 529 8. Select DB2 Universal JDBC Driver Provider (Figure 8-33). Figure 8-33 Select JDBC provider 9. Click Next. 10.Enter the following information in the database panel (Figure 8-34 on page 531): – – – – Database name: DB9F Driver type: 2 Server name: wtsc42.itso.ibm.com Port number: 37893 Clear the Use this datasource in container managed persistence (CMP) check box. 530 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 8-34 Database properties 11.Click Next. 12.Click Finish, and Save. Configure the Monitor SI Bus ME to use the new datasource Perform the following steps to configure the Monitor SI Bus ME to use the new datasource: 1. In the Integrated Solutions Console, navigate to Service Integration → Buses. 2. Select MONITOR.mdcell.Bus (Figure 8-35 on page 532). Chapter 8. Monitoring the Business Process Management environment 531 Figure 8-35 Monitor messaging engine 3. Select Messaging engines. 4. Select MDSR1.000-MONITOR.mdcell.Bus (Figure 8-36). Figure 8-36 Monitor message bus 5. Click Message store (Figure 8-37 on page 533). 532 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 8-37 Monitor message bus properties 6. Change the values on this panel to reflect our new datasource and database (Figure 8-38 on page 534). • • • Data source JNDI name: jdbc/monitor/MDSR1.Monitor.mdcell Schema name: MD01M Authentication alias: SCAME00_Auth_Alias Clear the Create tables check box. Chapter 8. Monitoring the Business Process Management environment 533 Figure 8-38 Monitor SI Bus datastore 7. Click OK, and Save. 8. Stop and restart the node servers and adjuncts For us, following these changes, the Monitor SI Bus initialized correctly. 8.10.2 Monitor Foreign Bus configuration After correctly configuring the Monitor SI Bus, we received errors attempting to resolve the Foreign Bus definitions. We discovered an error in the way the host name of the remote system was defined for the Foreign Bus. We made the following changes to address this problem. 1. In the Integrated Solutions Console, navigate to Service Integration → Buses. 2. Select MONITOR.mdcell.Bus. 3. Select Foreign buses. 4. Select MONITOR.lnxsu1Cell01.Bus (Figure 8-39 on page 535). 534 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 8-39 Monitor SI Bus Foreign bus 5. Select Service Integration Bus Link (Figure 8-40). Figure 8-40 Foreign Bus properties 6. Change the host name specified in Bootstrap endpoints to the host name / IP address of the remote Monitor cell. In our case, the IP address of our Linux System z server is 9.12.5.41. See Figure 8-41 on page 536. Chapter 8. Monitoring the Business Process Management environment 535 Figure 8-41 Foreign Bus properties 7. Stop and restart the node servers and adjuncts. For us, following these changes, the Monitor SI Bus initialized correctly and the WebSphere Business Monitor in the Linux for System z cell communicated successfully with the WebSphere Process Server for z/OS in the z/OS cell. 536 z/OS: WebSphere Business Process Management V6.2 Production Topologies 8.11 Verifying WebSphere Business Monitor functions with the BPM reference topology on z/OS To check the functionality of the WebSphere Business Monitor installation, it is recommended to install a sample monitor model to check the event consumption and dashboard capabilities. In this section, we will go through the steps of deploying a monitor model and check the consuming of events. 8.11.1 Preparing the business process for monitoring After creating the business process in WebSphere Integration Developer, you should perform the following steps: 1. Choose the required events to be generated and emitted according to the business situation for each business entity. 2. Generate the corresponding monitor model from the created business process. 3. During generation, you can select the required events to be monitored from the list of emitted events. 8.11.2 Preparing the monitor model in the toolkit After generating the monitor model, you can tune and modify the model in monitoring perspective (WebSphere Business Monitor toolkit plugin in WebSphere Integration Developer) to fit the monitoring business requirements. Use this to create new KPIs, timers, triggers, dimensions, and so forth. Upon finishing implementation of the monitor mode, generate the corresponding J2EE applications from the monitor model in WebSphere Integration Developer. There will be three projects created: Monitor model application project. Monitor model logic part project. Monitor model moderator part project. 8.11.3 Monitor model deployment By exporting the ear file from any of above projects, you are exporting the monitor model application that you should deploy on WebSphere Business Monitor Server. Chapter 8. Monitoring the Business Process Management environment 537 Deployment process consists of two parts: Deploying the monitor model application ear. It should be deployed on the WebSphere Business Monitor server over the Model Logic and Model Moderator clusters. Deploying the corresponding business application ear. It should be deployed on the WebSphere Process Server server over the Application Target cluster. The following are the steps to deploy a monitor model using a remote CEI configuration: 1. From WebSphere Business Monitor deployment manager Integrated Solutions Console, click Applications→ Monitor Models. Click Install. 2. In the “Preparing for application installation” panel (Figure 8-42), perform the following steps: a. Type the path of the required monitor model ear file. b. Select the Show me all installation options and parameters check box. c. Click Next. Figure 8-42 Preparing for application installation page 3. In the next panel, click Next. 4. In the next panel, click Continue. A wizard will be displayed. 5. In the “Step 1: Select installations options” window, use the defaults and click Next. 538 z/OS: WebSphere Business Process Management V6.2 Production Topologies 6. In the “Step 2: Map modules to servers” window, assign each monitor model module to the corresponding cluster. The MMLogic module of the monitor model will be assigned to Model Logic cluster and MMModerator module will be assigned to Model Moderator cluster. To perform this step, select the module, select the corresponding cluster, and click Apply, as in Figure 8-43. Click Next. Figure 8-43 Map modules to server step 7. For steps 3 through Step 11, use defaults and click Next. If any warning appears, click Continue. 8. In the “Step 12: Select monitor model CEI option” window, perform the following steps, as shown in Figure 8-44 on page 540: a. In Location i. Select Remote. ii. Type the host name of the remote CEI server. iii. Type the RMI port number of the remote CEI server. b. In the Security area, if security is enabled, type the username and password for the remote CEI server. c. Click Refresh List. d. In Distribution mode area, select Active (monitor model queue-based). e. Click Next. Chapter 8. Monitoring the Business Process Management environment 539 Figure 8-44 Select monitor model CEI step 9. Review the summary and click Finish. For more information about how to deploy a monitor model, refer to the following link in WebSphere Business Monitor Information Center topic. http://publib.boulder.ibm.com/infocenter/dmndhelp/v6r2mx/topic/com.ibm. btools.monitor.admin.doc/admin/admin_deploy.html 540 z/OS: WebSphere Business Process Management V6.2 Production Topologies 8.11.4 Inspecting the monitor model and business process functionality After deploying the business process application on the WebSphere Process Server Application Target cluster, you can start to emit events and check the following: 1. The events are emitted correctly at the WebSphere Process Server server side. 2. The events are transmitted correctly to WebSphere Business Monitor server side. 3. The corresponding deployed monitor model consumed the events correctly. To check that the events are emitted correctly at the WebSphere Process Server side, perform the following steps: 1. Log in to Business Process Choreographer (BPC) Explorer. 2. Click Process Templates. 3. Check on the corresponding BPEL process and click Start Instance. There should be some events are generated and emitted now. You can check the emitted events in CBE Browser. Note: If the SIB link is running, you will not be able to preview the emitted events in the CBE Browser as the events are transferred automatically to WebSphere Business Monitor server side and get deleted from WebSphere Process Server side. If you want to preview the emitted events, stop the SIB link first. To check that the events are transmitted to WebSphere Business Monitor server side correctly, perform the following steps: 1. Stop the corresponding monitor model. 2. Emit events are WPS side using BPC. 3. Preview events using CBE browser. Chapter 8. Monitoring the Business Process Management environment 541 To check the consumption of the events by the corresponding monitor model, continue from the above steps: 4. Start the corresponding monitor model. 5. Check the existence of the events using CBE Browser. If the events no longer exist, then the monitor model already consumed them (this will be valid if the datastore is disabled). If the messaging engine datastore is enabled, you have to check the event consumption either through dashboard views or by inspecting the corresponding monitor model tables in Monitor database. 8.11.5 Monitor dashboards After deploying the monitor model you can start to view your dashboards using Monitor widgets in Business Space. You can access the business space dashboards using the following URL: http://WBM_Dmgr_hostname:9080/BusinessSpace There are eleven dashboard widgets to provide all possible data display options. The following procedure details steps for configuring the instance view widget as an example: 1. Log in to Business Space. 2. Got to Business Space Manager by clicking Welcome→ Manage Business Spaces. 3. Click Create new business space. 4. Enter the business space name as shown in Figure 8-45 on page 543. 542 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 8-45 New business space 5. Click Create new page. 6. Enter the page name as shown in Figure 8-46. Figure 8-46 New business space page 7. Click the newly created business space page. 8. Click Add Widgets as shown in Figure 8-47 on page 544. Chapter 8. Monitoring the Business Process Management environment 543 Figure 8-47 Add new widget 9. Select the Instances widget and drag it into the empty page. 10.Click Configure, as in Figure 8-48 to configure the instance view. Figure 8-48 Default Instance view 11.In the configuration view, select the monitor model, the monitor model context, and the required columns to be displayed, as shown in Figure 8-49 on page 545. 544 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 8-49 Instance view configuration page Note: If it is the first time you have configured the instance view, click Set as default to keep this settings as the default configuration. 12.The instance view should look as shown in Figure 8-50. Figure 8-50 Instance views Chapter 8. Monitoring the Business Process Management environment 545 546 z/OS: WebSphere Business Process Management V6.2 Production Topologies 9 Chapter 9. Creating an integrated user experience using Business Space powered by WebSphere Business Space powered by WebSphere is a component of WebSphere Process Server for z/OS V6.2. It provides a common interface for application users to create, manage, and integrate Web interfaces across the IBM WebSphere Business Process Management portfolio. Business Space powered by WebSphere functions are enabled by default for a stand-alone WebSphere Process Server for z/OS server. For a clustered implementation, you must complete the configuration manually. Business Space in WebSphere Process Server for z/OS V6.2 makes extensive use of REST Services. As part of the configuration of Business Space powered by WebSphere you must configure the REST Services Gateway and the required Rest Services EJBRoles. Business Space powered by WebSphere is a component of WebSphere Process Server for z/OS V6.2. It provides a common interface for application users to © Copyright IBM Corp. 2009. All rights reserved. 547 create, manage, and integrate Web interfaces across the IBM WebSphere Business Process Management portfolio. Perform the following steps to configure Business Space powered by WebSphere: 1. Create a data source for Business Space. See 9.1, “Create a data source for Business Space” on page 549. 2. Enable widgets in Business Space. See 9.1, “Create a data source for Business Space” on page 549. 3. Run the Business Space Configuration wizard. See 9.3, “Run the Business Space Configuration wizard” on page 551. 4. Prepare the Business Space DDL. See 9.4, “Prepare the Business Space DDL” on page 553. 5. Execute the Business Space DDL. See 9.5, “Execute the Business Space DDL” on page 553. 6. Define EJBROLE profile for role Administrator and the Rest Services Gateway user. See 9.6, “Define EJBROLE profile for role Administrator” on page 554. 7. Start the applications. See 9.7, “Start the applications” on page 555. 8. Enable business rules in the Business Space. See 9.8, “Enable business rules in the Business Space” on page 555. 9. Configure local Business Space help. See 9.9, “Configure local Business Space help” on page 557. 10.Clear the OSGI cache. See 9.9.1, “Clear the OSGI cache” on page 557. 548 z/OS: WebSphere Business Process Management V6.2 Production Topologies 9.1 Create a data source for Business Space The reason to create a data source for Business Space before running the Business Space configuration wizard is that the wizard will look for an existing data source with a JNDI name of jdbc/bpm/BusinessSpace and will use this if it is available. If it does not find an existing data source it will create one by modelling it on the existing data source called WBI_Datasource. Perform the following steps to create a data source for Business Space: 1. Create the data source at the cluster level by performing the following steps. a. In the “Step 1: Enter basic data source information” window, enter the following attributes: • • • Name: Business Space Data Source JNDI Name: jdbc/bpm/BusinessSpace Component Managed authentication alias: Choose the alias used by the BPC data source. For our MD cell that was BPCDB_mdcl01_WPS_Auth_Alias. b. In the “Step 2: Enter database specific properties for the data source” window, enter the following attributes: • • • • Database name: DB9F (the DB2 for z/OS Location name) Driver type: 2 Server name: wtsc42.itso.ibm.com (host name of DB2 DDF listener) Port number: 37893 (port number of DB2 DDF listener) c. In the “Step 3: Summary” window, click Finish. 2. Save the configuration changes. 3. Click Business Space Data Source. Click Custom properties. Set property fullyMaterializeLobData=false. 4. Save the configuration changes. 9.2 Enable widgets in Business Space All widgets are installed with Business Space powered by WebSphere but they must be enabled before they can be used in Business Space. The WebSphere Process Server for z/OS Information Center says to perform this step after configuring Business Space, but in our experience it is necessary to have the customized endpoint xml files in place before you run the Business Space configuration wizard. Chapter 9. Creating an integrated user experience using Business Space powered by WebSphere 549 When WebSphere Process Server for z/OS is configured into a stand-alone server, the widgets are enabled when the server is configured by zWPSConfig.sh. However, when WebSphere Process Server for z/OS is a network deployment configuration (managed node or a cluster) you must customize endpoint registry files that point to the virtual URLs for your cluster. The original endpoint files provided in the product registryData directory contain tags with relative paths that work well when all files are on a local host. They must be updated with the full URL when not. Perform the following steps to enable widgets in Business Space. 1. Create both of the directories /BusinessSpace/registryData under each node’s <profile_root> and copy the endpoint registration files wpsEndpoints.xml and bpcEndpoints.xml from the product’s <install_root>/BusinessSpace/wps/registryData directory to <was_home>/profiles/default/BusinessSpace/registryData on each node. We created a shell called copyendpoints.sh that makes the directories /BusinessSpace/registryData under <was_home>/profiles/default and copies the files. The shell also sets ownership of everything below the BusinessSpace directory to the cell’s file system owner and configuration group by issuing the following chown command: chown -Rh MDOWNER:MDCFG BusinessSpace 2. In bpcEndpoints.xml, edit the tags<tns:url> in each file to set a fully-qualified URL and port for the cluster where Business Process Container is configured. The port you must specify is the wc_defaulthost port for the cluster. For example, when configuring Business Space in our two-clusters topology MP cell we added the hostname:port of the cluster to the <tns:url> tags as high-lighted in bold in Example 9-1. Example 9-1 Customizing the bpcEndpoints.xml file <tns:Endpoint> <tns:id>{com.ibm.bspace.htm}bspaceTeamTaskListServiceRootId</tns:id> <tns:version>1.0.0.0</tns:version> <tns:url>http://wpsplex.itso.ibm.com:20048/rest/bpm/htm</tns:url> <tns:description>Location of services for HTM widgets</tns:description> </tns:Endpoint> 550 z/OS: WebSphere Business Process Management V6.2 Production Topologies 3. In wpsEndpoints.xml, edit the tags<tns:url> in each file to set a fully-qualified URL and port of the cluster running the core WebSphere Process Server for z/OS functions. If you have configured an HTTP Server to balance work across the cluster, specify the distributed VIPA address and port of the HTTP Server in both of the endpoint files. 9.3 Run the Business Space Configuration wizard Perform the following steps to run the Business Space Configuration wizard. 1. In the navigation pane click Servers → Clusters and click the cluster name. Business Space is part of the support infrastructure and so is configured in the support and messaging cluster when you are configuring it in the two-cluster topology. 2. On the Configuration page, under Business Integration, click Business Space Configuration, as shown in Figure 9-1. Figure 9-1 Launch the Business Space Configuration wizard 3. On the Business Space Configuration panel (Figure 9-2 on page 552), if Business Space has already been configured, you can view the panel but cannot edit the fields. When configuring Business Space for the first time, complete the panel as follows. Chapter 9. Creating an integrated user experience using Business Space powered by WebSphere 551 Figure 9-2 Configure Business Space schema and data source a. Select the Install Business Space service check box. b. In the Database schema name text box, enter the schema name you want to use for Business Space in DB2 for z/OS. We entered the default schema name of IBMBUSSP because in WebSphere Process Server for z/OS V6.1.2.0, Business Space did not use the schema name we entered. c. If you already created the data source Business Space data source with a JNDI name of jdbc/bpm/BusinessSpace, that data source will already be set in the Existing Business Space data source field. If you did not pre-define the Business Space data source, select a datasource from the list box in the Create Business Space data source using: field. For our cell the only datasource listed was WBI_DataSource, which is the one used by the WebSphere Process Server common components. Note: Selecting the WBI_DataSource does not mean that Business Space will use that datasource. Instead a new data source is created called Business Space DataSource with a JNDI name of jdbc/bpm/BusinessSpace and with the attributes of WBI_DataSource. 4. Save the configuration changes. 552 z/OS: WebSphere Business Process Management V6.2 Production Topologies 9.4 Prepare the Business Space DDL The DDL for the Business Space tables is located in the deployment manager’s configuration at <dmgr_root>/profiles/default/dbscripts/BusinessSpace/<dbtype>/<db_name>. For example, for our MD cell the Business Space DDL was in /wasv61config/mdcell/mddmnode/DeploymentManager/profiles/default/dbscripts /BusinessSpace/DB2zOSV9/DB9F. The DDL in the createTable_BusinessSpace-sql file is in ASCII. It is only partly customized. The DDL must be converted to EBCDIC and edited before you can run it using DSNTEP2. We created sample job MDCPYBSP to convert the Business Space DDL to EBCDIC and copy it to a z/OS sequential file for editing. The schema specified on the Business Space Configuration wizard is used to fully-qualify all the DDL. However, the following changes are required to the DDL: Split lines that continue beyond column 80. Change @STOGRP@ to our DB2 Storage group name, MDWPSSG. Change @DBNAME@ to our DB2 Database name, MDBPCDB. Change Bufferpools to those we wanted to use. Note: Use caution when changing buffer pools from their defaults. Some of the tables used by Business Spaces store large objects, and thus require 8 K and 16 K buffer pool definitions. After editing the DDL, we copied it to a partitioned dataset with LRECL 80 called WASCFG.MDCELL.WPS.SQL. 9.5 Execute the Business Space DDL We created a database for Business Space called MDBSPDB. We created job MDDDLBSP to execute the Business Space DDL in WASCFG.MDCELL.WPS.SQL(MDBSPTB), which contained our chosen schema of MDWPS. When we discovered that Business Space used a hard-coded schema of IBMBUSSP, we created new DDL with a schema of IBMBUSSP in member IBMBUSSP of dataset WPS.SQL. We created job IBMBUSSP in the WPS.CNTL dataset to execute that DDL. Chapter 9. Creating an integrated user experience using Business Space powered by WebSphere 553 9.6 Define EJBROLE profile for role Administrator When using RACF to perform EJBROLE authorization checking, it is necessary to define several EJB roles for Business Space use. The role Administrator is used for the Business Space administrator. Note that the role name starts with an upper case “A.” This means this is a different role to “administrator” (lowercase A), which is the Integrated Solutions Console administrator role. For our MD cell we created the role MD.Administrator as part of the job MDEJBROL. An extract of that job showing the commands applicable to Business Space is shown in Example 9-2. Example 9-2 Commands to define EJBROLE Administrator for Business Space RDEFINE EJBROLE PERMIT PERMIT MD.Administrator UACC(NONE) OWNER(WASCFG) DATA('MD BSpace Admin role') MD.Administrator CLASS(EJBROLE) RESET MD.Administrator CLASS(EJBROLE) ID(MDBSPADG) ACCESS(READ) + + + + + + + As shown in Example 9-2, we defined a specific group called MDBSPADG (BSPADG=Business Space Administrator Group) to which other user IDs can be connected to be given the role Administrator. If you prefer to use GEJBROLE profiles, use job GEJBROLE, which assigned the Business Space role Administrator to the profile protecting administration roles. We also defined several other roles including those used for access to the REST services Gateway functions provided by WebSphere Process Server for z/OS. We also defined a role for use with the Fabric REST Services Gateway. All of these definitions are included in a job we created for that purpose, MDEJBBSR. Refer to 7.3.3, “Configure the Fabric Rest Services gateway” on page 435 for a through treatment of all roles required for Business Spaces. 554 z/OS: WebSphere Business Process Management V6.2 Production Topologies 9.7 Start the applications To start the applications navigate to Applications → Enterprise applications, select the BusinessSpaceManager and IBM BSPACE WIDGETS check boxes, and click Start (Figure 9-3). Remember to verify that the applications start and that there are no errors in the logs. Figure 9-3 Start the Business Space applications 9.8 Enable business rules in the Business Space After installing Business Space powered by WebSphere in a clustered environment, you must run a script so that business rule widgets are available in Business Space at run time. This is configured automatically on a stand-alone server profile, but when you configure a cluster you need to run a JACL script. The script that you use to enable business rules in Business Space is called installBRRestAPI.jacl and is located in the WebSphere Process Server for z/OS <install_root>/bin directory. We created sample job MDBUSRUL to run installBRRestAPI.jacl, as shown in Example 9-3 on page 556. Chapter 9. Creating an integrated user experience using Business Space powered by WebSphere 555 Example 9-3 Sample JCL MDBUSRUL to run installBRRestAPI.jacl //MDBUSRUL JOB (0),CLASS=A,MSGCLASS=A,REGION=0M, // USER=MDADMIN,PASSWORD=MDADMIN //***********************************************************/ //* Configure Business Rules in Business Space */ //***********************************************************/ //INSTO EXEC PGM=IKJEFT01,REGION=0M,TIME=1440 //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BPXBATCH SH + cd /wasv61config/mdcell/mddmnode/DeploymentManager/bin; + ./wsadmin.sh -user MDADMIN -password MDADMIN + -f installBRRestAPI.jacl + -clusterName mdcl01_WPS + 1> /tmp/mdbussrul_1308.out + 2> /tmp/mdbussrul_1308.err; //***********************************************************/ //* STEPS TO COPY THE OUTPUT THE JOB LOG */ //***********************************************************/ //CPOUT EXEC PGM=IKJEFT01,REGION=0M //SYSEXEC DD DISP=SHR,DSN=BBO6142.MDDMNODE.SBBOEXEC //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BBOHFSWR '/tmp/mdbussrul_1308.out' BBOHFSWR '/tmp/mdbussrul_1308.err' /* An extract of the messages issued is shown in Example 9-4. Example 9-4 Messages issued when running installBRRestAPI.jacl WASX7303I: The following options are passed to the scripting environment and are available as arguments that are stored in the argv variable: "[-clusterName, mdcl01_WPS]" WASX7327I: Contents of was.policy file: grant codeBase "file:${application}" { permission java.security.AllPermission; }; ADMA5016I: Installation of BusinessRuleRestAPI_mdcl01_WPS started. ADMA5058I: Application and module versions are validated with versions of deployment targets. . . ADMA5013I: Application BusinessRuleRestAPI_mdcl01_WPS installed successfully. >>>>> The application is successfully installed. >>>>> Please wait for starting the application... >>>>> The application is successfully started. 556 z/OS: WebSphere Business Process Management V6.2 Production Topologies 9.9 Configure local Business Space help By default, Business Space uses the Internet to access the help on the IBM Web site. If you want to change the configuration of the help so that business users access the help on your local intranet, you can change the help URL on the Business Space landing page. Perform the following steps to change the help URL on the Business Space landing page: 1. Obtain the URL and port of your local help server. The default help port is 8888. Look in either the IC_start.bat file or the IC_start.sh file to see the port the help server is using. 2. Go to the BSpaceManager.ear file, where it is deployed. 3. Open the landingpage.html with an ASCII text editor. The file is located at BSpaceManager.ear_location/WebContent/bspace_help/framework_help/. 4. Change the help URL to an internal URL, like the following URL: http://wpsplex.itso.ibm.com:8888/help/topic/com.ibm.btools.help.\ modeler.collab.publish.doc/doc/tasks/viewing/viewingdataitems.html 5. Save and close the landingpage.html file. 9.9.1 Clear the OSGI cache Once you have completed the configuration steps for Business Space powered by WebSphere you must clear out the OSGI cache. To do this, perform the following steps: 1. Make sure the deployment manager, node agents, and clustered servers are all stopped. 2. For the Deployment Manager, from a telnet or OMVS session, go to /wasv61config/mdcell/mddmnode/DeploymentManager/profiles/default/ configuration and delete the contents of this directory. 3. For node A, from a telnet or OMVS session, go to /wasv61config/mdcell/mdnodea/AppServer/profiles/default/ configuration and delete the contents of this directory. 4. For node B, from a telnet or OMVS session, go to /wasv61config/mdcell/mdnodeb/AppServer/profiles/default/ configuration and delete the contents of this directory. Chapter 9. Creating an integrated user experience using Business Space powered by WebSphere 557 9.9.2 Verify Business Space Perform a brief test of Business Space to verify the installation. 1. Stop and restart your cluster and servers to pick up all changes to the configuration that were made. 2. Launch the Business Space application from a browser using context root /BusinessSpace. For example, for our MD cell, the URL was as follows: http://wpsplex.itso.ibm.com:20048/BusinessSpace 3. Log in with a user ID that has been permitted to the Business Space role Administrator (with an upper case A). Earlier we had permitted RACF group MDBSPADG to the EJBROLE profile called MD.Administrator and the group MDBSPADG contained the user ID MDBSPADM. Therefore we logged in with user ID MDBSPADM. If RACF APPL class is active, ensure that the user ID you log in with has access to the APPL profiles protecting the cell. 4. The Business Space log on window should appear (Figure 9-4). Figure 9-4 Business Space login window 558 z/OS: WebSphere Business Process Management V6.2 Production Topologies 5. After logging in, you see the Business Space manager workspace shown in Figure 9-5. It is largely empty at this point because the user ID has no Business Spaces. The Business Space Manager workspace is shown so you can create a new Business Space and add new pages to the space. Then you can add widgets to the pages, a bit like added portlets to pages in WebSphere Portal Server. Figure 9-5 Initial Business Space Welcome window Chapter 9. Creating an integrated user experience using Business Space powered by WebSphere 559 6. A user can have many business spaces with each one having a different purpose but Business Space can display the contents of one space at a time. That space is known as the open Business Space. Create a new Business Space by navigating to the Business Space Manager. Select Manage Business Spaces from the Welcome drop-down menu. See Figure 9-6. Figure 9-6 Navigate to the Business Space Manager 7. Click the icon on the top left in the tool bar as shown in Figure 9-7. Figure 9-7 Click the icon to create a new Business Space 560 z/OS: WebSphere Business Process Management V6.2 Production Topologies 8. Enter a name for the Business Space (such as TestSpace1), choose the Business Monitoring template and click OK, as shown in Figure 9-8. Figure 9-8 Create a new space Chapter 9. Creating an integrated user experience using Business Space powered by WebSphere 561 9. A new Business Space named TestSpace1 is created. Because we used the Business Monitoring template, our space was pre-populated with several pages. See Figure 9-9. Figure 9-9 Business Space with Business Monitoring pages deployed 10.Select the KPIs page and observe the widgets that are deployed on the page (Figure 9-10 on page 563). To get more specific information from the deployed widgets we would need to configure a WebSphere Business Monitor server for the widgets to communicate with. 562 z/OS: WebSphere Business Process Management V6.2 Production Topologies Figure 9-10 KPI widgets deployed on KPI Business Space Now let’s create a Space we can do something with. 1. Enter a name for the Business Space (such as TestSpace2), make this an empty page by selecting Empty, as shown in Figure 9-11. Figure 9-11 Create an empty page Chapter 9. Creating an integrated user experience using Business Space powered by WebSphere 563 2. The new Testspace 2 is created. Select it and click the new page icon as shown in Figure 9-12. Figure 9-12 New TestSpace 2 created 3. Select a spreadsheet widget from the left pane and drag it into your new TestPage1. 564 z/OS: WebSphere Business Process Management V6.2 Production Topologies 4. Click Save to download the sample spreadsheet or Open to open it in the browser. We first tested that we could download and save the spreadsheet. If you decide to save the spreadsheet on your workstation, your browser will probably issue a warning about downloading a file. Click Continue if you see the message shown in Figure 9-13. Figure 9-13 Click continue to allow download of the spreadsheet 5. Save the file on your workstation. 6. Click Open when the download is complete. The spreadsheet will open in Excel®. This has verified the file-download function. 7. Click the spreadsheet widget on the page. You are prompted to Open or Save (Figure 9-14). Click Open. Figure 9-14 Click Open to open the file on the page Chapter 9. Creating an integrated user experience using Business Space powered by WebSphere 565 8. The sample spreadsheet widget is deployed on your TestPage2 will open in the area on the page reserved for the first widget, as shown in Figure 9-15. Figure 9-15 Sample spreadsheet widget deployed on TestPage1 of Testspace2 9. Click the Add a widget icon that is in the right pane. 566 z/OS: WebSphere Business Process Management V6.2 Production Topologies 10.Add the My Tasks widget. At this time you will have no tasks to display so the My Tasks pane is empty, as shown in Figure 9-16. You have now verified the basic functionality of Business Space. Figure 9-16 Business Space with Tasks widget deployed Chapter 9. Creating an integrated user experience using Business Space powered by WebSphere 567 568 z/OS: WebSphere Business Process Management V6.2 Production Topologies Part 4 Part 4 Problem determination © Copyright IBM Corp. 2009. All rights reserved. 569 570 z/OS: WebSphere Business Process Management V6.2 Production Topologies 10 Chapter 10. Problem diagnosis and prevention This chapter discusses some of the problems we encountered, and describes how we discovered and fixed them. We also provide some techniques and tools we use in our troubleshooting experiences. This chapter presents the following topics: 10.1, “Overview of problem diagnosis and prevention” on page 572 10.2, “Problem prevention” on page 573 10.3, “Tools and techniques” on page 575 10.4, “Configuration problems” on page 590 © Copyright IBM Corp. 2009. All rights reserved. 571 10.1 Overview of problem diagnosis and prevention Effective problem diagnosis is not an exact science. It is based on a thorough understanding of WebSphere products and z/OS, experience (usually unsuccessful ones), pattern recognition, contacts, and the ability to use many tools. Your best tools are your brain, memory, ability to search many information sources, and communication with other subject-matter experts. Listed here are some important points: Problem prevention – – – – Good naming conventions, organization and documentation Diverse skills Comprehensive information resources Good practices with WebSphere products Tools and Techniques – – – – – – TSO UNIX DB2 Other z/OS Security Tracing Categories of problem areas and methodologies – – – – – Configuration Security DB2 Runtime Top ten problem categories Problems can be categorized into Configuration problems, Deployment problems, Runtime problems (and No problems), but let’s start with prevention. 572 z/OS: WebSphere Business Process Management V6.2 Production Topologies 10.2 Problem prevention Avoiding problems depends on the skills of the people, the preparation of the material and resources, the organization and documentation of the implementation plan. The importance of a well-organized and documented naming standard cannot be over-emphasized. A good naming standard such as the one documented by the Washington Systems Center in Techdoc WP100653 does a good job of naming the components of a WebSphere Application Server Network Deployment for z/OS cell. The Techdocs mentioned in the following sections can be found at the following Web page: http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101138 10.2.1 Good management practices A smoothly running set of servers will help in preventing and recovering from problems. Here are some good practices to help with that. For some good advice on managing your servers in WebSphere Application Server for z/OS, see Techdoc article WP101138, Hidden Gems: Little known features of WebSphere Application Server for z/OS. Managing messages Too many messages from WebSphere Application Server for z/OS servers fill up SYSLOG and make it difficult to see the important messages there. To help with the routing of messages away from SYSLOG, see Techdoc TD10369. Managing timeouts In the WebSphere Application Server for z/OS environments, your application servers must cope with a variety of work requests. In normal operations, the servers handle the beans, servlets, and other the types of application workloads efficiently and without incident. What happens, however, when processing does not go as planned? That is, when applications have bugs, network connections hang, needed resources are slow or unresponsive, and bad things just happen? See Techdoc article WP101233 Configuration Options for Handling Application Dispatch Timeouts. Chapter 10. Problem diagnosis and prevention 573 10.2.2 Assemble required resources You should plan for adequate resources to ensure that your production topology can perform up to your expectations of performance and availability goals. System z processor requirements As pointed out in 1.1.2, “Hardware considerations” on page 6, multiple LPARs with multiple CPUs, including general-purpose and zAAP processors are required for most production topologies. Contact your IBM representative to have a capacity planning or sizing study run for your environment. Real memory requirements Do not try running WebSphere Process Server for z/OS on any systems with less than 2 GB memory. Memory requirements depend on the number of servers, the size of the JVM heaps, and whether you are running with 64-bit heaps. DB2 storage requirements should also be considered. Roughly speaking, you should allow 1 GB for system tasks, and 1 GB for each server and deployment manager. Adjust up for multiple servant regions and 64-bit heaps. Having a good auxiliary storage (paging) subsystem may also be required if you are pushing the limits or real storage. Virtual storage requirements Do not be constrained in your private region size by REGION size specifications, or other limits such as the IEALIMIT exit. As with real storage requirements, having a good auxiliary storage (paging) subsystem will also help. DASD space requirements Do not under-estimate the room required to hold ZFS and HFS file systems, backups of the same, DB2 database tables, auxiliary (paging) data sets, JES SPOOL, and so forth. In our testing, we were fortunate enough to have a substantial farm of 3390-9 volumes. 574 z/OS: WebSphere Business Process Management V6.2 Production Topologies 10.2.3 Acquire appropriate skills Many skills are required to configure and trouble-shoot these products: MVS systems programming UNIX, telnet, vi, scripting TSO, ISPF, SDSF Security (RACF, LDAP) Communications (TCP/IP) Java programming Ensure you have these skills on your team, or access to them when necessary. 10.3 Tools and techniques Here are some valuable techniques that we learned, along with some useful tools you may find helpful. 10.3.1 Parallel testing cells We configured multiple cells for our use, including the MDCELL, M1CELL, and M2CELL. The M1CELL and M2CELL were almost exact copies of the MDCELL, except for the cell prefix (MD → M1, md → m1, MD → M2, md → m2). This proved valuable in our implementation, testing, problem determination, and repair work. While some team members were configuring Business Space powered by WebSphere, and WebSphere Business Services Fabric for z/OS, other team members could use parallel cells for experimentation or other testing. For example, if there were configuration errors when creating clusters, we could verify various resource settings on data sources and messaging engines by comparing the cells. 10.3.2 TSO techniques As MVS systems programmers, this is the most common interface you have to do your work. Make sure you master the efficiencies of TSO, ISPF, SDSF, and other products like RACF panels, OMVS, IShell, WLM, RMF, and DB2ADM to assist you. Chapter 10. Problem diagnosis and prevention 575 Using wide 3270 screens When using TSO and SDSF, it is difficult to browse logs with an 80-character wide 3270 screen because you have to scroll right and left to see complete messages. See Techdoc TD102151 for directions to set up your PCOM sessions to display a 132-column by 50- or 60-row format, as in Figure 10-1 Figure 10-1 Displaying square brackets in your JES logs Do you ever see strings in your logs that look like the following example: Ýjcc¨Ý50053¨Ý12311¨Ý3.53.81¨ T2zOS exception: Ýjcc¨ÝT2zos¨ When it should look like this: [jcc][50053][12311][3.53.81] T2zOS exception: [jcc][T2zos] 576 z/OS: WebSphere Business Process Management V6.2 Production Topologies The display profile for your PCOM 3270 session can be specified to translate square brackets with the following xlt file: Example 10-1 display.xlt file for translating square brackets [Profile] id=XLT Description=Translation table to correctly display C square brackets [] [Option] Replace=Y ErrorMessage=Y [SB Xlate] AD=5B BD=5D Save this display.xlt file in the following location: c:\Program Files\Personal Communications\private\display.xlt Then, add the stanza in Example 10-2 to your PCOM .WS profile: Example 10-2 Translation directive in your PCOM WS file [Translation] IBMDefaultView=0 DefaultView=c:\Program Files\Personal Communications\private\display.xlt SDSF tricks SDSF is the primary interface for z/OS operations in most installations. There are many panels and commands in SDSF that can make you more productive in the operation and trouble-shooting of these products. See Using SDSF for WebSphere for z/OS Operations TD100589. Generalizing RACF profiles If you build a production topology with WebSphere Application Server Network Deployment for z/OS and BPM products, you can set up a generic set of RACF profiles to cover multiple servers and nodes in your cell from the beginning. See Using Generic RACF Profiles with WebSphere on z/OS V7 Techdoc WP101427. Chapter 10. Problem diagnosis and prevention 577 10.3.3 Scripting There are many useful scripts which can help diagnose and correct problems in the WebSphere Application Server for z/OS environment. Fixing permissions & ownership of files in configuration HFS The following scripts were run in the BBOWHFSB job when we configured the standalone server: bbowhfsb: Sets the permission bits bbowhfsc: Sets ownership attributes There may be times when the permission and ownership attributes need to be corrected after some activities or mistakes. Here is an example of running these scripts in a telnet session: Example 10-3 Running the bbowhfsc and bbowhfsc scripts for deployment manager and nodeb cd /usr/lpp/zWebSphere/V6R1/bin ./bbowhfsb.sh /wasv61config/mdcell/mddmnode/DeploymentManager ./bbowhfsc.sh /wasv61config/mdcell/mddmnode/DeploymentManager mdadmin mdcfga ./bbowhfsb.sh /wasv61config/mdcell/mdnodea/AppServer ./bbowhfsc.sh /wasv61config/mdcell/mdnodea/AppServer mdadmin mdcfg wsadmin scripting Many scripts use the wsadmin.sh tool to automate configuration tasks. See the following techdocs articles for some good examples: WSADMIN Primer (with Jython),WP101014 – Includes several sample hands-on exercises. – Replaces WSADMIN Primer (with JACL) - WP100421 Using Jython Scripting Language with WSADMIN, WP100963 Includes several sample scripts to do common administrative tasks. Many Information Center articles have good examples. Updating the ports, virtual host aliases, and short names Techdoc article TD104066 can help update or create new application servers in WebSphere Application Server for z/OS. This Techdoc article also includes sample jython scripts to correct automatically the port assignments to the WSC standard numbering scheme. 578 z/OS: WebSphere Business Process Management V6.2 Production Topologies Running scripts in batch jobs You have many choices of how to run your scripts besides telnet or OMVS. Example 10-4 is a sample job to run the zWPSConfig.sh script. Example 10-4 Running a shell script in a batch job //MYSCRJOB JOB (0),CLASS=A,MSGCLASS=A,REGION=0M,... //***********************************************************/ //* Run zxy.sh */ //***********************************************************/ //INSTO EXEC PGM=IKJEFT01,REGION=0M,TIME=1440 //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BPXBATCH SH + cd /wasv61config/mdcell/mddmnode/DeploymentManager/bin; + ./zWPSConfig.sh + 1> /tmp/MDdmgr_CwpsAug.out 2>&1 //***********************************************************/ //* STEP TO COPY THE OUTPUT THE JOB LOG */ //***********************************************************/ //CPOUT EXEC PGM=IKJEFT01,REGION=0M //SYSEXEC DD DISP=SHR,DSN=BBO6142.MDDMNODE.SBBOEXEC //SYSTSPRT DD SYSOUT=* //SYSTSIN DD * BBOHFSWR '/tmp/MDdmgr_CwpsAug.out' /* Running scripts without interruption in telnet sessions You can use the nohp utility to ignore the SIGHUP signal, which may occur if you lose your connection with the host. Whether you use Windows telnet client or TeraTerm or PuTTy, to execute scripts in the background without being interrupted or disconnected, see the example in Example 10-5. Note the position of quotes around the command being executed, and the ampersand (&) at the end to run the command in a different thread so you can disconnect or continue with other commands in your telnet session. Example 10-5 Running shell scripts in background to disconnect nohup sh –c ’./wasv6config/mdcell/mdnodea/AppServer/bin/zWPSConfig.sh \ -augment -response /u/user1/wpswork/mdProfile.rsp' \ >/tmp/mdWPSConfig.out 2>&1 & Chapter 10. Problem diagnosis and prevention 579 Tracing shell scripts If you want to see the detailed results of a shell script you are running, you can precede it with the “sh -x” string as in Example 10-6. Example 10-6 Invoking a shell script with tracing sh –x ’./wasv6config/mdcell/mdnodea/AppServer/bin/zWPSConfig.sh \ -augment -response /u/user1/wpswork/mdProfile.rsp' \ >/tmp/mdWPSConfig.out 2>&1 10.3.4 UNIX system services: Tools and techniques You have many choices of tools to explore logs, property & other files in the HFS and ZFS file systems besides OMVS and IShell. 10.3.5 Using telnet, vi and ISPF 3.17 The following profile and setup files in your home directory will make your life easier when operating in a telnet session. Tailoring your .profile and .setup files In your home directory, you can customize your UNIX environment to provide short-cuts and settings that will make you more productive when you are exploring and working in the UNIX file system environment. See 2.4.8, “Customizing .profile and .setup files for users” on page 98 Creating a .setup file To help with typing in telnet sessions, we created some alias names in the .setup files in the same home directories as the .profile files. Example 10-7 is a sample setup file that is run automatically when you log in using telnet. You may want to tailor it according to your preferences. Example 10-7 Sample setup file alias ll="ls -lat" alias dirs="ls -alLD" set -o vi In Example 10-7, "ll" allows you to list the files details, with the newest ones at the top. 580 z/OS: WebSphere Business Process Management V6.2 Production Topologies "set -o vi" helps you retrieve commands from the access history. Press the escape key once (esc), release it, and press k. The last command that you typed will appear on the window. Keys j and k help you scroll back and forth in the history buffer. 10.3.6 UNIX System Services tools on z/OS There are many tools available. Here are a few to help you. Dealing with ASCII files z/OS (and MVS before it) uses an EBCDIC code page, but the WebSphere Application Server for z/OS runtime uses an ASCII code page for a lot of its xmp files, property files and logs. Most TSO file editors like ISPF do not deal well with ASCII files (until recently). Here are some tools that do. iconv Convert a file from ASCII (ISO8859-1) to EBCDIC (IBM-1047) with the iconv command: iconv -f ISO8859-1 -t IBM-1047 x.asc > x.ebc viascii A useful tool for editing ascii files with the 'vi' editor. http://www.ibm.com/servers/eserver/zseries/zos/unix/bpxa1ty2.html Other z/OS UNIX tools For some other useful tools see the following web sites: ASCII to EBCDIC conversion: http://www.ibm.com/servers/eserver/zseries/zos/unix/bpxa1p03.html Tools and Toys: http://www.ibm.com/servers/eserver/zseries/zos/unix/bpxa1ty2.html Write your own There are some samples in Techdoc PRS2520. TSO/E tools dealing with ASCII files You do not have to use telnet and viascii to browse or edit ASCII files. There are tools now available for TSO users. Chapter 10. Problem diagnosis and prevention 581 ISPF 3.17 - the z/OS UNIX Directory List Utility This was first provided in z/OS 1.8, and gives you a full-screen column-arranged panel where you can browse and edit files in EBCDIC and in ASCII. Use the Menu, Utilities, View, Options, and Help drop-down menus to learn about this useful panel on Figure 10-2. Figure 10-2 ISPF 3.17 z/OS UNIX Directory List panel Other tools The above tools are preferred by the author, but there are others listed below. Tag the file as ASCII and enable automatic conversion. This technique has been around for a long time; we recommend using ISPF 3.17 instead of this. Example 10-8 Using the AUTOCVT conversion utility SETOMVS AUTOCVT=ON (MVS console command) export _BPXK_AUTOCVT=ON chtag -tc ISO8859-1 ascii_file ? oedit ascii_file Once you issue the chtag command, it stays on until you turn it off. (Do not enable during Java compilation or execution.) When AUTOCVT(ON) is set, every read or write operation for a file is checked to see if conversion is necessary. A performance penalty is therefore involved, even if no conversion occurs. It is recommended that AUTOCVT be left off and each program be enabled for conversion. 582 z/OS: WebSphere Business Process Management V6.2 Production Topologies FTP client Another technique some people prefer is to use a FTP client, and download the file to your workstation where you can view or edit it. Some FTP clients may be able to view it directly in binary. NFS Workstation users can use the Network File System (NFS) connect to a host through TCP/IP and mount a file system that on the host so it appears as part of the user's local file system. For more information, see SC26-7417 z/OS Network File System Guide and Reference. DFS SMB The z/OS Distributed File Service (DFS) Server Message Block (SMB) support provides a server that makes Hierarchical File System (HFS) files available to SMB clients. The SMB protocol supported through TCP/IP allows clients to access shared directory paths from workstations in the network. For more details, see SC24-5918 z/OS Distributed File Service SMB Administration. 10.3.7 DB2 tools If you do not have a DB2 administrator or expert on your team, it is good to know how to use some of the helpful tools available. Using ADBL: The DB2 administrator’s friend DB2 Admin is an interface between ISPF and DB2. It can issue SQL calls and display any resulting rows in an ISPF table panel. It is a panel-driven ISPF application that allows you to drill down into the DB2 catalog, databases, tables, table spaces, and other DB2 objects. For example, to see what tables are in a particular database, enter 1 (DB2 catalog) and D (databases), then enter the database name prefix at the top of the Name column, as in Figure 10-3 on page 584. Chapter 10. Problem diagnosis and prevention 583 Figure 10-3 DB2 Administrative tool showing databases for the “MD” cell You can then enter T next to one of the database names to display the tables in the database. You can use the sort command to sort various columns, and drill down deeper to see the columns and comments of the tables. DB2 tricks The following techniques may help you manage the DB2 administration tasks. DASD space used by your table spaces DB2 Table spaces are backed by VSAM datasets whose names have the following format: Vcatname.DSNDBD.dbname.tsname.y0001.znnn where … In the dataset above, the variables are explained as follows: Vcatname: The integrated catalog name (high level index), or VCAT DSNDBD: The data component of the VSAM cluster dbname: The database name tsname: The table space name or index name Example 10-9 Displaying the DB2 table spaces sorted with the largest ones on top Command ===> sort tracks TRACKS --------------------------------------------------WPSDB2.DSNDBD.MDESBDB.ESBCLOB.I0001.A001 150 WPSDB2.DSNDBD.MDWPSDB.BBLOB2TS.I0001.A001 150 WPSDB2.DSNDBD.MDWPSDB.TICKETTS.I0001.A001 150 WPSDB2.DSNDBD.MDBPEDB.SCHE1Y3Y.I0001.A001 15 WPSDB2.DSNDBD.MDESBDB.ESBTS.I0001.A001 15 WPSDB2.DSNDBD.MDEVT.BLOB00RV.I0001.A001 6 584 z/OS: WebSphere Business Process Management V6.2 Production Topologies Executing DDL with errors Many time during the initial attempts at building your DB2 databases and creating the tables, you may be forced to drop the databases, storage groups, table spaces, or tables. When you re-create them using a large generated set of SQL or DDL statements, there may duplicate objects such as functions or sequences that were not dropped. Usually when DSNTEP2 encounters ten SQLCODE=-601 errors, it will flush the remaining SQL statements. If you use DSNTEP2, you can add the statement at the beginning of your DDLs, as in the following example: --#SET MAXERRORS n In the example above, n is the number of errors allowed before the program should continue. Use -1 to indicate that the program should tolerate an unlimited number of errors. Severe SQL errors will cause program termination whenever encountered. Using the DBUtility.sh script The DBUtility.sh supplied with WebSphere Process Server for z/OS in the {app_server_root}/bin/ directory of the configuration HFS (for example: /wasv61config/mdcell/mddmnode/DeploymentManager/bin/). Use this to execute some SQL statements that you receive from another installation, such as the WebSphere Business Monitor installation on another system. The following procedure is an example to use it to execute the SQL statements that you may have on your workstation. Note: The SQL statements are not well-formed for execution using the DB2 DSNTEP2 program or SPUFI, because they are much longer than 72 columns. 1. Edit the file with WordPad and save it on your local workstation. This adds the cr/lf characters so the record delimiters will be proper when transferred to the z/OS system. 2. Use FTP to transfer the file in binary mode to your working UNIX System Services directory on z/OS. Chapter 10. Problem diagnosis and prevention 585 3. Edit the SQL file, changing the following for your installation: – $STOGRP$: This variable represents the name of the Monitor database. (We used MDMONSG.) – $DBNAME$: This variable represents the name of the Monitor database. (We used MDMONDB.) – $SCHEMA$: This variable represents the name of the Monitor schema prefix. (We used MDMON.) Note: You must use a separate schema prefix from the BPC and WPS Common database tables because some of the Monitor table names are identical table names in those databases. 4. Make sure the file permissions allow it to be read by the following script. 5. Create a shell script to invoke the DBUtility.sh script, using Example 10-10 as a template. Example 10-10 Sample DBUtility.sh script #!/bin/sh # # This shell script will run the DButility.sh script # pointing to the sql statements for # WebSphere Business Monitor. # Make sure there are no trailing blanks after the back-slashes! # $WAS_HOME/bin/DBUtility.sh createTable \ -DdbStorageGroup=MDMONSG \ -DdbSchemaName=MDMON \ -DsqlScriptName.default=BusMon_createTables.sql \ -DsqlScriptPath.default=/var/WebSphere/home/MDCFG/wpswork/BusMon_createData baseDb2Zos.sql \ -DdbType=DB2UDBOS390_V8_1 \ -DdbName=MDMONDB \ -DprofileName=default \ -DprofilePath=$WAS_HOME/profiles/default \ -DdbJDBCProperties=/etc/d9fg \ -DdbConnectionLocation=DB9F \ -DdbJDBCClasspath=/usr/lpp/db2/d9fg/db2910_jdbc/classes \ -DdbUserId=MDADMIN -DdbPassword=MDADMIN \ -DdbDelayConfig=false \ -DdbCreateNew=false \ -DdbHostName=wtsc42.itso.ibm.com \ -DdbServerPort=37893 \ >/tmp/mdcell/BusMonDdl.out 2>/tmp/mdcell/BusMonDdl.err 586 z/OS: WebSphere Business Process Management V6.2 Production Topologies 6. Export the LIBPATH for the DB2 libraries: EXPORT LIBPATH=/usr/lpp/db2/d9fg/db2910_jdbc/lib:$LIBPATH 7. Export the $WAS_HOME and $STEPLIB variables by changing to the <WAS_HOME>/bin/ directory and running the setupCmdLine.sh script as in Example 10-11. Example 10-11 Running the setupCmdLine.sh script cd /wasv61config/mdcell/mdnodea/AppServer/bin . setupCmdLine.sh Notice that the command . setupCmdLine.sh is preceded by a period and a space (“. “). Running the command in this fashion retains the export of WAS_HOME that is performed by setupCmdLine.sh and makes WAS_HOME available to the createDB.sh script. Verify the variables that were set by running the setupCmdLine.sh script (and the user ID that you are running with) using the UNIX commands in Example 10-12. Example 10-12 Verifying the variables set for running DBUtility.sh >id uid=209(MDADMIN) gid=16(MDCFG) >echo $WAS_HOME /wasv61config/mdcell/mdnodea/AppServer >echo $STEPLIB none:BBO6142.MDNODEA.SBBOLOAD:BBO6142.MDNODEA.SBBOLD2: BBO6142.MDNODEA.SBBOLPA: DB9F9.SDSNEXIT:DB9F9.SDSNLOAD:DB9F9.SDSNLOD2 >echo $LIBPATH usr/lpp/db2/d9fg/db2910_jdbc/lib:. . . 8. Return to your working directory, and run your modified script to execute the DBUtility.sh script. 9. Verify that the script ran successfully and the SQL statements were run by looking in the following places: – /tmp/mdcell/BusMonDdl.out – /tmp/mdcell/BusMonDdl.err – The DB2 Administrator’s Utility (ADBL) on the z/OS system (or SPUFI) to see that the 95 tables and 97 table spaces were created for the Monitor. Chapter 10. Problem diagnosis and prevention 587 10.If you need to modify the SQL statements and re-run the script, you must delete the SQL file in the following directory, or the old SQL statements will be used: <profile_root>/dbscripts/CommonDB/DB2zOSV8/MDMONDB If you want to see the detailed results of the shell script you are running, you can precede it with the “sh -x” string as in Example 10-13. Example 10-13 Invoking a shell script with tracing sh –x $WAS_HOME/bin/DBUtility.sh createTable \ -DdbStorageGroup=MDMONSG \ -DdbSchemaName=MDMON \ -DsqlScriptName.default=BusMon_createTables.sql \ . . . 10.3.8 Tracing When you cannot determine the cause of a problem, you may have to turn on tracing to get more information. zSMPInstall tracing Add the following string to your zSMPInstall.sh parameters: -trace *=all=enabled The trace logs go to the {app_server_root}/logs/ directory. zWPSConfig.sh tracing Add the same string to the zWPSConfig.sh command: -trace *=all=enabled These trace logs go to the {profile_root}/logs/ directory. We recommend that you do not enable tracing until you need to. You get plenty of information in the logs by default. WebSphere Java runtime tracing The MVS modify command (‘f’f for short) is extremely powerful and dynamic: F <server_name>,tracejava='com.ibm.xxx.*= all=enabled' Here is an example to start BPE tracing dynamically: F server-name,TRACEJAVA='com.ibm.bpe.*=all=enabled' 588 z/OS: WebSphere Business Process Management V6.2 Production Topologies To turn off tracing, issue: F server-name,TRACEINIT wsadmin tracing Edit the {profile_root}/properties/wsadmin.properties file, by uncommenting the line as shown in Example 10-14. #com.ibm.ws.scripting.traceString=com.ibm.*=all=enabled Example 10-14 Portion of the {profile_root}/properties/wsadmin.properties file #---------------------------------------------------------------------# The traceFile property determines where trace and logging # output are directed. If more than one user will be using # wsadmin simultaneously, different traceFile properties should # be set in user properties files. # The default is that all tracing and logging go to the console; # it is recommended that a value be specified here. # If the file name contains DBCS characters, use unicode format such as #-------------------------------------------------------------------com.ibm.ws.scripting.traceFile=/wasv6config/mdcell/mdnodea/AppServer/pr ofiles/default/logs/wsadmin.traceout #--------------------------------------------------------------------# The traceString property governs the trace in effect for # the scripting client process. # The default is no tracing. #--------------------------------------------------------------------com.ibm.ws.scripting.traceString=com.ibm.*=all=enabled Then run the script and go look in the location specified by the property scripting.traceFile= {profile_root}/logs/wsadmin.traceout JDBC tracing This can be enabled by specifying a traceFile in the DB2JccConfiguration properties file, which is pointed to from the servant Java options. In the Integrated Solutions Console, navigate to Application servers → mdsr01a_WPS → Server Instance → Process Definition → Servant → Java Virtual Machine → Custom Properties, and enter a Name of db2.jcc.propertiesFile with a value pointing to the location of your properties files, such as: /wasv61config/mdcell/mddmnode/DB2JccConfiguration.properties Chapter 10. Problem diagnosis and prevention 589 A typical file showing the DB2 subsystem ID and the parameter to turn on JDBC tracing is shown in Example 10-15. Example 10-15 DB2JccConfiguration.properties file db2.jcc.ssid=D9FG db2.jcc.override.traceFile=/tmp/MD_tracejdbc.txt For detailed guidance on reading the traces, see DB2 Universal Database for z/OS Application Programming Guide and Reference FOR JAVA Version 8 SC18-7414, available at the following Web page: http://publibfi.boulder.ibm.com/epubs/pdf/dsnjvj11.pdf 10.4 Configuration problems Most of the problems we encountered were during the configuration of the various products. WebSphere Application Server for z/OS WebSphere Process Server for z/OS WebSphere Business Services Fabric for z/OS Business Space powered by WebSphere The following sections are some summarized approaches for troubleshooting that we use: 10.4.1 Methodologies for determining configuration problems The general approach we use depends on the phase of configuration: Running scripts Creating DB2 databases and tables Starting up servers Testing the runtime with sample applications. Running scripts Before you run the scripts, make certain that you have prepared all the prerequisite material and know how to run it. Verify the correct syntax. Verify proper user identities (and passwords). Verify any environmental settings, such as STEPLIBs, $WAS_HOME variables. 590 z/OS: WebSphere Business Process Management V6.2 Production Topologies After you run the scripts perform the following tasks: Check the return codes. Browse the output. Examine the logs. If you encounter errors, perform the following tasks: Desk-check the above preparatory steps. Read the output messages and logs. Research any error messages using the Integrated Solutions Console, IBMLink, and Google. Turn on tracing. Call for help. Creating DB2 databases and tables Before you start, make certain you have good database naming conventions for your DB2 objects. See 2.4.2, “Planning DB2 names” on page 77. Before you generate the SQL or DDL statements, understand the different ways you can create them. See 2.4.6, “Deciding how you will create and execute the DDL” on page 93. Using createDB.sh scripts, DButility.sh, or others such as bpeconfig.jacl, config_event_database.sh (for CEI), sibDDLGenerator.sh. Using a single or multiple databases, schema prefixes, or SQLIDs. Before you execute the SQL or DDL statements, understand the different ways you can run them (see page 2.4.6, “Deciding how you will create and execute the DDL” on page 93.) Using createDB.sh scripts, or convert them to EBCDIC and copy them to z/OS files and run with DSNTEP2. Verify proper user identities and GRANTs. Verify any environmental settings, such as STEPLIBs, $WAS_HOME variables. After you run the scripts perform the following tasks: Check the return code: Use the following command to look for negative SQLCodes: find ‘code = -’ Use the DB2 Administrative panels to check the number of tables and table spaces. See “Using ADBL: The DB2 administrator’s friend” on page 583. Chapter 10. Problem diagnosis and prevention 591 If you encounter errors, perform the following tasks: Desk-check the above preparatory steps Read the output messages and logs. Check to see if you have enough room on your DASD volumes for the table spaces. See “DASD space used by your table spaces” on page 584. See the section 10.4.6, “DB2 problems” on page 603. If necessary, research any error messages using the Integrated Solutions Console, IBMLink, and Google. If still necessary, turn on tracing. Call your DBA, or resort to desperate measures. Starting up servers Before you start the servers, make certain that you have prepared all the executables: Know the exact syntax of the Start command. Verify proper user identities (and passwords). Verify any environmental settings, such as STEPLIBs. After the servers are started: Check the return codes (make sure they stay up). Browse the JES logs for the controller, adjunct, and servant started tasks: – Issue the FIND SEVERE ALL command in SDSF – Look for warnings with “find warning all” – Look for Java exceptions: “find exception: all” If there are ffdc logs, you can examine them, but our experience is that they seldom hold more useful information than the JES logs (SYSOUT, SYSPRINT, and the JES JOBLOG.). Take a look at SYSLOG. Some error messages are easier to find there. If you encounter errors, perform the following tasks: Desk-check the above preparatory steps. Study the output messages and logs. If necessary, research any error messages using the Integrated Solutions Console, IBMLink, and Google. If still necessary, turn on tracing. Call for help. 592 z/OS: WebSphere Business Process Management V6.2 Production Topologies Testing the runtime with sample applications You must deploy the application in the runtime, and start it. If you have problems, follow these steps: 1. Before you run the test, make sure the servers are started, and check the logs for any SEVERE error messages. 2. Run your tests and perform the following tasks: – Check the return messages in your browser. – Browse the JES logs for the servant started task: • • • • See what messages occurred since you ran the test Issue the FIND SEVERE ALL command in SDSF. Look for warnings with find warning all. Look for Java exceptions: find exception: all. – Take a quick look at SYSLOG, for some error messages are easier to find there. 3. If you encounter errors, perform the following tasks: – Desk-check the above preparatory steps. – Study the output messages and logs. – If there are ffdc logs, you can examine them, but our experience is that they seldom hold more useful information than the JES logs (SYSOUT, SYSPRINT, and JES JOBLOG). – If necessary, research any error messages using the Integrated Solutions Console, IBMLink, and Google. – If still necessary, turn on tracing. – Call for help. 10.4.2 Top ten problems There are a few problems we encountered (besides the usual typing, spelling, and not following directions) that are listed here. Configuration Several tasks during configuration seem to be frequent stumbling blocks: Reading, following directions, spelling, typing, and clicking STEPLIBs for WebSphere Application Server for z/OS and DB2 GRANTs and DB2 privileges EJBROLEs Symbolic Links UNIX file permissions and ownership Chapter 10. Problem diagnosis and prevention 593 SQLID versus Schema DASD space Virtual and real storage Known APARs Here are some detailed problems we encountered along with the solutions we used. Spreadsheet and zPMT errors We used the Excel spreadsheet to create response files for the zPMT tool when configuring the servers. Here are some manual corrections we had to make in the zPMT. Administrator’s user ID For the job to federate an empty node, this showed up as WSADMIN in the zPMT and the generated job stream instead of the correct administrator’s user ID (MDADMIN in our case.) Symbolic Link for WebSphere Application Server for z/OS smpe root The zPMT in ASTK has one variable to define the path to WebSphere install directory - zSmpePath. WCT has 3 variables that have to do with the definition of intermediate symbolic link and the path to the WebSphere install directory: zEnableIntermediateSymlink: true or false. zIntermediateSymlink: The intermediate symbolic link name to be used. zSmpePath: The real path to SBBOHFS no matter the setting of zEnableIntermediateSymlink. The default in the WCT zPMT is to define an intermediate symbolic link. If you use the V6.1 spreadsheet for input to WCT with just zSmpePath set to the value of your intermediate symbolic link, you get the same value for the symbolic link and the target directory (it ends up being a circular definition). Configuring WebSphere Application Server for z/OS The following sections detail errors we encountered while building the servers. BBODBRAK This job runs the RACF commands for the deployment manager’s certificates. See Example 10-16 on page 595. 594 z/OS: WebSphere Business Process Management V6.2 Production Topologies Example 10-16 Errors in the BBODBRAK job Generating certificates for WebSphere Deployment Manager 228 *-* "RACDCERT ID (MDACRU) GENCERT SUBJECTSDN(CN('wpsplex.itso.ibm.com') O" ,"('IBM') OU('MDCELL')) ... NOTAFTER(DATE(2010/12/31))" The certificate that you are creating has an incorrect date range. The certificate is added with NOTRUST status. We fixed it by changing the RACDCERT ID (MDACRU) GENCERT command to use an earlier date: NOTAFTER(DATE(2010/11/29)) Be careful when you generate new certificates and sign them with CA certificates that were created in the past. To avoid problems, do not use the same expiration date for your new certificate as the one specified in your CA certificate. Subtract a day from the expiration date of your CA certificate, and use that date for the expiration date of your new certificate. Otherwise, the new certificate's expiration date and time may be later than the CA certificate that signed it, which leads to your new certificate having the NOTRUST status in RACF, which leads to this message in WebSphere: ExtendedMessage: BBOO0220E: SSLC0008E: Unable to initialize SSL connection. Unauthorized access was denied or security settings have expired. Exception is javax.net.ssl.SSLHandshakeException: Client requested protocol Unknown 0.2 not enabled or not supported. See Techdoc paper PRS3584 Renewing expiring RACF certificates for WebSphere on z/OS for more information, and alternate solutions. BBOWWPFA job fails This job sets up the profiles in the runtime file system. We received the following error: INSTCONFFAILED: The profile could not be created. For more information, consult the /wasv61config/mdcell/mdnodea/AppServer/logs/manageprofiles. The only hit we found in IBMLink was forAPAR PK50623, which cited a circumvention of regenerating the customization jobs without a HTTP Server, and that worked. Missing STEPLIBs They are required in the servant’s Z=parm procedures for the application servers, and for the deployment manager’s servant if you do not have these libraries in link list or LPA. Chapter 10. Problem diagnosis and prevention 595 They are also required in the setupCmdLine.sh script in the {profile_root}/bin/ directory for each node, and for the deployment manager. We added STEPLIBs for the following libraries: WebSphere runtime libraries DB2 for z/OS runtime libraries MQ for z/OS runtime libraries Federation errors Here are a few errors we encountered when we tried to federate an empty node. BBOWMNAN job fails ADMU0027E: An error occurred during federation The following port properties were not recognized: {IPC_CONNECTOR_ADDRESS=}; rolling back to original configuration. . . ADMU0111E: Program exiting with error... This was probably caused by using the WCT V7.0.0.0 zPMT which added the IPC_CONNECTOR_ADDRESS= variable to the following target dataset member: <hlq>.<empty_node>.DATA(BBOWMNPF) Solution: This can usually be fixed by commenting out or deleting the line: #IPC_CONNECTOR_ADDRESS= NodeAgent wouldn't start This may happen if the BBOWMNAN job had been run earlier (unsuccessfully) with the incorrect user ID. When re-run with the correct (administrator's) ID, the federation succeeded, but the node agent failed to start because of bad permissions and ownerships in the configuration HFS. Solution: Usually fixed by running the commands shown in Example 10-17: Example 10-17 Running the bbowhfsb.sh and bbowhfsc.sh scripts cd /usr/lpp/zWebSphere/V6R1/bin ./bbowhfsb.sh /wasv61config/mdcell/mddmnode/DeploymentManager ./bbowhfsc.sh /wasv61config/mdcell/mddmnode/DeploymentManager mdadmin mdcfg ./bbowhfsb.sh /wasv61config/mdcell/mdnodea/AppServer ./bbowhfsc.sh /wasv61config/mdcell/mdnodea/AppServer mdadmin mdcfg Nodes do not appear the in Integrated Solutions Console After federation, sometimes the nodes will not appear in the Environment → Nodes pane of the Integrated Solutions Console. A common solution is to log out of the Integrated Solutions Console, and log back in. Sometimes recycling the deployment manager is required (Stop and restart it.). 596 z/OS: WebSphere Business Process Management V6.2 Production Topologies Clustering servers When an unclustered server is added to a cluster, several resources are re-defined from the server scope to the cluster scope. In some cases, such as the messaging engines for the CEI bus, the data stores are re-defined with default values. Fixing the CEI messaging engine after cluster is created When the CEI messaging engine was moved to the cluster level, several attributes were lost: The Create Tables may be reset to their default values in the data store custom properties. Dynamic allocation of database objects is not allowed on z/OS. Use the Integrated Solutions Console, in the CEI bus ME datasource. Navigate to servers → server_name → Messaging engines → cei_bus_me_name → Data store. The schema names were reset to their default values in the data store custom properties (IBMWSSIB). So we logged into the Integrated Solutions Console, in the CEI bus ME datasource. Navigate to servers → server_name → Messaging engines → cei_bus_me_name → Data store. We set the correct values for Schema (MDSR1C) and cleared the Create tables check box. Authentication Alias on ME Datastore missing (Blank). Add it back. CEI bus destinations were lost: BPCCEIConsumerQueueDestination_mdnodea_mdsr01a_WPS BPCTransformerQueueDestination_mdnodea_mdsr01a_WPS Add these queue destinations back using the Integrated Solutions Console. If the CEI Bus cannot obtain a lock on its data source. (CWSIS1501E: ...because the persistence layer has been stopped.) Drop and re-create SIB Tables for CEI messaging engine to keep the database tables in sync with the data source. Note: Because we have other cells (M1CELL and M2CELL) handy to examine the settings, the process of resolving errors was much simpler. It proved to be valuable to have a back-up configuration to compare when things changed. Chapter 10. Problem diagnosis and prevention 597 10.4.3 Configuring WebSphere Process Server for z/OS Most of the problems we encountered while augmenting the various nodes (stand-alone server, empty nodes, deployment manager) occurred because of spelling and typing errors. It is important to double-check your response files, because errors due to the parameter values in the response files not matching the values used in WebSphere Application Server for z/OS and DB2 are hard to diagnose. It is much easier to desk-check your parameters, than to figure out the mistake from the symptoms in many cases. zSMPEInstall.sh log files If any errors occur while running the script, start by looking in the <server_root>/logs/wbi/install/installconfig.log. When you find the error, if you issue FIND ‘buildfile’ PREVIOUS, it will tell you the ant file that failed. This will direct you to one of the following logs to troubleshoot the problem. It is helpful to sort the logs by date to look at the most recent log file first by using the “ls -alt” UNIX command. In our configuration, <server_root>, this was as shown in Example 10-18. Example 10-18 Log files created by the zSMPInstall.sh script /wasv61config/mdcell/mdnodea/AppServer /<server_root>/logs/wbi/c2n.log /<server_root>/logs/wbi/zSMPInstall.log /<server_root>/logs/wbi/zSMPInstall.trace /<server_root>/logs/wbi/100SCleanOSGICache.ant.log /<server_root>/logs/wbi/15SCopyBSpaceIfNewer.ant.log /<server_root>/logs/wbi/80SCopyInstallValidatorLog.ant.log /<server_root>/logs/wbi/85SConfigNoProfileFirstStepsWBI.ant.log /<server_root>/logs/wbi/90SCleanDeployTool.ant.log /<server_root>/logs/wbi/90SConfigureWSProfileForWBI.ant.log /<server_root>/logs/wbi/90SConfigWBIMigrationScript.ant.log /<server_root>/logs/wbi/91SConfigNoProfileFirstStepsCharset.ant.log /<server_root>/logs/wbi/93SDeployBPCAdminConsolePlugins.ant.log /<server_root>/logs/wbi/93SDeployBusinessSpaceAdminConsoleFeature.log /<server_root>/logs/wbi/93SDeployServerAdminConsolePlugins.ant.log /<server_root>/logs/wbi/93SDeployWBICommonAdminConsolePlugins.ant.log /<server_root>/logs/wbi/94SDeployCoreAdminConsolePlugins.ant.log zWPSConfig.sh log files If any errors occur while running zWPSConfig.sh script, start by looking in the <server_root>/logs/wbi/zWPSConfig.trace file for a high-level view. 598 z/OS: WebSphere Business Process Management V6.2 Production Topologies Next, look in the <server_root>/logs/manageprofiles/default_augment.log file. When you find the error, if you issue FIND ‘buildfile’ PREVIOUS, it will tell you the ant file that failed. This will direct you to one of the following logs in the <server_root>/logs/manageprofiles/default/ directory, as shown in Example 10-19. Example 10-19 Log files created by zWPSConfig.sh (partial view) Feb Feb Feb Feb Feb Feb Feb Feb Feb Feb Feb Feb Feb Feb Feb 18 16 16 16 16 16 16 16 16 16 16 16 16 16 16 18:37 20:15 20:15 20:15 20:15 20:15 20:15 20:15 20:15 20:14 20:14 20:14 20:14 20:14 20:13 commonDBUtility.ant.log stopDatabaseServer.ant.log createBSpaceInstancePY.wsadmin.traceout.log createDefaultBSpaceInstance.log wsadminListener.log createBSpaceInstancePY.wsadmin.log configRESTDeploymentTargets.log configRESTDeploymentTargets_py.wsadmin.log configRESTDeploymentTargets_py.wsadmin.traceout.log aboutThisProfileDefault_wbiserver.ant.log configFirstStepsWBIDefault.ant.log createProfileShortCut2StartMenuDefaultWBIServer.ant.log updateNodeMetaData.ant.log configRelationship.ant.log configMediation.ant.log zSMPEInstall.sh failed RC=12 Problem: When running the zSMPEInstall.sh for the second node, it failed with a RC=12 and the messages shown in Example 10-20. Example 10-20 zSMPEInstall.sh failed RC=12 error message BPXBATCH SH cd /wasv61config/mdcell/mdnodeb/AppServer/bin; export PATH=.:$PATH; /wasv61config/mdcell/mdnodeb_wps_smpe/zos.config/bin/zSMPInstall.sh '-smproot /wasv61config/mdcell/mdnodeb_wps_smpe' '-runtime /wasv61config/mdcell/mdnodeb/AppServer' '-install' 1> /tmp/mdnodebinstall_0511.out 2> /tmp/mdnodebinstall_0511.err READY END IKJ56644I NO VALID TSO USERID, DEFAULT USER ATTRIBUTES USED READY BBOHFSWR '/tmp/mdnodebinstall_0511.out' ERROR: Unable to read file PATH. Error: 81 5620062 READY BBOHFSWR '/tmp/mdnodebinstall_0511.err' ERROR: Unable to read file PATH. Error: 81 5620062 READY Chapter 10. Problem diagnosis and prevention 599 Cause: The problem was caused by having an invalid UNIX environment for the user ID executing the job. The user ID (MDADMIN) did not have a home directory on the secondary LPAR because the BBOSBRAM job, which creates the home directory for MDADMIN, had only been run on the primary LPAR. Solution: Ensure the user ID running zSMPEInstall.sh has a valid UNIX environment by logging on to a telnet client using the user ID. Alternatively, use an OMVS session and try to su to that user ID. Echo $PATH, $CLASSPATH, and so forth, and try commands like java -fullversion to ensure that the user ID is working normally in UNIX on the LPAR where you are running the job. zWPSConfig.sh fails with Invalid profile template When we tried the zWPSConfig.sh, we got the following errors when trying to augment a standAlone Server: Example 10-21 Invalid profile template error Invalid profile template:The specified operation cannot proceed because /wasv61config/b6cell/b6nodea/AppServer/profileTemplates/default.wbicore is not a valid profile template. The zWPSConfig script was looking for the template for the Feature Pack for Web Services: <prerequisiteTemplate>/WEBSERVICES/default.wsfep</prerequisiteTemplate> We had not started with a server that was configured with the Feature Pack for Web Services, so we rebuilt the server with the FPWS, and started over. Failure in loading T2 native library db2jcct2zos Problem: Failure in loading T2 native library db2jcct2zos when initializing the servant or adjunct. This error can have several causes. Cause 1: The SDSNLOAD and SDSNLOD2 libraries are not present on the STEPLIB of the address space reporting the error. Solution 1: Add the SDSNLOAD and SDSNLOD2 to the STEPLIB. Cause 2: You are trying to use the Test Connection function on a Type 2 data source using the Integrated Solutions Console and the data source is defined at cluster level. Typically the error also includes the text shown in Example 10-22 on page 601. 600 z/OS: WebSphere Business Process Management V6.2 Production Topologies Example 10-22 Error text for second cause of failure in loading T2 native library db2jcct2zos reason: java.lang.UnsatisfiedLinkError: db2jcct2zos (Not found in java.library.path)DSRA0010E: SQL State = null, Error Code = -99,999 Solution 2: There is a limitation in WebSphere Application Server for z/OS that Test Connection will not work for data sources defined at node or cluster level. You will find that you can test the connection of the jdbc/WPSDB data source that is defined at cell level but not the other data sources defined at cluster level. -922 SQLCODEN922 connecting to DB2 Problem: The SQLException in Example 10-23 is issued when connecting to DB2. Example 10-23 SQLException issued when connecting to DB2 java.sql.SQLException: [IBM/DB2][T2zos/3.6.67]T2zosReusableConnection.flowConnect:execConnect: DB2 engine SQL error, SQLCODE = -922, SQLSTATE = 42505, error tokens = CONNECT;00D31024DSRA0010E: SQL State = 42505, Error Code = -922 Cause: The most common reason for this error is specifying an incorrect Location name in the Database name field of a Type 2 data source definition. This error is likely to happen because several of the wizards will set the database name in the Database name field. For a Type 2 data source, you need to set the DB2 Location name in the Database name field. Solution: Make sure that the Database name property on the data source is set to the DB2 z/OS Location name rather than the database name. -551 SQLCODEN551 connecting to DB2 Problem: When trying to start the BPC Event Collector and Observer applications, they failed to start giving -551 errors in the servant logs. The message in Example 10-24 appears in the servant log. Example 10-24 -551 SQLCODEN551 connecting to DB2 servant log message ExtendedMessage: CWWBO4005E: A failure occurred during the check of database catalog: com.ibm.db2.jcc.t2zos.cb: [IBM/DB2][T2zos/3.6.67] T2zosPreparedStatement.readPrepareDescribeOutput_: nativePrepareInto:1512: DB2 engine SQL error, SQLCODE = -551, SQLSTATE = 42501, error tokens = MDASRU;SELECT;SYSIBM.SYSTABLES. Chapter 10. Problem diagnosis and prevention 601 Cause: The data source called Business Process Choreographer data source was created without the schema being set on either currentSQLID or currentSchema. Solution: Set the currentSQLID property on the Business Process Choreographer Event Collector data source to the schema of the BPC Event Collector and Observer tables and restart the server. SECJ4062W: Cannot find the credential information Problem: The errors shown in Example 10-25 were seen in the adjunct logs just after running the BPC Configuration wizard. Example 10-25 Adjunct log errors Trace: 2008/08/24 21:34:31.698 01 t=7BF2D8 c=UNK key=P8 (13007002) ThreadId: 00000030 FunctionName: com.ibm.ws.security.auth.j2c.WSDefaultPrincipalMapping SourceId: com.ibm.ws.security.auth.j2c.WSDefaultPrincipalMapping Category: WARNING ExtendedMessage: BBOO0221W: SECJ4062W: Cannot find the credential information. Trace: 2008/08/24 21:34:31.698 01 t=7BF2D8 c=UNK key=P8 (0000000A) Description: Log Java Message Message: BBOO0221W: SECJ4062W: Cannot find the credential information. Trace: 2008/08/24 21:34:31.703 01 t=7BF2D8 c=UNK key=P8 (13007002) ThreadId: 00000030 FunctionName: com.ibm.ejs.j2c.PrivExAction SourceId: com.ibm.ejs.j2c.PrivExAction Category: SEVERE ExtendedMessage: BBOO0220E: J2CA0044E: The ConnectionManager failed to get a Subject from the security service associated with connection factory jdbc/com.ibm.ws.sib/mpcl01_WPS-BPC.mpcell.Bus. Received exception javax.security.auth.login.LoginException: Incorrect authDataEntry and alias is: BPCME_00_Auth_Alias Cause: The credential in question is the user ID and password in the J2C authentication alias called BPCME_00_Auth_Alias. At the time the error occurred, this alias had just been created by running the BPC configuration wizard but the cluster had not been restarted. RACF-related control blocks for the user IDs on J2C authentication aliases are built during server initialization so a restart is needed before they can be used. Solution: Restart the cluster. 602 z/OS: WebSphere Business Process Management V6.2 Production Topologies 10.4.4 Configuring WebSphere Business Services Fabric for z/OS There were several problems encountered when configuring WebSphere Business Services Fabric for z/OS. We found most of these errors while deploying the Fabric sample application. See 7.2, “Errors we encounters while testing the Fabric sample application” on page 427 for details. 10.4.5 Configuring Business Space powered by WebSphere Business Space powered by WebSphere data source currentSchema defaults to IBMBUSSP. Schema names In the Database schema name text box, enter the schema name you want to use for Business Space in DB2 for z/OS. We entered the default schema name of IBMBUSSP because in WebSphere Process Server for z/OS V6.1.2.0, Business Space powered by WebSphere did not use the schema name we entered. This problem is resolved in WebSphere Process Server for z/OS V6.1.2.1 and V6.2. Other Business Space problems Business Space powered by WebSphere uses many capabilities of both WebSphere Process Server for z/OS and WebSphere Business Services Fabric for z/OS. In particular, it makes extensive use of REST services. If you are getting unpredictable results in the widgets deployed in Business Space powered by WebSphere, make sure you followed the directions in 7.3.3, “Configure the Fabric Rest Services gateway” on page 435. There are also many EJBROLE definitions required for Business Space powered by WebSphere. Check for ICH408I messages in your SYSLOG. If you see any roles you do not recognize, refer to 7.3.3, “Configure the Fabric Rest Services gateway” on page 435. 10.4.6 DB2 problems Most of these problems occurred when we tried starting the servers, or start up the messaging engines. Chapter 10. Problem diagnosis and prevention 603 Schema problems (general) You should specify DB2 Version 8 in the createDB.sh script. Not all the components provide version 9 DDL yet. At this time, the schema for V8 and V9 is identical. The WPSDB (common datasource) should be the only datasource that needs the currentSQLID, and we have a PMR opened for this. All other datasources use a DataStore. They store the schema qualifier in the Integrated Solutions Console model and build fully qualified SQL statements. (<OWNER>.table_name.) You should not use both currentSchema and currentSQLID. Business Space schema The Business Space data source was hard-coded to have a Schema name of IBMBUSSP in some cases. See “Schema names” on page 603 and 9.3, “Run the Business Space Configuration wizard” on page 551. CEI ME schema The CEI Bus ME data store sometimes defaults to IBMWSSIB after a cluster is created. See “Fixing the CEI messaging engine after cluster is created” on page 597. Grants (SQLCODE -553 errors) To figure out the root of these problems, you can trace the SQL and see what user ID is being used in the set current sqlid, and what ID the SQL is trying to use to access DB2. Once you know what ID is being used and what objects are being accessed, you can then look in DB2 catalog to see if the ID has the authority to access that ID. Desperate approaches When all else fails, you can get beyond the immediate DB2 permission problem with some heavy-handed circumvention: GRANT ALL ON DATABASE <database> TO PUBLIC; GRANT SYSADM to the groups or user IDs attempting to access the objects (the authentication aliases.) This will get you around the problem, but is not acceptable in a production environment. See 2.4.4, “Using GRANT statements in DB2” on page 89 for more discussion on this subject. 604 z/OS: WebSphere Business Process Management V6.2 Production Topologies createDB.sh script problems We encountered several errors while running the createDB.sh script, and they were almost always caused by the sequence of running the script, or user errors. See 2.7.1, “Using the sample createDB.sh script” on page 123 Creating tables for Fabric before augmentation If you try to run the createDB.sh script with the following parameters before the node has been augmented for WebSphere Business Services Fabric for z/OS, and you specify the Fabric Definitions = true parameter in the createDB.sh script, you will get error messages such as those shown in Example 10-26. Example 10-26 createDB.sh scripts when creating tables for a component that has not been configured Configuring WBSF Definitions cat: fabric/define_db.sql: EDC5129I No such file or directory. cat: fabric/define_tables.sql: EDC5129I No such file or directory. cat: fabric/define_index_objects.sql: EDC5129I No such file or direct. cat: fabric/define_unique_index.sql: EDC5129I No such file or direct. cat: fabric/define_foreignkeys.sql: EDC5129I No such file or directory. cat: fabric/define_sequence_objects.sql: EDC5129I No such file or dir. "/SC42/var/WebSphere/home/MDCFG/wpswork/tmp/wbsf.ddl" +++++++++++++++++++++++++++ Configuring WBSF Definitions cat: wsrr/schedtab.sql: EDC5129I No such file or directory. cat: wsrr/xmetatab.sql: EDC5129I No such file or directory. cat: wsrr/tstortab.sql: EDC5129I No such file or directory. "/SC42/var/WebSphere/home/MDCFG/wpswork/tmp/wsrr.ddl" Specifying the DB2JccConfiguration.properties file location The following parameter must specify the directory where the DB2JccConfiguration.properties file is located, but not the file itself. If you specify the complete file name, such as DBJDBCProperties=/u/user1/wpswork/DB2JccConfiguration.properties, you will get the error shown in Example 10-27. Example 10-27 Error opening DB2JccConfiguration.properties file [sql] java.util.zip.ZipException: Error opening zip file /u/user1/wpswork/DB2JccConfiguration.properties If you code DBJDBCProperties=/u/user1/wpswork you will get no errors. Chapter 10. Problem diagnosis and prevention 605 Saving output from createDB.sh When diagnosing problems with createDB.sh, you can pipe the output to a file so that all the messages which get written to the window are not lost. Use the following format: createDB.sh -All +SIBDB > traceDb.out 2> traceDb.err Creating and dropping databases and tables See Appendix A, “Additional material” on page 615 for several examples in WASCFG.WPS62.SQL(DROPFUNC and DROPSEQ members. The following objects do not get dropped when the databases are dropped: Functions Sequences SQLCODE = -454 error - CREATE FUNCTIONs This will occur if you drop the BPC database and try re-creating the tables, because the five functions being defined in the BPC.sql file did not get dropped when the database was dropped. You can ignore these messages if the functions being defined are identical, or you can explicitly drop the functions when you drop the database. SQLCODE = -454 error - CREATE SEQUENCEs This will occur if you drop the WPS common database and try re-creating the tables, because the 14 sequences being defined in the WPS.sql file did not get dropped when the database was dropped. You can ignore these messages if the sequences being defined are identical, or you can explicitly drop the sequences when you drop the database. Dropping functions, sequences and table spaces Here are some sample drop statements for functions and sequences: DROP FUNCTION MDWPS.OBSVR_JAR_ACTIVENT (); DROP SEQUENCE SEQ_W_OBJ_LIT_DATE_ID; Sample DDL statements for these can be found in Appendix A, “Additional material” on page 615. Dropping functions: WASCFG.MDCELL.WPS.SQL(DROPFUNC) Dropping sequences: WASCFG.MDCELL.WPS.SQL(DROPSEQ) SIB ME TableSpaces: WASCFG.WPS.SQL(MD1ADROP, MD1BDROP, MD1CDROP, and MD1SDROP members.) 606 z/OS: WebSphere Business Process Management V6.2 Production Topologies 10.5 Runtime problems We found that most of the problems we encountered occurred when we were configuring the topology, or when the servers were first started after configuration. We did however encounter the problems detailed in the sections that follow. Excessive memory use with data source custom properties Several JDBC data sources are defined for WebSphere Process Server for z/OS and other stacked products such as Fabric, Business Spaces, or Business Monitor. If you are running in 31-bit mode, we recommend setting three custom properties to avoid failures to read the database properly on DB2 for z/OS V.9. Otherwise, excessive memory may be used for retrieving LOB data. This should be done to all BPM data sources that contain LOB data. After you successfully augment your configuration with WebSphere Process Server for z/OS and any other stacked products, you should set these properties. Log in to the Integrated Systems Console and set these custom properties for each data source. Table 10-1 Custom properties for DB2 Version 9 Name Value Description Type enableRowsetSupport 2 Do not use multiple-row FETCH java.lang.Integer progressiveStreaming 2 Do not use progressive streaming for LOB data. java.lang.Integer fullyMaterializeLobData false Use LOB locators to fetch data from LOB columns. java.lang.String To create these two new custom properties, complete the following steps: 1. Navigation to Resources→ JDBC→ Data sources. 2. Select each of the listed data sources. 3. From the Additional Properties section, click Custom Properties. 4. Click New. 5. Enter the values from the table into the following fields: – – – – Name Value Description Type Chapter 10. Problem diagnosis and prevention 607 6. Click OK. 7. Click Save. To modify the fullyMaterializeLobData custom property, complete the following steps: 1. Navigate to Resources→ JDBC→ Data sources. 2. Select each of the listed data sources. 3. From the Additional Properties section click Custom Properties. 4. Click the existing property fullyMaterializeLobData. 5. Change the property in the Value field to false. 6. Click OK. 7. Click Save. Figure 10-4 is a glimpse of the settings you should see. Figure 10-4 Data source custom properties (partial view) EJBROLE authorization Note: These are really configuration errors, but do not show up until you try to run an application, or open the BPC explorer. Many EJBROLE profiles are used in WebSphere Process Server for z/OS, Business Space powered by WebSphere, and WebSphere Business Services Fabric for z/OS. 608 z/OS: WebSphere Business Process Management V6.2 Production Topologies When troubleshooting an application or administrative function in your browser, you may not see the error message in your browser, but it might be in the system log. Using SDSF, browse the log, and look for ICH408I messages (as in Example 10-28) around the time of your problem. Example 10-28 RACF ICH408I message on console (syslog) ICH408I USER(MDFABADM) GROUP(MDFADMG ) NAME(MD FABRIC ADMIN ) MD.FabricStudioUser CL(EJBROLE ) INSUFFICIENT ACCESS AUTHORITY ACCESS INTENT(READ ) ACCESS ALLOWED(NONE ) +BBOO0222I: SECJ0129E: Authorization failed for MDFABADM while invoking GET on default_host:fabric/app, Authorization failed, Not granted any of the required roles: FabricAdministrator FabricGovernanceAdministrator FabricPerformanceUser FabricSubscriberManager FabricBasicUser FabricStudioUser To resolve this error, permit the user ID to one of the EJBROLE profiles. In this case the user ID MDFABADM, which we wanted to be the Fabric Administrator user ID, had not been granted to any of the roles. Refer to 5.2.7, “Verify the WebSphere Business Services Fabric for z/OS installation and configuration” on page 252. Adjunct errors - see APAR PK74286 When our server attempted to start messaging errors, we encountered the following message: CWSIS0002E: The messaging engine encountered an exception while starting. Exception: com.ibm.ws.sib.msgstore.PersistenceException: CWSIS1501E: The data source has produced an unexpected exception: java.lang.IllegalStateException: CWSIS1530E: The data type, 12, was found instead of the expected type, -1, for column, URI, in table, MD01A.SIBCLASSMAP. Circumvention: Add the following line to a sib.properties file in the properties subdirectory of all profiles where servers that host messaging engines run: sib.msgstore.jdbcPerformColumnChecks=false This disables column checking, allowing the messaging engine to start. Remove the symbolic link in /wasv61config/mdcell/mdnodeb/AppServer/properties/sib.properties, and replace it with a real file with the above line in it. Chapter 10. Problem diagnosis and prevention 609 CWSIS1501E - msgstore.PersistenceException After restoring the configuration to a previous state, we received the message shown in Example 10-29 when starting up the adjunct. Example 10-29 Message received after restoring configuration to previous state CWSIS0002E: The messaging engine encountered an exception while starting. Exception: com.ibm.ws.sib.msgstore.PersistenceException: CWSIS1501E: The data source has produced an unexpected exception: com.ibm.ws.sib.msgstore.persistence.DatasourceWrapperStateException: New connections cannot be provided because the persistence layer has been stopped To fix this, drop the tables (or TABLESPACEs which is easier), and re-create the tables. CWSIV0757E - Destination on CEI bus not found When starting the server after making it into a cluster member, we encountered Example 10-30. Example 10-30 CWSIV0757E: The destination BPCTransformerQueueDestination_mdnodea_mdsr01a_WPS on bus CommonEvent Infrastructure_Bus was not found. was thrown when processing the startup of messaging engine mdsr01.000-CommonEventInfrastructure_Bus To fix this issue, see “Fixing the CEI messaging engine after cluster is created” on page 597. Daemon ABEND RC=8004 - RRS problems The Daemon ABENDed with the following message: BBOT0004E RRS SERVICE 'CRGSEIF' FAILED WITH RETURN CODE=8004. We issued the D RRS,RM command, and went into the ISPF RRS panels and unregistered them, but when we restarted the server, it locked up again, as you can see by the D GRS,C command response in Example 10-31. Example 10-31 Commands to investigate RRS problems D RRS,RM BBO.MDBASEA.MDBASEA.MDDEMNA.IBM UnsetInProgress BBO.MDBASEA.MDSR1.MDSR1A.IBM UnsetInProgress D GRS,C showed this: ISG343I 21.06.55 GRS STATUS 298 610 SC42 SC42 z/OS: WebSphere Business Process Management V6.2 Production Topologies WTSCPLX1 WTSCPLX1 S=SYSTEMS SYSZATR WTSCPLX1-RESTART SYSNAME JOBNAME ASID TCBADDR EXC/SHR SC50 RRS 002C 007DEE88 EXCLUSIVE SC42 RRS 002C 007DE4E0 EXCLUSIVE SC42 RRS 002C 007DE788 EXCLUSIVE SC42 RRS 002C 007DC178 SHARE NO REQUESTS PENDING FOR ISGLOCK STRUCTURE NO LATCH CONTENTION EXISTS STATUS OWN WAIT WAIT WAIT It turned out that DFHSM started dumping the pack that contained a RSS RESTART dataset. We suspect it ENQed on the VTOC, which caused RRS to have a problem opening the dataset. Later on we replied RETRY to the outstanding reply and saw that the UNSET completed. The D GRS,C command showed no contention. Chapter 10. Problem diagnosis and prevention 611 612 z/OS: WebSphere Business Process Management V6.2 Production Topologies Part 5 Part 5 Appendixes © Copyright IBM Corp. 2009. All rights reserved. 613 614 z/OS: WebSphere Business Process Management V6.2 Production Topologies A Appendix A. Additional material This book refers to additional material that can be downloaded from the Internet as described in the sections that follow. © Copyright IBM Corp. 2009. All rights reserved. 615 Locating the Web material The Web material associated with this book is available in softcopy on the Internet from the IBM Redbooks Web server. Point your Web browser at: ftp://www.redbooks.ibm.com/redbooks/SG247733 Alternatively, you can go to the IBM Redbooks Web site at: ibm.com/redbooks Select the Additional materials and open the directory that corresponds with the IBM Redbooks form number, SG24-7733. Using the Web material The additional Web material consists of the following file: File name SG247703.zip Description Additional material How to use the Web material Create a subdirectory (folder) on your workstation, and download the SG247773.zip file in BINARY. It is about 2000 KB. The zip file contains the Web material organized into the following directories. Unzip the ones you want into these folders: SampleWpsHtmApps Scenarios – FABRIC – WPS zos_wbm – WBM.CNTL – WBM.SQL zos_wbsf – WBSF.CNTL – WBSF.SQL zos_wps – WPS.CNTL – WPS.SQL zos_LDAP_config 616 z/OS: WebSphere Business Process Management V6.2 Production Topologies zos_planning_spreadsheet zos_wpswork zPMT_ResponseFiles The contents of each of these directories are now described in more detail. SampleWpsHtmApps The SampleWpsHtmApps directory contains three application ear files, and a read_me file: Echo_Ht_WpsApps_ReadMe.txt wpsecho-nocei-20080909.ear WPSEcho.ear WscHtTest.ear These are used in the end of chapter 2. Scenarios The Scenarios directory contains applications used in the scanarios of this Redbooks publication, and has two sub-directories: FABRIC: Has 2 sub-sub directories: – EAR – FCA WPS: Has 3 sub-sub directories: – EAR - has 2 sub-sub-sub directories: • v6.2_HumanTask • v6.2_Plain – HTM – PI zos_wbm The zos_wbm directory contains the following files, which relate to configuring WebSphere Business Monitor: WBM.CNTL.XMIT, WBM.SQL.XMIT These two files contains all the sample JCL, DDL, and shell scripts that are referred to in Chapter 8, “Monitoring the Business Process Management environment” on page 465. Appendix A. Additional material 617 The files are partitioned datasets that have been transmitted into a sequential dataset format using the TSO XMIT command. We explain how to upload these files to your z/OS system in “Installing the sample JCL and DDL on a z/OS system” on page 622. The WBM.CNTL.XMIT file is the XMITed contents of the dataset WASCFG.MDCELL.WBM.CNTL. This contains all the sample JCL referred to in this Redbooks publication. The WBM.SQL.XMIT file is the XMITed contents of the dataset WASCFG.MDCELL.WBM.SQL. This contains all the DDL that we generated and customized when configuring our MD cell. zos_wps The zos_wps directory contains the following files, which relate to configuring WebSphere Process Server for z/OS: WPS.CNTL.XMIT, WPS.SQL.XMIT These two files contains all the sample JCL, DDL, and shell scripts that are referred to in Chapter 2, “Configuring a WebSphere Process Server for z/OS standalone server” on page 39. The files are partitioned datasets that have been transmitted into a sequential dataset format using the TSO XMIT command. We explain how to upload these files to your z/OS system in “Installing the sample JCL and DDL on a z/OS system” on page 622. The WPS.CNTL.XMIT file is the XMITed contents of the dataset WASCFG.MDCELL.WPS.CNTL. This contains all the sample JCL referred to in this Redbooks publication. The WPS.SQL.XMIT file is the XMITed contents of the dataset WASCFG.MDCELL.WPS.SQL. This contains all the DDL that we generated and customized when configuring our MD cell. zos_wbsf The zos_wbsf directory contains the following files, which relate to configuring WebSphere Business Services Fabric for z/OS: WBSF.CNTL.XMIT, WBSF.SQL.XMIT 618 z/OS: WebSphere Business Process Management V6.2 Production Topologies These three files contains all the sample JCL, DDL and shell scripts that are referred to in Chapter 5, “WebSphere Business Services Fabric for z/OS” on page 201. We explain how to upload these files to your z/OS system in “Installing the sample JCL and DDL on a z/OS system” on page 622. The WBSF.CNTL.XMIT file is the XMITed contents of the dataset WASCFG.MDCELL.WBSF.CNTL. This contains all the sample JCL we created when adding Fabric to our MD cell. The WBSF.SQL.XMIT file is the XMITed contents of the dataset WASCFG.MDCELL.WBSF.SQL. This contains all the DDL that we generated and customized when adding Fabric to our MD cell. zos_LDAP_config The zos_LDAP_config directory contains the following files: zos_MD_itso.ldif zos_MD_itso_wbsf.ldif You only need to consider using the file zos_MD_itso.ldif if you want your cell to use an LDAP user registry. This ldif file contains sample commands that could be used to update Tivoli Directory Server for z/OS (TDS z/OS) with the user IDs and groups required to support WebSphere and WPS when the cell is using TDS z/OS as a user registry. These commands are identical to those on the WPS_LDAP worksheet of the spreadsheet provided in the zos_planning_spreadsheet directory. Usage of the spreadsheet is described in 2.2.2, “Using a spreadsheet to plan names for configuration objects” on page 45. Choosing user registries is discussed in 2.3.7, “Planning security” on page 67. Configuring a LDAP user registry is discussed in 5.3.4, “Configure LDAP security” on page 280. The file zos_MD_itso_wbsf.ldif is needed if you install WebSphere Business Services Fabric for z/OS. Even if your cell is using a SAF-based user registry like RACF you will need to use an LDAP server with Fabric. Although Fabric can use RACF for authentication and J2EE role checking, it needs to use an LDAP user registry for various application functions. Therefore, you will need to create an LDAP server Appendix A. Additional material 619 and load entries similar to those in file zos_MD_itso_wbsf.ldif. This is described in 7.1.5, “Create an external LDAP federation project” on page 409. zos_planning_spreadsheet The zos_planning_spreadsheet directory contains the MD - zPMT WPS V6.2 Configuration.xls files. The MD - zPMT WPS V6.2 Configuration.xls file is the spreadsheet we used to plan our single-cluster topology MD cell in 2.2.2, “Using a spreadsheet to plan names for configuration objects” on page 45. You could use the same spreadsheet to plan topology with two clusters. We did not supply the spreadsheet we used for our two-cluster topology cell, MP cell, because there is no difference between the two spreadsheets apart from the cell prefix (MP instead of MD) and the port ranges. Notes: We do not expect you actually to use the DDL we provide in the additional material. It is better to generate your own using the procedures described in this Redbooks publication. Our DDL is provided as a reference only. Some of the JCL members in WPS.CNTL and WBSF.CNTL have the same names, so be sure to unload the XMIT datasets to different datasets. zos_wpswork The zos_wpswork directory contains the sample shell scripts that we mention in this Redbooks publication. There are also the response files we used for configuring our MD cell. The zos_wpswork directory contains shells relating to both WebSphere Process Server for z/OS and WebSphere Business Services Fabric. copyfabric.sh Copies required Fabric files from the Fabric install directory to the work directories for the deployment manager and one or more nodes. MD62_createDB.sh A copy of our modified createDB.sh script that we used in generating the DDL for our DB2 tables for WebSphere Process Server for z/OS components cvtA2E.sh A shell script to convert an ASCII file to EBCDIC 620 z/OS: WebSphere Business Process Management V6.2 Production Topologies cvtE2A.sh A shell script to convert an EBCDIC file to ASCII. mdbaseaStandaloneDB2.rsp Response file to augment the standalone server’s node with WebSphere Process Server for z/OS. mdDmgrDB2.rsp Response file to augment the deployment manager’s node with WebSphere Process Server for z/OS. mdnodeaManagedDB2.rsp Response file to augment the empty nodea with WebSphere Process Server for z/OS. mdnodebManagedDB2.rsp Response file to augment the empty nodeb with WebSphere Process Server for z/OS. MyLDAPTransformation.xsl updNewServer.py Jython script to update the server’s port numbers and short names. Based on Techdoc TD104066. viascii A shell script to edit or view ASCII files. We explain how to upload these files to your z/OS system in “Installing the sample JCL and DDL on a z/OS system” on page 622. zPMT_ResponseFiles This directory contains copies of our response files used as input to the zPMT: MD_baseaStandAlone.txt MD_DMGR.txt MD_FederateNodeA.txt MD_EmptyNodeB.txt Appendix A. Additional material 621 Installing the sample JCL and DDL on a z/OS system To install the sample JCL, DDL and shell scripts, perform the following steps: 1. Allocate four sequential datasets on your z/OS system with suitable dataset names. For example, hlq.WPS.CNTL.XMIT. The dataset format is RECFM=FB,LRECL=80,BLKSIZE=3120. You could use JCL like that shown in Example 10-32. Example 10-32 Sample JCL to allocate the .XMIT datasets for additional material //MDALLOC JOB NOTIFY=&SYSUID,CLASS=A,MSGCLASS=A,REGION=0M //* //* ALLOCATE FILES TO RECEIVE XMIT DATASETS //* //ALLOC PROC DSN=,PRI=4 //IEFBR14 EXEC PGM=IEFBR14 //ALLOC DD DSN=&DSN, // DISP=(NEW,CATLG), // UNIT=3390,SPACE=(TRK,(&PRI.,5),RLSE), // DCB=(RECFM=FB,LRECL=80,BLKSIZE=3120) // PEND //* //S1 EXEC ALLOC,DSN=WASCFG.MDCELL.WPS.CNTL.XMIT,PRI=8 //S2 EXEC ALLOC,DSN=WASCFG.MDCELL.WPS.SQL.XMIT,PRI=53 //S4 EXEC ALLOC,DSN=WASCFG.MDCELL.WBSF.CNTL.XMIT,PRI=3 //S5 EXEC ALLOC,DSN=WASCFG.MDCELL.WBSF.SQL.XMIT,PRI=4 2. Transfer the .XMIT files to your z/OS system in BINARY mode, choosing dataset names on the transfer that ensures the .XMIT files go into the datasets you just allocated. 3. On TSO, use ISPF option 3.4 to list the .XMIT datasets. 4. Next to each dataset, type the following line command: receive indsn(/) Example 10-33 on page 623 shows the command being used to receive the WBSF.CNTL dataset. Execute the same command against each of the datasets listed in the example. 622 z/OS: WebSphere Business Process Management V6.2 Production Topologies Example 10-33 Receiving the XMIT files using the TSO RECEIVE command DSLIST - Data Sets Matching WASCFG.MDCELL.*.*.XMIT Command ===> Command - Enter "/" to select action -------------------------------------------------receive indsn(/)MDCELL.WBSF.CNTL.XMIT WASCFG.MDCELL.WBSF.SQL.XMIT WASCFG.MDCELL.WPS.CNTL.XMIT WASCFG.MDCELL.WPS.SQL.XMIT You will receive the following prompt: Dataset WASCFG.MDCELL.WPS.CNTL from SENIOKJ on WTSCPLX1 Enter restore parameters or 'DELETE' or 'END' + 5. Press enter to allocate a dataset under your user ID with a similar name. For example, if you are logged on as FRED and you press enter in response to the prompt, the sequential XMIT dataset is expanded into a partitioned dataset called ‘FRED.MDCELL.WPS..CNTL’. If you want to specify a completely different name, enter the DSN(...) option with the dataset you want to create, and press enter. For example: DSN(‘WASCFG.MDCELL.WPS.CNTL’) Installing the sample shell scripts on a z/OS system Perform the following steps to install the sample shell scripts on a z/OS system: 1. Create a work directory in the UNIX file system under your user ID. For example, if you are logged on as FRED, create /u/fred/wpswork, where /u/fred is your OMVS home directory. Note: In our examples, the wpswork diectory is located at /var/WebSphere/home/MDCFG/wpswork. We refer to this location throughout this book. 2. Transfer the contents of the zos_wpswork directory in the additional material from your workstation into a ../wpswork directory using BINARY mode. 3. Change the permission on any files in the wpswork directory to ensure that they include the execution bit. For example: > cd /u/fred/wpswork > chmod 755 * Appendix A. Additional material 623 624 z/OS: WebSphere Business Process Management V6.2 Production Topologies Abbreviations and acronyms AST Application Server Toolkit TDS Tivoli Directory Server BPC Business Process Choreographer USS UNIX System Services WCT BRM Business rules manager Workstation Customization Tool CBEs Common Base Events WPS WebSphere Process Server CEI Common Event Infrastructure WSC Washington Systems Center CMP container managed persistence zAAPs z Application Assist Processors DBA Database Administrator zPMT zProfile Management Tool DDF Distributed Data Facility server DDL Data Definition Language DFS Distributed File Service DMZ De-Militiarized Zone FPWS Feature Pack for Web Services HA high availability HFS Hierarchical File System ISS Integrated Security Services KPIs key performance indicators LB load balancing LNA LDAP Native Authentication LPA Link Pack Area LPAR logical partition ND Network Deployment NFS Network File System ODR On-Demand Router PSP Preventative Service Planning RRS Resource Recovery Services SCRT Sub-capacity reporting tool SMB Server Message Block SMF System Management Facility SOA Service-Oriented Architecture © Copyright IBM Corp. 2009. All rights reserved. 625 626 z/OS: WebSphere Business Process Management V6.2 Production Topologies Related publications The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this book. IBM Redbooks For information about ordering these publications, see “How to get Redbooks” on page 631. Note that some of the documents referenced here may be available in softcopy only. WebSphere Business Process Management V6.2 Production Topologies, SG24-7732 z/OS: WebSphere Business Process Management V6.1.2 Production Topologies, SG24-7703 Getting Started with IBM WebSphere Business Services Fabric V6.1, SG24-7614 Getting Started with IBM WebSphere Process Server and IBM WebSphere Enterprise Service Bus Part 1: Development, SG24-7608 Getting Started with IBM WebSphere Process Server and IBM WebSphere Enterprise Service Bus Part 2: Scenario, SG24-7642 Getting Started with IBM WebSphere Process Server and IBM WebSphere Enterprise Service Bus Part 3: Run time, SG24-7643 Security in WebSphere Application Server V6.1 and J2EE 1.4 on z/OS, SG24-7384 WebSphere Process Server for z/OS: Configuring a Network Deployment Environment, REDP-4388. (This paper discusses WebSphere Process Server for z/OS V6.0.2.) zSeries Application Assist Processor (zAAP) Implementation SG24-6386 z/OS Getting Started: WebSphere Process Server & WebSphere ESB V6 SG24-7378 © Copyright IBM Corp. 2009. All rights reserved. 627 Other publications These publications are also relevant as further information sources: Techdocs The Techdocs site at http://www.ibm.com/support/techdocs contains a rich collection of documents about WebSphere Application Server for z/OS. In particular, we found these resources valuable: White papers WP100312 Use of WebSphere for z/OS with Sysplex Distributor WP100396 Planning for Test, Production and Maintenance WP100424 WebSphere V5 for z/OS JMS and MDB IVP WP100431 zAAP Projection Tool Instrumented SDK in WAS for z/OS V5 WP100570 Building a Quick and Simple WebSphere for z/OS V6 Runtime WP100653 WebSphere z/OS V6 -- WSC Sample ND Configuration WP100833 WPS for z/OS Datasource & Database Configuration Hints & Tips WP100871 Introducing the zPMT Configuration Tool for WebSphere z/OS WP100878 WPS/WESB v6.0.1 for z/OS: ND Configuration Lab WP100920 WAS for z/OS V6.1.0.4 - 64 Bit Addressing Support WP100963 Using Jython Scripting Language With WSADMIN WP100999 Using zPMT and Spreadsheet to Build Quick Standalone WP101014 WebSphere z/OS V6.1 - WSADMIN Primer (with Jython) WP101030 WebSphere z/OS V6.1 - A Top Down Configuration Approach WP101084 WebSphere for z/OS - Feature Pack for Web Services WP101121 The 64-bit Effect Five Different Ways to Look at Applications WP101138 WebSphere z/OS V6.1 - Hidden Gems and Little Known Features WP101140 WebSphere z/OS V6.1 - Making the DMGR Mobile WP101157 z/OS Large Memory Performance Studies WP101170 IBM HTTP Server V6.1 - The IBM HTTP Server for z/OS Powered by Apache WP101206 Installing ITCAM V6.1 on WebSphere for z/OS 628 z/OS: WebSphere Business Process Management V6.2 Production Topologies WP101209 WPS/WESB for z/OS V6.1: Network Deployment Configuration Lab WP101218 Performing Installation Verification for WPS on z/OS V6.1 WP101225 Extending the IHttpServer for z/OS Powered by Apache w/ Custom modules WP101233 Configuration Options for Handling Application Dispatch Timeouts WP101249 Configuring and Clustering IBM WebSphere Portal Enable for z/OS WP101253 WPS 'Easy' Network Deployment Configuration WP101300 The Mixed Platform Stack Project: Deploying a secure SOA solution into z/OS Presentations and downloads PRS1331 WebSphere for z/OS V6 - Configuration Planning Spreadsheet PRS1558 WAS z/OS: Updating Config. HFS to Use Intermediate Symbolic Links PRS2520 WPS and WESB Ver. 6.0.x z/OS Configuration Tools and Samples PRS2537 A Practical Introduction to Web Services, SOA and ESB on z/OS PRS2663 WebSphere z/OS - Comparing Front End HTTP Options PRS2853 Avoiding Potholes on the WebSphere Application Server On Ramp PRS3317 Workload Manager Configuration & Advanced Topics for WebSphere PRS3357 WebSphere Application Server z/OS V7.0 - Introducing the WCT PRS3421 WebSphere Process Server for z/OS V6.1 Handouts for ZWPS6 Wildfire Workshop PRS3584 Renewing expiring RACF certificates for WebSphere on z/OS Hints and tips TD100589 Using SDSF for WebSphere for z/OS Operations TD101087 Directing SYSPRINT Output to an HFS File in WebSphere for z/OS TD101115 RACF Tools for WebSphere for z/OS Ver.5 TD101975 What does "Test Connection" do in WAS z/OS AdminConsole? TD102662 Overcoming Temp Space Shortage When Installing Large Apps TD102724 Navigating the IBM Web in search of WebSphere Appl. Server z/OS TD103695 Managing Operator Message Routing in WebSphere z/OS Servers TD103863 Managing LE Options in WebSphere for z/OS Servers TD104066 Creating new Application Servers in WAS V6.1 for z/OS Related publications 629 TD104172 WSC Guidelines for a Healthy WebSphere Runtime on z/OS Online resources These Web sites are also relevant as further information sources: (These are the most valuable, and sometimes hard to find.) WPS V.6.2 for z/OS Software Support http://www.ibm.com/software/integration/wps/sysreqs http://www.ibm.com/support/docview.wss?rs=2307&uid=swg27013442 Program Directory ftp://ftp.software.ibm.com/software/websphere/integration/wps/library/p dfz620/program_directory_620.pdf PSP (Preventive Service Planning) Buckets: https://techsupport.services.ibm.com/server/390.psp390 Upgrade WPSZ Subset HWPS620 Information Centers WebSphere Process Server for z/OS V 6.2 Information Center: http://publib.boulder.ibm.com/infocenter/dmndhelp/v6r2mx WebSphere Application Server for z/OS V 6.1 Information Center: http://publib.boulder.ibm.com/infocenter/wasinfo/v6r1 DB2 V.8 Information Center: http://publib.boulder.ibm.com/infocenter/db2luw/v8/index.jsp IBM Education Assistant http://www.ibm.com/software/info/education/assistant/ Look under WebSphere Business Process Management - WebSphere Process Server for z/OS V6.1 630 z/OS: WebSphere Business Process Management V6.2 Production Topologies WebSphere Process Server for z/OS PDF Files These books are assembled from the Information Center by general topic areas so they can be viewed or printed as a coherent document. http://www.ibm.com/software/integration/wps/library/infocenter/doc/ 1. 2. 3. 4. 5. Click the PDF title you want Click the 6.1 folder Click the zos folder Highlight english or japanese pdf file Download PDF, or view in your browser. How to get Redbooks You can search for, view, or download Redbooks, Redpapers, Technotes, draft publications and Additional materials, as well as order hardcopy Redbooks, at this Web site: ibm.com/redbooks Help from IBM IBM Support and downloads ibm.com/support IBM Global Services ibm.com/services Related publications 631 632 z/OS: WebSphere Business Process Management V6.2 Production Topologies z/OS: WebSphere Business Process Management V6.2 Production Topologies (1.0” spine) 0.875”<->1.498” 460 <-> 788 pages Back cover ® z/OS: WebSphere Business Process Management V6.2 Production Topologies Creating production topologies for WebSphere Process Server Incorporating WebSphere Business Services Fabric Integrating WebSphere Business Monitor This IBM Redbooks publication addresses the configuration, administration, and security of the key runtime environments in business process management: WebSphere Process Server V6.2 and WebSphere Business Services Fabric V6.2 for the z/OS platform, and WebSphere Business Monitor V6.2 integration to the z/OS platform. Part 1 of this IBM Redbooks publication introduces production topology concepts and terminology. In Part 2, through a series of step-by-step instructions you will learn how to create a production topology environment for WebSphere Process Server for z/OS. Part 3 extends the production topology for WebSphere Process Server, describing step-by-step how to add WebSphere Business Services Fabric for z/OS into the topology. This section also addresses integration with WebSphere Business Monitor, running on a z/Linux partition for monitoring business process management solutions. Additionally this section shows how Business Space powered by WebSphere can provide an integrated user experience for this topology. Part 4 describes problem diagnosis and prevention guidance that will prove invaluable in creating your own production topologies. ® INTERNATIONAL TECHNICAL SUPPORT ORGANIZATION BUILDING TECHNICAL INFORMATION BASED ON PRACTICAL EXPERIENCE IBM Redbooks are developed by the IBM International Technical Support Organization. Experts from IBM, Customers and Partners from around the world create timely technical information based on realistic scenarios. Specific recommendations are provided to help you implement IT solutions more effectively in your environment. For more information: ibm.com/redbooks SG24-7733-00 0738432997