Creating your Sizing Guide at the
iSeries Benchmark Center
....getting started
Helen Olson-Williams
Richard Smitherman
Howard Sykes
Cindy Mestad
Mike Meyers
Terry Ford
April 4, 2004
PROCESS OVERVIEW
3
1.0
DOCUMENTATION
3
2.0
PLANNING SESSION CALLS
4
2.1
2.2
2.3
INTRODUCTION TO WORKLOAD SOLUTION OFFERING
PLANNING CALLS – IN DEPTH PLANNING AND REVIEW
FINAL CALL - READINESS REVIEW
4
5
5
3.0
END-USER QUESTIONS
6
4.0
PREPARATION: SPREADSHEETS AND QUESTIONNAIRES
8
4.1
4.2
4.3
4.4
4.5
4.6
4.7
4.8
4.9
4.10
4.11
4.12
WORKLOAD DEFINITION TEMPLATE
DETAILED USER GROUP SCENARIOS - PAPER SCRIPTS
RUN MATRIX TEMPLATE
HARDWARE/SOFTWARE QUESTIONNAIRE
SYSTEM CUSTOMIZATION, APPLICATION INSTALL PROCEDURE
DATABASE QUESTIONNAIRE
RUN PROCEDURES
WORKLOAD SIMULATION QUESTIONNAIRE
CPW/CIW REQUIREMENTS TEMPLATE
MEMORY REQUIREMENTS TEMPLATE
DISK REQUIREMENTS TEMPLATE
DAILY ACTIVITIES WORKPLAN
9
9
10
13
15
16
17
18
19
19
19
20
5.0
CREATE AND TEST THE WORKLOAD
22
6.0
SAVE THE WORKLOAD AS XML AND HTML
22
7.0
VALIDATION
23
APPENDIX A – EXAMPLE OF PAPER SCRIPTS
24
APPENDIX B – LEGAL DOCUMENTS REQUIRED
30
CUSTOMER COPYRIGHT RELEASE (CUSTOMER OWNS THE COPYRIGHT)
CUSTOMER THIRD-PARTY COPYRIGHT RELEASE (CUSTOMER IS A LICENSED USER)
INADVERTENT DISCLOSURE LETTER
INTERNET USAGE AGREEMENT
REMOTE USAGE AGREEMENT
30
31
32
33
36
2
Process Overview








Submit a Benchmark Nomination, http://www.ibm.com/eserver/iseries/developer/cbc
Read the documentation provided.
Participate in planning conference calls with the iSeries Benchmark Center.
Decide on the end-user questions.
Fill out the planning spreadsheet templates and questionnaires provided by the iSeries Benchmark
Center. Provide paper scripts of Business Transactions.
Run tests, gather and analyze performance data at the iSeries Benchmark Center.
Create the Workload Solution for your application at the iSeries Benchmark Center, using the Workload
Developer.
Have your solution approved in the IBM Global Solutions Directory (GSD) and enrolled in IBM eServer
Solution Connection (eSC).
The Workload Solution that you create at the iSeries Benchmark Center will be the foundation for your
sizing tool. In it you will specify the end-user questions and the calculations that will produce the
recommended servers. Your customers and end users can link to your web site to execute the IBM
Workload Estimator and to get the recommended servers for your solution.
1.0 Documentation
The following documents should be read and understood prior to filling out the planning templates and
questionnaires.

“Developing Workload Solutions for the IBM eServer Workload Estimator” provides a step-by-step
how-to guide for creating a workload solution.
http://www.developer.ibm.com/graphics/csf/common/pdf/IBMeServerWL-HowTo.pdf

“IBM eServer Workload Developer Users Guide” is the Users Guide for a tool called the Workload
Developer. You will create the Workload Solution for your application using this tool and the data
collected at the Benchmark Center.
http://www.developer.ibm.com/graphics/csf/common/pdf/IBMeServerWL-UsersGuide.pdf
3
2.0 Planning Session Calls
Planning Time Line
Planning Phase
Conference calls scheduled as needed
Nomination
Form Rec’d
2.1
Introductory
Planning Call
Preliminary Planning
Documentation Due
Test Phase
in Rochester
Readiness Review:
Final Planning
Documentation Due
Sizer
Available for
Publication
Introduction to Workload Solution Offering
The purpose of the Introductory Planning Call is primarily to describe the offering. The program
requirements will be described along with the scope of the offering, hardware available, and the effort
involved, including required planning and preparation. The schedule for preparation and a timeframe for the
week-long test in Rochester will be discussed.
Following this call the customer will need to identify the application(s), the database and the business
transactions to be tested. Templates and questionnaires will be made available to collect this information.
These must be filled out and returned prior to the next planning call.
The requirements of the Workload Solution offering are as follows:






A fully tested and stable application - no test code, please
A database representative in size and number of records of a typical customer’s data
An application or sub-set of an application to be sized
NOTE: Because of the short duration and the amount of specialized work involved, the workloads to be
sized should be targeted toward those most often marketed or those workloads that are problematic to
your marketing efforts
The interface to identified workload must be HTML or 5250-based
An understanding of your customers interaction with the application, i.e., how do they navigate though
the application
An understanding of your customer set, i.e., number of users, business transaction volumes, orders
entered in a day, hits-per-second, number of customer accounts, etc.
4
2.2
Planning Calls – In depth planning and review
Prior to the first planning conference call, draft versions of the following documentation should be
completed and sent to the Benchmark Center.







End-User Questions,
Workload Definition Template,
Run Matrix Template,
Hardware/Software Questionnaire,
Database Questionnaire,
Workload Simulation Questionnaire, and
Daily Activities Workplan
These planning conference calls are used as working sessions to answer any questions on the process and
the preparation templates. The number of calls needed will vary.
At the end of the planning phase, “final” versions of all the documentation mentioned above should be
available, in addition to paper scripts documenting the end-user’s navigation through the application.
2.3
Final Call - Readiness Review
Prior to this call, the preparation templates and questionnaires, including the paper scripts should be
complete and emailed to the Benchmark Center. The purpose of this call is to complete a final review of the
planning templates, review the paper scripts, and assess the readiness of the benchmark effort. Typical
questions include:













Have the End-User Questions been defined?
Has the Workload Definition Template been completed? We will walk through these items.
Has the Run Matrix Template been completed including prioritization of the runs? We will walk
through these items.
Have Paper Scripts been created? We will review these items.
Has the Hardware/Software Questionnaire been completed? We will walk through these items.
Has the Database Questionnaire been completed? We will walk through these items.
Has the Workload Simulation Questionnaire been completed? We will walk through these items.
Daily Activities Workplan – does a detailed benchmark plan exists which encompasses all necessary
steps to bring this Workload Sizing exercise to a successful completion? This includes a detailed,
realistic plan of events in Rochester that is documented, from the install through system clean-up.
Are you adequately staffed and prepared to assume all responsibilities and complete all tasks as required
in your benchmark activities work plan?
Is a stable version of the database and programs saved off to tape?
Is the Restore/Install procedure documented and fully “debugged”?
Are all application / system set up procedures (creation of user profiles, authority changes, pool sizes,
etc. for the scripts) identified?
Are copyright releases and any necessary confidential disclosure and internet usage agreements signed?
5
3.0 End-User questions
The Workload Estimator is primarily a sales tool for your solution. It should be designed to be used quickly
by the end user. Pick a few significant features of your solution that the sales people will be asked about by
potential customers. These will generally relate to the input questions for the tool. You could ask your sales
representatives to develop a list of questions that they feel are the most important ones they will get asked
when doing a sizing effort for a customer or prospect. Keep the list of questions under a dozen or so. Your
questions will be presented to the end-user as in the following example. You can access this example WLE
Workload Solution at the following URL. Look toward the bottom of the web page. Documentation on
developing this workload is also available at:
http://www.developer.ibm.com/welcome/eserver/e3/CSFServlet?mvcid=main&packageid=3000
6
The screen above shows some questions from a order entry application: How many active users, peak
number of orders entered per hour, etc. The questions will vary by application type. Some more examples
are below.








How many users will be logged on? How many will be actively using the system?
How many concurrent active web sessions will you have?
How many messages will you receive/send per hour? What is the average message size?
How many transactions will be run per hour? (or minute) (by transaction type if you have multiple
transaction types)
How many of your users are light users, typical users, heavy users? (Don't forget to define the details of
each of these user types.)
What size database will you have (number of customers, items, orders,...)?
Do you have seasonal peaks? If so how much higher is you peak load?
What life span do you want for this system? How much growth should be allowed for over this life
span?
Help text should be provided describing what is meant by each question.
Questions for your application are:





7
4.0 Preparation: Spreadsheets and Questionnaires
In order to come up with the performance calculations for the workload, it will be necessary to run the
application on different iSeries models and gather performance data. The goal is to link the end-user
questions to the processing resources required. (Refer to the following document:
http://www.developer.ibm.com/graphics/csf/common/pdf/IBMeServerWL-HowTo.pdf
for detailed information on this process.) Several tests will be required, varying the workload and the
hardware until all of the necessary performance information is gathered. This data is analyzed, and
calculations or formulas are determined to input into the workload generation tool.
These tests will be run in the iSeries Benchmark Center on Benchmark Center hardware. The LoadRunner
tool will most likely be used to generate scripts to drive the application. There are a series of templates
available to guide you through the planning the tests. The above mentioned document must be read and the
templates must be filled out and “paper scripts” must be developed before you will be ready for the oneweek test at the iSeries Benchmark Center. Excel versions of the templates are available.
8
4.1
Workload Definition Template
The first template is the workload definition template. It is used to identify the different user groups and the
key functions they perform. Scripts will be created for each User Group or Business Transaction, depending
on the level of granularity desired in the sizing tool. Virtual users will execute the scripts on the iSeries.
While the scripts are executing, system resource utilization will be measured. This data forms the basis of
the sizing tool. Care should be taken to select the most representative functions. Due to practical and time
constraints the number of functions must be kept to a minimum.
A key column in this template is “Peak Transactions per Hour”. The workload should represent a typical
peak hourly rate of work for a given set of users.
The workload definition should be reviewed and approved prior to the creating paper scripts. No
modifications can be made during the benchmark test.
Here is an example.
User Group
Order Management
Inventory
Management
# Users Business Transactions
50
Order Entry
Order Inquiry
50
Ship Confirm
Ship Tracking
Peak Trans/Hr
150
125
50
100
Trans/User/Hr
3
2.5
1
2
Note: In the previous table, Peak Trans/Hr is the total transactions executed by all of the users,
i.e. Peak Trans/Hr = Trans/User/Hr * # Users.
4.2
Detailed User Group Scenarios - Paper Scripts
For each User Group (or Business Transaction) identified in the Workload Definition Matrix, there is a
corresponding set of keystrokes that represent the typical workflow for the group of users. These keystrokes
(and/or mouse movements) must be documented. Screen captures can be used to document the workflow.
Variable and constant data should be identified.
See Appendix A for a partial example of a paper script.
More information about the LoadRunner product can be found on the Mercury Interactive website:
http://www.mercuryinteractive.com/products/loadrunner/
9
4.3
Run Matrix Template
The Run Matrix template lays out the various runs or tests that are planned to be executed in the Benchmark
Center. The User Group, hardware configuration and number of users are key components in this template.
The number of users is a target for planning purposes. It is not known until after running some of the scripts
if in fact the hardware configuration will support the targeted number of users. The three different numbers
of users become three points on a curve in that illustrates how well the application scales and how much
CPU is required per user or per transaction. Typically a first run is done with a minimal number of users on
a small configuration. (In the example below, 50 users on the 810 running Order Management.) Then the
number of users is increased until the CPU is about 60% busy. A final point is taken with the goal of 8090% CPU utilization. Quite often the scripts can be designed such that the 3 measurement points are
executed as a single run with users added and data gathered in stages. The following is an example of a
completed Run Matrix.
Run User Group Business
Hardware
Target Number of
#
Transactions Configuration Users
1
Order
Order
Management Entry
810-2465
50
100
150
Order
1-way
Inquiry
2
Order
Order
Management Entry
810-2469
100
150
500
Order
2-way
Inquiry
3
Order
Order
Management Entry
825-2473
150
500
1000
Order
3-way
Inquiry
4
Inventory
Ship
Management Confirm
810-2465
50
100
150
Ship
1-way
Tracking
5
Inventory
Ship
Management Confirm
810-2469
100
150
500
Ship
2-way
Tracking
6
Inventory
Ship
Management Confirm
825-2473
150
500
1000
Ship
3-way
Tracking
7
Order
Order
Management Entry
810-2469
50
100
150
Order
2-way
Inquiry
8
Inventory
Ship
Management Confirm
810-2469
50
100
150
Ship
2-way
Tracking
Priority Comments
1
7
3
2
6
8
4
5
Memory
test
Memory
test
10
Notes:
1. The example in the table above assumes that Order Entry and Order Inquiry are combined together as a
single workload. For greater granularity in the sizing tool, each of the Business Transactions could be
scripted and run separately. This would be advantageous if two different customer installations would
typically have different ratios of order entry transactions to order inquiry transactions. For example, in a
traditional phone order entry environment there may be twice as many new orders entered as there are
inquiries on existing orders. In a web environment, where the inquiry function is available to the enduser customer, there may be many more inquiries than new orders. Capturing each transaction separately
allows the sizing tool to size various ratios of the different transactions. If this variability in transaction
mix is not necessary, then the transactions could be captured together in a single “order management”
workload, as shown here.
2. In the example above we show 2 different 810 models, a one-way and a two-way. These tests could be
run on a single 810-2469 by logically removing a processor from the configuration to approximate an
810-2465.
3. The first 6 runs listed are designed to size CPU and the last two are tests used to gather information for
memory sizing. See the “Developing Workload Solutions” document for further information on sizing
memory and disk. For the CPU tests listed here, memory and disk configurations are not specified. They
need to be sized large enough so that neither memory nor disk are bottlenecks.
4. After completing this matrix, the Daily Activities Workplan can be filled out. Each run can be mapped
to a specific timeframe for planning purposes.
5. The matrix here shows a fairly complete set of runs: 2 workloads on 3 different processor
configurations. In many cases it won’t be necessary to complete every point in the matrix. Keep in mind
that the goal of this exercise is to determine the relationship between business transactions and CPU
required to process them. It may be sufficient to run one of the workloads on all 3 configurations and run
the other workload(s) on just one or two of the configurations. In this example, the runs that are lower
priority would be good candidates to leave out of the Workplan.
11
In the chart below we show how a single run can be performed that has 3 data points: 50 users, 100 users
and 150 users.
8 2 5 - 2 4 7 3 w it h 3 processors and 8 GB
m em ory
160
140
120
100
80
60
40
20
0
Virtual Users
% CPU
1
10
19
28
37
46
55
64
73
82
91 100
Ela pse d t im e in m in u t e s
Graph notes:
 The first ramp up (0-50 virtual users) sets up these users for the first measurement timeframe
 Following the completion of the user ramp up, a stabilization period occurs before steady state is
reached.
 Measurement data is based on the average value in the steady state period.
 After the first steady state measurement period, the process is repeated for a second and third load.
 A ramp down occurs at the end of the test.
12
4.4
Hardware/Software Questionnaire
The hardware that is typically available in the Benchmark Center is listed below. Exact configurations may
vary depending on availability.
800-2463 1-way:






8GB memory
3 - 35GB DASD drives
3 - 70GB DASD drives
GB Ethernet
CDROM
1/4" Tape Drive
810-2466 1-way:





4GB memory
6 - 17GB DASD drives
GB Ethernet
CDROM
1/4" Tape Drive
810-2469 2-way:





4GB memory
6 - 35GB DASD drives
100MB Ethernet
CDROM
1/4" Tape Drive
825-2473 3-6way:
 8GB memory
 10 - 35GB DASD drives
 100MB Ethernet
 CDROM
1/4" Tape Drive
All systems will be pre-loaded with V5R2 and most of the typical LPPs. The latest CUM PTF level and
group PTFs for Database, HIPERs, and Performance will also be loaded. Use the Hardware/Software
Questionnaire to list any additional, specific requirements.
Hardware Questionnaire
Target Configurations
Are there hardware configurations that are of particular interest? e.g. are there certain base configurations
that are typically recommended to your customers?
____________________________________________________________________________________
Disk Configuration Specifics
List here any special disk configuration requirements. For example, are there any user ASPs required? Is
there a specific number of arms that you wish to test with? And why?
____________________________________________________________________________________
Are you interested in testing a competitive configuration (i.e., minimal configurations), or a disk
configuration that will give optimal performance?
____________________________________________________________________________________
13
Memory Configuration
The memory listed in the configurations above is the maximum memory available. If less memory is
desired, memory can be removed from the configuration by using LPAR or by temporarily moving memory
to a subsystem that isn’t used during the test.
CPU
On the n-way configurations, it is possible to temporarily remove a processor from the configuration using
LPAR. This should be noted in the Run Matrix.
Communications Configuration Specifics
Generally, all systems will be configured with a single Ethernet connection and connected via a switched
network. As an option, you may connect into the benchmark center systems via VPN as well as access the
Internet from these systems (a separate no-charge agreement must be signed prior to the engagement to
enable this feature – See Appendix A).
____________________________________________________________________________________
____________________________________________________________________________________
Software Questionnaire
What media type will be used to bring the application and data to the Benchmark Center?
____________________________________________________________________________________
List any LPPs are required that are not on the standard set.
____________________________________________________________________________________
Any other software notes:
____________________________________________________________________________________
3rd Party Software
List all 3rd party software that you will be bringing to the Benchmark Center. A Copyright Release Letter is
required which covers all 3rd party software, including the application you are building the Workload
Estimator for. See Appendix A for templates for the Copyright Release Letter.
____________________________________________________________________________________
____________________________________________________________________________________
14
4.5
System Customization, Application Install Procedure
The iSeries systems will be installed with OS/400 and most Licensed Programs. You will be responsible for
loading your own application and database and for any system customization. The procedure must be
carefully thought out and documented. Define the procedure below or in a separate document.
Restore/Install Plan
What libraries will you need to bring to the Benchmark Center? Will data and program libraries be saved
from system at home and a RSTLIB done here? Or will you be bringing SAVSYS tapes? Do you have any
special requests of the Benchmark Center for setting up the system, installing the Operating System and
Program Products?
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
System Customization/Application Set-up
What system customizing steeps will need to be done? i.e., user profiles, subsystems, system values?
Communications lines configured? Application set-up and install? What are tasks that need to be done, who
will be responsible to complete the task.
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
15
4.6
Database Questionnaire
A test database must be provided that is representative of target customers in size and structure. Sometimes
a database from an actual customer can be saved off to tape and restored on a test machine to be used for the
testing at the Benchmark Center. Another solution is to generate a database with the necessary number of
records and structure. Care must be taken to have proper indices in place for good performance.
What is the format of the data your application runs against? UDB/400, Notes DB, IFS files, other?
____________________________________________________________________________________
What is the planned source for your database?
____________________________________________________________________________________
____________________________________________________________________________________
What are the sizes for some of the key files, in either gigabytes and/or record counts?
____________________________________________________________________________________
Are there other considerations for the structure, amount or content of the data? Some key factors may
include how long historical data is kept on-line, what percent of the data is at a particular processing status,
i.e. 15% of the records in the transaction file are in “hold” status, etc. The goal is to represent the typical
customer as closely as possible.
____________________________________________________________________________________
Resetting the database is necessary to ensure a consistency between tests. What method do you plan to use
to reset the database?
____________________________________________________________________________________
____________________________________________________________________________________
16
4.7
Run Procedures
The “Run Procedures” checklist is the list of tasks done between runs to ensure a consistent starting point
for each performance test. An important part of the Run Procedures is the task of documenting each run.
This can be as simple as noting a description and the start and stop time for each run.
The following checklist should be tailored to meet the needs of your specific application and test
environment.
1. Run sheet - log the run on a run sheet (or spreadsheet) with date/time, workload description, name of
QPFRDATA member, and LoadRunner results file name. After the run, record key performance
metrics such as CPU %, response time, etc.
2. iSeries checklists:

Collection Services - verify that Collection Services has a status of “started.” Typically, data is
collected at 1-minute or 5-minute intervals, depending on the overall length of the measurement
period. Verify the properties for collection services. The collection profile should be standard plus
protocol, the retention period for detailed data should be “permanent” and for graph data should be 7
days.

Memory Allocation - review pool sizes, activity levels, paging characteristics (*CALC versus
*FIXED). Consider using CLRPOOL to clear memory pools between runs.

Database Reset Procedure - list the specific steps required to reset the database. These may
include: clearing files, resetting a date, restoring a subset of files, etc.

Application Reset Procedure - list any other specific steps needed to verify the application
environment is set up appropriately for the run: library list, data areas, etc.

Web servers, other subsystems - are applicable subsystems and servers started? HTTP server,
WebSphere, etc. Are the appropriate server-specific tuning parameters set correctly for the run?
Garbage collection, timeout values, queue size, etc.

Interactive Jobs - check for disconnected jobs from a previous run. Check controllers and devices
used by Mercury interactive jobs. Make sure all are in the correct state (i.e., none are varied off).

Batch - check job queues, are any applicable batch jobs submitted and on hold? Check output
queues. Clear output queues if necessary. Put output queues on hold so any spool files created during
the run do not print.

Communications - check communication lines, if applicable. Check communications between any
local systems in the benchmark and to the simulation tools. Vary off internet or remote dial-up
connections so remote users cannot log on during the run.

Journaling, if applicable - is Journaling/MIMIX turned on? If a user ASP is used, is it big enough?
Do any journal receivers need to be deleted? If not using a user ASP for journal receivers, consider it
for performance.
17
3. Simulation Tool checklists:

IPL Results System

IPL Controller and Load Generator Systems

Verify Controller Settings - Run-Time File Storage (on a shared network drive), Results Name,
Scripts, Data Files, Run-Time Settings, Scenario Schedule Settings

Connect to Load Generators

Start Scenario

Manually monitor CPU & Memory Utilization of Controller and Load Generator Systems (are they
in an acceptable range?)
4.8
Workload Simulation Questionnaire
Typically, we will have 8 PC’s for workload simulations configured as follows:








Dual 1ghz processor system (or better, if required)
1GB of RAM (or better, if required)
Windows 2000 with the latest Service Pack and Hot Fixes
Internet Explore version 6 with the latest Service Pack and Hot Fixes
18GB HDD
1 GB Ethernet card
100 MB Ethernet card
Attached to a GB Ethernet capable switch
Is a different PC Operating Systems required?
____________________________________________________________________________________
Are there specific Service Packs or Fixes required?
____________________________________________________________________________________
Is a different internet browser required? Are any Service Pack levels required? Are any plugins required?
____________________________________________________________________________________
Are there other considerations or requirements for the simulation environment?
____________________________________________________________________________________
____________________________________________________________________________________
____________________________________________________________________________________
18
4.9
CPW/CIW Requirements Template
This template will be filled out as the runs are completed in the iSeries Benchmark Center. This
information will form the basis for the CPU calculations in the workload. An example follows
Syst e m
CPW Ra t ing CI W Ra t ing CPU% CPW
750
250
30%
810-2465
750
250
56%
1-w ay
750
250
82%
2700
950
16%
810-2469
2700
950
22%
2-w ay
2700
950
78%
6600
2890
8%
825-2473
6600
2890
26%
6-w ay
6600
2890
48%
6600
2890
77%
Use d CI W Use d Or de r s/ Hr # Use r s CPW /
225
75
500
50
420
140
1000
100
615
205
1450
150
432
152
1000
100
594
209
1480
150
2106
741
5023
500
528
231.2
1530
150
1716
751.4
5100
500
3168
1387.2
10200
1000
5082
2225.3
15200
1500
Or de r CI W /
0.450
0.420
0.424
0.432
0.401
0.419
0.345
0.336
0.311
0.334
Or de r
0.150
0.140
0.141
0.152
0.141
0.148
0.151
0.147
0.136
0.146
4.10 Memory Requirements Template
This template will be filled out as the memory tests are completed. In the example below, three tests were
conducted, varying the number of users and the amount of memory. This information will be used to
determine the correct memory calculation for your application.
50 User Test
Memory (mb) Faults/ sec
88
93.7
100
67
112
27.4
125
4.8
150
2.6
175
0.9
225
0.4
100 User Test
Memory (mb)
Faults/ sec
175
62.1
188
40.3
200
7
225
2
250
4.6
300
0.6
350
2.5
150 User Test
Memory (mb)
Faults/ sec
225
126
275
33.8
325
6.1
375
6.8
425
1.2
475
2.1
4.11 Disk Requirements Template
This template is optional. Most of the time it won’t be necessary to do extensive testing with varying disk
configurations. However, if such testing is done, then this template would be used to assist in analyzing the
data and determining the disk calculations for the workload.
Disk Drive Requirements
system
810-2465
810-2465
810-2465
810-2465
810-2465
810-2465
810-2465
810-2465
810-2465
810-2465
810-2465
# of drives
10
10
10
10
10
10
20
20
20
20
20
% busy
7%
12%
33%
45%
57%
88%
7%
19%
25%
31%
40%
CPU % busy
13%
21%
47%
60%
73%
88%
21%
47%
60%
73%
88%
Orders/hour Drives at 25%
50
2.8
100
4.8
300
13.2
400
18
500
22.8
600
35.2
100
5.6
300
15.2
400
20
500
24.8
600
32
19
4.12 Daily Activities Workplan
8AM
9AM
10AM
11AM
Noon
1PM
2PM
3PM
4PM
MONDAY
Welcome & Orientation
Chalk talk
Load/Restore iSeries data on 810 and 825
Configure iSeries systems as necessary
Verify applications run correctly - tweak as necessary
Lunch
Develop Order Management: Order Entry Script
Capture Order Entry Script
Parameterize Order Entry Script
Test Order Entry Script with single user
Test Order Entry Script with multiple users
TUESDAY
Daily Status meeting
Complete Testing of Order Entry Script
Develop Order Management: Order Inquiry Script
Capture Order Inquiry Script
Parameterize Order Inquiry Script
Test Order Inquiry Script with single user
Lunch
Test Order Inquiry Script with multiple users
Run 1: Order Management on 810-2464: 50,100,150 users
This is an iterative process
Several runs may be required to debug script
WEDNESDAY
Daily Status meeting
Analyze Run 1 results and add to WLE Workload Solution
Develop Inventory Management: Ship Confirm Script
Same steps as previous scripting effort
Develop Inventory Management: Ship Tracking Script
Same steps as previous scripting effort
Lunch
Complete Inventory Management Scripts / testing
Run 2: Inventory Management on 810-2464:
50, 100, 150 users
This is an iterative process
Several runs may be required to debug script
THURSDAY
Daily Status meeting
Run 2: (Continued)
Analyze Run 2 results and add to WLE Workload Solution
Lunch
Run 3: Order Management on 825-2473:
500, 1000, 1500 users
This is an iterative process
Several runs may be required to debug script
Analyze Run 3 results and add to WLE Workload Solution
FRIDAY
Daily Status meeting
Run 4: Memory Test for Order Management on 810-2469
Run 5: Memory Test for Inventory Mgmt on 810-2469
Analyze Run 4 results and add to WLE Workload Solution
Analyze Run 5 results and add to WLE Workload Solution
Lunch
Workload Solution (sizer) test and tweak
Exit meeting – what’s next?
Save data from system, clean up
20
Create your Daily Activities work plan on the following sheet or using your tool of choice. The key is that you
document the plan and follow it as closely as possible. Having a clear idea of priorities will help in the event
changes are required during the benchmark.
8AM
9AM
10AM
11AM
Noon
1PM
2PM
3PM
4PM
MONDAY
TUESDAY
WEDNESDAY
THURSDAY
FRIDAY
21
5.0 Create and Test the Workload
After the tests have been executed and the data analyzed, you are ready to use the IBM eServer Workload
Developer to create the workload. Reference the “IBM eServer Workload Developer Users Guide” for
complete details on how to create a workload. See the “Example Workload Solution for the IBM eServer
Workload Estimator” for an example workload.
6.0 Save the workload as XML and HTML
Save the workload as a Workload Solution XML file. This is the file that will be added to your solution.
Save the workload as an HTML file for testing.
After you have saved the workload, you can test it by bringing the HTML file up in a browser and linking to
the IBM eServer Workload Estimator on the test server. You can use Netscape or Microsoft Internet
Explorer. Once in the Estimator, select the defaults and test your workload. After you have completely
tested your workload, save the final workload XML file. This final XML will be the file that you will add to
your solution.
22
7.0 Validation
A final step following the testing and creation of the Sizing Guide in the Benchmark Center will be to verify
the results against one, or ideally several, production servers. Do the recommendations made by the
Workload Solution fall in line with the day-to-day utilization that customers are actually experiencing? This
is the last opportunity, before publication of the sizing product, to ensure its reasonability in real-world
environments. Results of this testing will provide some feedback that can allow for tweaking the Workload
Solution so it is even more useful once it is made available.
23
Appendix A – Example of Paper Scripts
Example “paper script” from a 5250 application.
Take option 1 from the Menu to “Create a New Reservation” and press the Enter key
24
Type in the Reservation Agents name and password and press the F10 key.
25
Key in the date of the flight, and position the cursor to the “From City” field and press the F4 key. This will
take you to a pop up window to enable you to select the departure city from a list (this is an example of
variable data).
26
Key the first few characters of the City name you are departing from and press Enter. This will reposition
the list of Cities to the first city that most closely matches the input string.
27
Tab or position the cursor to the selection field for the line with “Rochester, MN” and key a “1” . Then
press Enter. This will fill in the “From City” field on the Create Order screen.
28
The following are examples of the scripting tool from Mercury Interactive called VUGEN. During the
scripting process we will use your paper scripts and capture the application using VUGEN. After we have
captured the script, we will rerun it to make sure it plays back correctly and then begin the process of
parameterizing the fields that contain variable data, ie., Customer Numbers, Item Numbers, etc.
29
Appendix B – Legal Documents Required
Customer Copyright Release (customer owns the copyright)
This sample letter should be typed on the customer’s letterhead exactly as shown below. Replace:
• ’ABC Corporation’ with the customer name
• ’XXXX’ with the software package name(s)
Any changes to the content other than the above names will require IBM legal approval.
____________________________
Mr. Ken Wise, Business Administrator
iSeries Benchmark Center
IBM Corporation, EKFA/664-1
3055 41st Street NW
Rochester, MN 55901
Dear Mr. Wise:
ABC Corporation will be using its copyrighted software package ‘XXXX’ during the upcoming benchmark at the
iSeries Benchmark Center in Rochester, Minnesota.
ABC Corporation warrants that the installation of this package will not constitute a violation of ABC Corporation’s
copyright. The package will be installed and used for this benchmark only.
ABC Corporation will be responsible for:
1.
2.
3.
4.
5.
Installation,
Integration,
Problem determination,
Problem resolution, and
Deletion from the IBM system of the ‘XXXX’ package.
IBM will be responsible for the installation, problem determination, problem resolution, and tuning of all supported
IBM software under which the ‘XXXX’ package will execute.
IBM will be given no material by ABC Corporation that is the confidential or proprietary information of ABC
Corporation or any other party. In no event will IBM be given any source program code.
ABC Corporation agrees not to externally publish or distribute the performance results obtained from this benchmark
without the express written consent of the iSeries Benchmark Center.
Sincerely,
Authorized Representative of ABC Corporation
30
Customer Third-Party Copyright Release (customer is a licensed user)
This sample letter should be typed on the customer’s letterhead exactly as shown below. Replace:
• ’ABC Corporation’ with the customer name
• ’XYZ Inc’ with the software owner’s name
• ’XXXX’ with the software package name(s)
Any changes to the content other than the names of the parties making the release will require IBM legal
approval.
____________________________
Mr. Ken Wise, Business Administrator
iSeries Benchmark Center
IBM Corporation, EKFA/664-1
3055 41st Street NW
Rochester, MN 55901
Dear Mr. Wise:
This letter will certify that ABC Corporation has the permission of XYZ Inc. to install XYZ Inc.’s copyrighted
software package ‘XXXX’ at the iSeries Benchmark Center in Rochester, Minnesota.
ABC Corporation warrants that the installation of this package will not constitute a violation of ABC Corporation’s
licensing agreement with XYZ Inc. The package will be installed and used for this benchmark only.
Either ABC Corporation or a duly authorized XYZ Inc. representative will be responsible for:
1.
2.
3.
4.
5.
Installation,
Integration,
Problem determination,
Problem resolution, and
Deletion from the IBM system of the ‘XXXX’ package.
IBM will be responsible for the installation, problem determination, problem resolution, and tuning of all supported
IBM software under which the ‘XXXX’ package will execute.
IBM will be given no material by ABC Corporation that is the confidential or proprietary information of ABC
Corporation, XYZ Inc., or any other party. In no event will IBM be given any source program code.
ABC Corporation agrees not to externally publish or distribute the performance results obtained from this benchmark
without the express written consent of the iSeries Benchmark Center.
Sincerely,
Authorized Representative of ABC Corporation
31
Inadvertent Disclosure Letter
The following will be provided on IBM Letterhead:
SUBJECT: Inadvertent Disclosure Letter Agreement
During your attendance at ______________ on ______________, you may be inadvertently exposed to IBM Confidential
Information.
“IBM Confidential” information is defined as all information, identified as IBM Confidential, that relates to IBM’s past, present,
and future research, development, and business activities.
IBM Confidential information does not include information that is publicly available, rightfully received from third parties,
without obligation of confidence, already in your possession, independently developed by you, or approved for release or
disclosure by IBM in writing.
You agree to hold all IBM Confidential information in trust and confidence for IBM and agree not to disclose any such
information to anyone for a period of five (5) years.
This signed Agreement constitutes a legal and binding Agreement between IBM and You. This Agreement may be executed by
facsimile copy.
COMPANY NAME
By:
INTERNATIONAL BUSINESS MACHINES
CORPORATION
By:
(Authorized Signature)
Name:
(Authorized Signature)
Name:
(Type or Print)
Title:
(Type or Print)
Title:
(Type or Print)
Date:
(Type or Print)
Date:
32
Internet Usage Agreement
The following will be provided on IBM Letterhead:
DATE
Company Name
Company Address
Attention: XX. XXXXX XXXXXX
Subject: Internet Usage Agreement number XXXXXXXXXX
1. IBM's Benchmark Center in Rochester, Minnesota (the “Center") has established a commercial connection to the
internet. This Agreement between International Business Machines Corporation ("IBM") and Company Name
("Company") covers the terms under which IBM agrees to grant Company internet access through Center PC's
and Systems ("BM Systems") or port connections for Company PC's, Laptops, or Handheld devices.
2. Company will ensure that no Harmful Code will enter IBM as a result of Company's internet access. For the
purposes of this Agreement, "Harmful Code" is defined as any computer programming code which is constructed
with the intent to damage, interfere with or otherwise affect other computer programs, data files or hardware,
without the knowledge or consent of the computer user. "Harmful Code" includes, but is not limited to, selfreplicating and self-propagating program instructions commonly referred to as "viruses" or "worms". If Company
discovers or reasonably suspects that Harmful Code has entered IBM, Company shall immediately notify IBM.
Additionally, Company agrees not to introduce any virus onto the internet, your laptop, Company system, BM
System(s), remotely attached computer systems or other IBM system(s). If you do so, the internet connection may
be immediately terminated without legal or other recourse to you. Further, you agree to hold IBM harmless from
and to indemnify IBM for any damages or claims for damages resulting from any viruses introduced by you onto
the internet, your laptop, Company system, BM System(s), remotely attached computer systems or other IBM
system(s) except if it is determined that such claims for damage are due to the intentional misconduct of IBM.
3. Company, with assistance from IBM, will control access to their benchmark environment (“BM Environment”) via
the appropriate facilities provided on the BM Systems. Company releases IBM from any liability for any
misappropriation of Company's BM Environment, data, or other assets, as a result of Company’s accessing the
BM System(s) except if it is determined that such claims for damage are due to the intentional misconduct of IBM.
4. Company, with assistance from IBM, shall be solely responsible for the security of its BM Environment, data and
other assets. Company acknowledges that others have access to the internet. Company releases IBM from any
liability for any misappropriation by any user of the internet or the IBM local area network, of Company's BM
Environment, data or other assets, whether located on the BM System(s) or elsewhere except if it is determined
that such claims for damage are due to the intentional misconduct of IBM.
5. Company, with assistance from IBM, will provide security and assure security compliance for Company laptop,
Company computer system, other Company-owned equipment, BM System(s), and remotely attached computer
systems when linked to the internet. Company agrees to hold IBM harmless from and to indemnify IBM, for any
damages or claims for damages resulting from any security breaches that occur when Company laptop, Company
computer system, other Company-owned equipment, IBM system(s), and remotely attached computer systems
are linked to the internet connection except if it is determined that such claims for damage are due to the
intentional misconduct of IBM.
6. IBM reserves the right to terminate the internet connection immediately without notice if in the sole discretion of
IBM, Company access has been used in such a manner that it restricts or inhibits any other person from using the
internet connection, and in particular, Company agrees not to use or to allow others to use the service directly or
indirectly to:
33
a.
b.
c.
d.
e.
f.
g.
h.
i.
j.
k.
Use simultaneous, unattended continuous connections to BM System(s) with one User ID;
Use the internet connection for non-benchmark related activity except as needed to maintain Company work environment
while visiting the Benchmark Center (i.e., email, work related programming, work related processing, etc.);
Connect remote devices to BM System(s) and networks for non-benchmark related activity
Post, transmit, or promote any unlawful, harmful, threatening, abusive, harassing, defamatory, vulgar, obscene, sexually
explicit, hateful, racially, ethnically or otherwise objectionable or offensive content;
Harass, threaten, embarrass, or cause distress, unwanted attention or discomfort to, any other person;
Impersonate any person or entity, or communicate under a false name or a name you are not entitled or authorized to
use;
Post or transmit chain letter or pyramid schemes;
Post or transmit any unsolicited advertising, promotional materials, or other forms of solicitation to other persons, or
Post or transmit any communications or solicitation designed or intended to obtain password, account or private financial
information from any other person;
Violate any operating rule, policy or guideline of any other service provider;
Violate any applicable local, state, national, international or foreign law, including but not limited to any rule or regulations
having the force of law.
7. Company right to use the internet connection is not transferable without the express written consent of IBM. You
agree to protect any and all passwords provided and to keep them secure from unauthorized users and use.
8. The term of this Agreement shall start on (insert date) and continue until (insert date). Either party may terminate
this Agreement upon fifteen (15) days prior written notice to the other party.
9. Except as expressly set forth in this Agreement, IBM makes no warranties nor assumes any liabilities in
connection with this Agreement including but not limited to accomplishment, completion, or correctness of any
result of this Agreement. Any information, materials, or services furnished by IBM pursuant to this Agreement are
on an "AS IS", "AS AVAILABLE" basis. IBM MAKES NO WARRANTIES, EXPRESS OR IMPLIED, INCLUDED
BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
PARTICULAR PURPOSE, AS TO THE DOCUMENTATION, FUNCTION OR PERFORMANCE OF SUCH
INFORMATION, MATERIALS, OR SERVICES. UNDER NO CIRCUMSTANCES SHALL IBM BE LIABLE FOR
ANY INDIRECT, INCIDENTAL, SPECIAL OR CONSEQUENTIAL DAMAGES, INCLUDING, WITHOUT
LIMITATION, LOSS OF PROFITS.
10. This Agreement shall be construed and the legal regulations created herein between the parties shall be
determined in accordance with the laws of the State of Minnesota.
11. Company agrees to fully indemnify, defend, and hold IBM harmless against any claim that the BM Environment
infringes on any intellectual property right of any third party or any other claims arising from the BM Environment
except as it is determined through the intentional misconduct of IBM staff.
34
If the above terms and conditions are acceptable to Company, please have this Agreement signed by an authorized
officer of Company and return one copy of this Agreement to the attention of Ken Wise, DEPT EKFA, Building 664-1
at the above address or fax number 507-253-4892.
COMPANY NAME
By:
IBM Corporation
By:
(Authorized Signature)
Name:
(Authorized Signature)
Name:
(Type or Print)
Title:
(Type or Print)
Title:
(Type or Print)
Date:
(Type or Print)
Date:
35
Remote Usage Agreement
The following will be provided on IBM Letterhead:
DATE
Company Name
Company Address
Attention: XX. XXXXX XXXXXX
Subject: Remote Usage Agreement number XXXXXXXXXX
1. IBM's Benchmark Center in Rochester, Minnesota (the “Center") has established a commercial
connection to the internet. This Agreement between International Business Machines Corporation
("IBM") and Company Name ("Company") covers the terms under which IBM agrees to grant Company
remote access to certain IBM Benchmark Center Systems (the "BM System") via the internet.
2. Company will establish its own commercial access to the internet at its own expense. Company shall
be solely responsible to ensure that its use of this connection complies with any laws and regulations,
and does not violate any obligations or rights of third parties. IBM may terminate this Agreement,
effective immediately, if notified of violations or, if in IBM's sole judgment, continued use of this
connection would be in violation of any laws, regulations, or policies of IBM or any internet access
provider.
3. IBM will provide remote access to one or more of the BM Systems that are attached to IBM's Local
Area Network (the "IBM LAN"). Availability of the BM Systems on the IBM LAN is subject to scheduled
emergency maintenance, power failure, emergency shutdown and other unanticipated conditions.
4. Company will use machine time on the BM Systems attached to the IBM LAN exclusively for Company to enable
and test its benchmark environment (“BM Environment”).
5. Without IBM's specific written advance authorization, Company will not copy or erase any code or data from any
of the systems directly attached to the IBM LAN, other than the BM Systems and BM Environment and data that
Company has been provided.
6. Company will ensure that no Harmful Code will enter IBM as a result of Company's internet access. For the
purposes of this Agreement, "Harmful Code" is defined as any computer programming code which is constructed
with the intent to damage, interfere with or otherwise affect other computer programs, data files or hardware,
without the knowledge or consent of the computer user. "Harmful Code" includes, but is not limited to, selfreplicating and self-propagating program instructions commonly referred to as "viruses" or "worms". If Company
discovers or reasonably suspects that Harmful Code has entered IBM, Company shall immediately notify IBM.
Additionally, you agree not to introduce any virus onto the internet, your laptop, Company system, BM System(s),
remotely attached computer systems or other IBM system(s). If you do so, the internet connection may be
immediately terminated without legal or other recourse to you. Further, you agree to hold IBM harmless from and
to indemnify IBM for any damages or claims for damages resulting from any viruses introduced by you onto the
internet, your laptop, Company system, BM System(s), remotely attached computer systems or other IBM
system(s) except if it is determined that such claims for damage are due to the intentional misconduct of IBM.
7. Company, with assistance from IBM, will control access to their BM Environment via the appropriate facilities
provided on the BM Systems. Company releases IBM from any liability for any misappropriation of Company's
36
BM Environment, data, or other assets, as a result of Company’s accessing the BM System(s) except if it is
determined that such claims for damage are due to the intentional misconduct of IBM.
8. Company, with assistance from IBM, shall be solely responsible for the security of its BM Environment, data and
other assets as well as for the security of its own Local Area Network and other facilities. Company
acknowledges that others have access to the internet. Company releases IBM from any liability for any
misappropriation by any user of the internet or the IBM LAN, of Company's BM Environment, data or other assets,
whether located on the BM System(s) or elsewhere except if it is determined that such claims for damage are due
to the intentional misconduct of IBM.
9. Company, with assistance from IBM, will provide security and assure security compliance for Company laptop,
Company computer system, other Company-owned equipment, BM System(s), and remotely attached computer
systems when linked to the internet. Company agrees to hold IBM harmless from and to indemnify IBM, for any
damages or claims for damages resulting from any security breaches that occur when Company laptop, Company
computer system, other Company-owned equipment, IBM system(s), and remotely attached computer systems
are linked to the internet connection except if it is determined that such claims for damage are due to the
intentional misconduct of IBM.
10. IBM reserves the right to terminate the internet connection immediately without notice if in the sole discretion of
IBM, Company access has been used in such a manner that it restricts or inhibits any other person from using the
internet connection, and in particular, Company agrees not to use or to allow others to use the service directly or
indirectly to:
a. Use simultaneous, unattended continuous connections to BM System(s) with one User ID;
b. Use the internet connection for non-benchmark related activity except as needed to maintain Company work
environment while visiting the Benchmark Center (i.e., email, work related programming, work related
processing, etc.);
c. Connect remote devices to BM System(s) and networks for non-benchmark related activity
d. Post, transmit, or promote any unlawful, harmful, threatening, abusive, harassing, defamatory, vulgar,
obscene, sexually explicit, hateful, racially, ethnically or otherwise objectionable or offensive content;
e. Harass, threaten, embarrass, or cause distress, unwanted attention or discomfort to, any other person;
f. Impersonate any person or entity, or communicate under a false name or a name you are not entitled or
authorized to use;
g. Post or transmit chain letter or pyramid schemes;
h. Post or transmit any unsolicited advertising, promotional materials, or other forms of solicitation to other
persons, or
i. Post or transmit any communications or solicitation designed or intended to obtain password, account or
private financial information from any other person;
j. Violate any operating rule, policy or guideline of any other service provider;
k. Violate any applicable local, state, national, international or foreign law, including but not limited to any rule or
regulations having the force of law.
11. Company right to use the internet connection is not transferable without the express written consent of IBM. You
agree to protect any and all passwords provided and to keep them secure from unauthorized users and use.
12. Company acknowledges that IBM may make available to Company third party code to use for accessing the BM
Environment. Company agrees to use the third party code in a way that does not violate any copyright, patent,
trade secret or other intellectual property right. Company agrees to use any and all code provided by IBM under
this Agreement solely for the purposes of the Agreement.
13. Each party will bear its own expenses in connection with this Agreement.
14. The term of this Remote Usage Agreement shall start on (insert date) and continue until (insert date). Either party
may terminate this Agreement upon fifteen (15) days prior written notice to the other party.
15. Except as expressly set forth in this Agreement, IBM makes no warranties nor assumes any liabilities in
connection with this Agreement including but not limited to accomplishment, completion, or correctness of any
result of this Agreement. Any information, materials, or services furnished by IBM pursuant to this Agreement are
on an "AS IS", "AS AVAILABLE" basis. IBM MAKES NO WARRANTIES, EXPRESS OR IMPLIED, INCLUDED
BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
37
PARTICULAR PURPOSE, AS TO THE DOCUMENTATION, FUNCTION OR PERFORMANCE OF SUCH
INFORMATION, MATERIALS, OR SERVICES. UNDER NO CIRCUMSTANCES SHALL IBM BE LIABLE FOR
ANY INDIRECT, INCIDENTAL, SPECIAL OR CONSEQUENTIAL DAMAGES, INCLUDING, WITHOUT
LIMITATION, LOSS OF PROFITS.
16. Company agrees to comply with all applicable federal, state and local laws, regulations and ordinances, including
but not limited to the Foreign Corrupt Practices Act; the Immigration Reform and Control Act of 1986, as
amended; and Regulations of the United States Department of Commerce relating to the Export of Technical
Data, insofar as they relate to the services to be performed under this Agreement. Company agrees to obtain all
required government documents and approvals prior to export of any technical data disclosed to Company or the
direct product related thereto.
17. Company represents that it is not subject, either directly or indirectly (by affiliation or any other connection with
another party), to any order issued by any agency of the United States Government revoking or denying, in whole
or in part, Company's United States export privileges. Company agrees to notify IBM immediately in the event
Company becomes subject to any such order.
18. Once signed, any reproduction of this Agreement made by reliable means (for example, photocopy or facsimile) is
considered an original.
19. Paragraph numbered 15 of this Agreement shall survive any completion, expiration or termination of this
Agreement.
20. This Agreement is the entire agreement on this subject between the parties. This Agreement supersedes all
previous written or oral agreements on this subject between the parties
21. This Agreement shall be construed and the legal regulations created herein between the parties shall be
determined in accordance with the laws of the State of Minnesota.
Company may not assign its rights, or delegate or subcontract its duties under this Agreement without the prior written
approval of IBM.
If the above terms and conditions are acceptable to Company, please have this Agreement signed by an authorized
officer of Company and return one copy of this Agreement to the attention of Ken Wise, DEPT EKFA, Building 664-1
at the above address or fax number 507-253-4892.
COMPANY NAME
By:
IBM Corporation
By:
(Authorized Signature)
Name:
(Authorized Signature)
Name:
(Type or Print)
Title:
(Type or Print)
Title:
(Type or Print)
Date:
(Type or Print)
Date:
38