®
IBM Software Group
Rational Developer for System z –
Unit Testing with zUnit
Jon Sayles / IBM - jsayles@us.ibm.com
© IBM Corporation
IBM Trademarks and Copyrights
© Copyright IBM Corporation 2013, 2014. All rights reserved.
The information contained in these materials is provided for informational purposes only, and
is provided AS IS without warranty of any kind, express or implied. IBM shall not be
responsible for any damages arising out of the use of, or otherwise related to, these
materials. Nothing contained in these materials is intended to, nor shall have the effect of,
creating any warranties or representations from IBM or its suppliers or licensors, or altering
the terms and conditions of the applicable license agreement governing the use of IBM
software. References in these materials to IBM products, programs, or services do not imply
that they will be available in all countries in which IBM operates.
This information is based on current IBM product plans and strategy, which are subject to
change by IBM without notice. Product release dates and/or capabilities referenced in these
materials may change at any time at IBM’s sole discretion based on market opportunities or
other factors, and are not intended to be a commitment to future product or feature availability
in any way.
IBM, the IBM logo, the on-demand business logo, Rational, the Rational logo, and other IBM
Rational products and services are trademarks or registered trademarks of the International
Business Machines Corporation, in the United States, other countries or both. Other
company, product, or service names may be trademarks or service marks of others.
2
Agenda
Topics:
 Unit Testing Value Proposition,
zUnit Terms, Concepts and Vocabulary
 Demonstration
 Next Steps
3
What is Unit Testing?
Unit Testing is a software testing
method by which individual
units of source code, sets of
one or more computer program
modules together with
associated control data, usage
procedures, and operating
procedures are tested to
determine if they are fit for use.
Wikipedia
An individual unit of software is a single
test-able logic construct or routine within
a call-able program:
•
Date validation
•
Credit Card number look-up
•
Tax computation
•
Co-pay calculation
This method of testing is sometimes
called “white Box testing”
(See Slide Notes for more on White Box Testing)
4
Black Box Testing
Black Box Testing examines the functionality of an application
(what the software does) without regard to its internal semantics
(how the program does what it does)
Analyze Input
Black Box
Analyze Output
Classic Example - Batch File Maintenance Testing
Run File Update Verify Changes
Program
Analyze
Data Files
QSAM
DB2 or IMS
to Data Files
QSAM
VSAM
5
DB2 or IMS
VSAM
White Box (Unit) Testing and zUnit
White Box Testing is a method of testing software that tests the internal
workings of an application, as opposed to its functionality or its outputs.
During White Box testing, code is run with preselected input values for
the validation of preselected output values.
Test Harness
Date Validation Routine
•
•
•
•
•
•
Initialize Street Address
Initialize City
Initialize State
Initialize Zip Code
Call Program using…
Evaluate test results
Unit Test…
Software Program
Address Validation Test
Internal Table Lookup
Address Validation
State Tax Calculation
IBM has implemented White Box testing as “zUnit”
6
Software Program
White Box Tests
zUnit and the xUnit Standard
 xUnit is an industry-standard, program testing framework - that helps create and
automatically run repeatable, self-checking unit test cases.
 xUnit provides:
 Assertion-style test validation capabilities and result reporting.
 Automated execution validation - instead of independent/interactive testing activity
 At run-time, the xUnit framework distinguishes between failures and errors:
 A failure is an anticipated problem (e.g., actual output does not match expected).
 An error is an unexpected, catastrophic problem (e.g., protection exception, null pointer)
 jUnit is an instance of the
xUnit architecture for Java
 zUnit is IBM’s implementation of the
xUnit architecture for System z
 zUnit provides the same
industry-standard output reports as jUnit
 You can use the same XML-reporting
dashboards and products that you
currently use with jUnit to review
and analyze results
7
Why bother to Unit Test?
By testing individual logic routines in your programs:
• You can move through the lifecycle more quickly, because you have
precise feedback about separate logic routines
•
So you can better understand cause & effect
•
And you know that your code works and you know how your code works,
which gives you confidence to make enhancements and modifications
• Because you execute zUnit Tests through JCL:
• The testing can be automated
• The end-to-end process takes less time than interactive debugging.
• And it can be more systematic
• By isolating and verifying code fragments Unit Testing allows
you to understand “cause & effect” in your program logic
All of the above benefits allow you to catch errors earlier in
the lifecycle…
8
The value of early and extensive testing
“80% of development costs are spent identifying
and correcting defects” **
Once released
into production
During the
Coding or
Unit Testing
phases
During the
BUILD phase
During
Quality Assurance
or the System Test
phases
$240/defect
$960/defect
$7,600/defect
+
Law suits, loss
of customer trust,
damage to brand
$80/defect
**National Institute of Standards & Technology
Source: GBS Industry standard study
Defect cost derived in assuming it takes 8 hours to find, fix and repair a defect when found in code and unit test.
Defect FFR cost for other phases calculated by using the multiplier on a blended rate of $80/hr.
9
What do you use to Unit Test?
Unit Testing is typically done using a product or a
framework that provides:
1. Standardized process, methodology and a component
framework (architecture)
2. Tools for code generation or some method of simplifying the
development Unit Tests
3. Automation - running scripts - or in the case of zUnit running JCL and not testing interactively
The industry leadership position in the Unit Testing space is
occupied by the xUnit component framework
• xUnit has been adopted by almost all of the major
computing languages/vendors/tools providers
• Including IBM
• Thus… zUnit
10
How do you Unit Test - with zUnit
Using zUnit you:
1. Develop Unit Tests for your programs
2. Execute the Unit Tests through JCL or interactively
3. Interpret/Analyze your Unit Test results
The Unit Tests you develop are COBOL programs or PL/I
procedures
• RDz generates 95+ percent of the COBOL and PL/I Unit Testing code
• RDz provides the test harness (Test Runner) as an installed run-time
• And RDz provides both an interactive and batch test execution workflow
The zUnit development process is:
• Wizard-driven
• Evolving - as customers adopt, and feedback next-generation
enhancements and suggestions
11
zUnit Support - as of RDz v9.1
• The call-able (zUnit targeted) program is a z/OS Load Module that
can be written in any z/OS language - including but not limited to:
 COBOL
 PL/I
 4GL z/OS applications, generated to; COBOL or PL/I; CA-Easytrieve, SAS, IBM Report
Writer, CA-Telon, NETRON/CAP™, CA-Meta COBOL, etc.
 Assembler (see slide notes)
(what this means…) You zUnit test at the Load Module level:
 The underlying language used to create the targeted business logic is irrelevant.
 Note that this is differs significantly from jUnit and the other xUnit-based
language-dependent frameworks, and product offerings
• In RDz v9.1 - zUnit supports Test Case + “stub module” generation,
plus automated testing and result XML generation for:
 COBOL batch call-able subroutines - with or without QSAM/VSAM and DB2 file I/O
 IMS and CICS are under development
12
zUnit Capabilities (High-level Feature/Function)
 zUnit Test Runner
• Runs on z/OS
• Installed and configured on z/OS as part of RDz Host install and customization
• Fetches and runs the Test Suite referred to in a zUnit configuration file
 zUnit Wizard used to generate Test Cases
• RDz client feature
• Eclipse based wizards allow creation of:
• Template Test Cases are generated in COBOL or PL/I
• Simple pass/fail assertion API
(RDz v9.1) Complete COBOL test cases:
• Identify the interface or set of copy book(s)
• Generate XML Schema to represent the interface
• Generate XML files where you would specify test input and expected output
• Generate a Test Case based on the XML file
• (Optionally) Generate stubs for called programs
 RDz viewers/editors for unit test XML results
13
zUnit/Unit Testing is Not a “Silver Bullet”
IBM recognizes this - and provides multiple testing solutions, each
optimized for different problem areas within the software testing
space:
Rational Function Tester
GUI
3270
Regression
Testing
Performance
Testing
Application Performance Analyzer
Interactive Testing
RDz Integrated Debugger
Integration Testing
IBM Debug Tool
Automated Unit Testing
zUnit
Test Driven Development
14
xUnit Components and Terminology
All xUnit frameworks share the following basic component architecture, with some varied implementation details.
Test Runner
A test runner executes test cases or test suites and makes the results available in some programmatically-consumable way (e.g.,
XML,...).
Test Case
Test cases execute within a test fixture and make assertions during test execution. Test cases contain any number of tests where
each test is a separate subprogram, function, method, or procedure in the test case.
Test Suite
Test suites define a set of test cases for test execution.
Assertions
Assertions verify the behavior and state of a test case. A failed assertion should result in an exception being thrown.
Test Fixture
A test fixture is a context and/or environment that the programmer creates (setup()) before a test in a test case is executed, and
discards (teardown()) immediately after the test is run.
Test Execution
Test execution involves invoking the Test Runner with input that includes, among other things, which test Cases or test suites
should be run.
Test Execution
Launches
Test Runner
Reads from
Test Results
(XML File)
Test Suite
Which identifies
There are hundreds of
distributed xUnit products and
implementations (jUnit, JSUnit,
NUnit, XMLUnit,etc.). They all
contain code frameworks in the
underlying supported language.
Many but not all are TAP
compliant.
Test Case
Which constructs
Test Fixture
Assertions
Which executes
15
Against…
Your program and the
program logic
to be tested
zUnit Component Implementation
RDz substitutes generated z/OS-specific resources for distributed technology classes, and frameworks.
A z/OS systems program developed by the RDz team, and installed with RDz. The Test Runner orchestrates the unit testing
process
Test Runner
Test Case
A generated COBOL or PL/I program compiled into a Load Module that calls your target Subroutine. Test Cases are executed
(Called) by the Test Runner
Test Suite
An XML file that provides a list of Test Cases to be executed by the Test Runner
Assertions
COBOL or PL/I conditions that test individual units of source code in your program
Test Fixture
A generated set of COBOL or PL/I programs that implement the xUnit framework standard; for setting up, running and tearing
down individual Unit Tests
Test Execution
z/OS JCL that invokes the Test Runner
Test Execution //JCL
Submit JCL
EXEC PGM=
Test Runner - RDz zUnit run-time
OPEN INPUT
READ Test Suite
Test Suite - List of Test Cases (Load Modules)
Which identifies
Test Case Load Module
Test Results
(XML File)
Test Case - Generated COBOL or PL/I program
The Test Runner CALLs (interacts with)
Test Case Load Module
Which contains Test Fixtures
(COBOL subprograms or PL/I procedures
Each Test Fixture sets up and CALLs
(executes) individual Unit Tests and
releases unit test resources (Tear Down)
Test Fixture - Generated COBOL or PL/I programs
Assertions - Conditional Expressions
coded in COBOL or PL/I
Makes
16
Your program and the
logic to be tested
zUnit Test Case Development Wizard
RDz provides a wizard-driven process to create your Test Cases and Test
Fixture programs
17
zUnit Test Case Module
zUnit Generated Test Case
USER.ZUNIT.LOAD(UNIT0001)
UNIT0001(...)
The Test Case is a generated COBOL
or PL/I module that contains:
• Nested programs with standard xUnit names
and functionality
• 1  M generated Test Fixtures (programs)
- One program for each of the Tests
defined using the wizard
ADDTESTS(...)
SETUP(...)
TEARDOWN(...)
TEST1(...)
......
TESTn(...)
You customize the Test Case
programs, and add statements
specific to your business logic and
testing requirements:
• Initialize test input variables
• Evaluate test outcome
RDz’s zUnit framework
takes care of the rest
18
Agenda
Topics:
 Unit Testing Value Proposition,
zUnit Terms, Concepts and Vocabulary
 Demonstration
 Next Steps
19
Demonstration
What you will see:
1. Develop Unit Tests for your programs
•
Use the zUnit Automated Testing
Framework to generate a Test Case
•
•
Which includes Test Fixtures that contain
Assertion logic that unit tests your program
Compile & Link the Test Case
2. Execute the Unit Test
3. Interpret/Analyze results
20
http://www.youtube.com/watch?v=vjNY7SMtVqU&list=UUdgVL1ysbledgitPTYKyJuQ&index=1&feature=plcp
IBM Rational Developer for System z version 8
21
21
Review - What you Saw - zUnit Run-time
Test Execution
JES
//
Submit JCL…
Launches…
EXEC PGM=Test
Runner
//AZUCFG  Test Suite
//AZURES  Test Results
//…Test Case Load Library
zUnit Test Runner
Test Suite
//AZUCFG DD DSN=…
• Names the Test Case module(s)
(Test Harness)
READ Test Suite
CALL (invoke) Test Case
Test Case
Invoke Test Fixture
Interact with different nested COBOL
subprograms or PL/I procedures within your
Test Case to:
- Setup unit test variable values
- Run a test.
- Evaluate the Assertion outcome
- Release system resources
Generate Results File
Test Fixture
-
Target Subroutine
Test Results
z/OS
SETUP Linkage Variables
Run Unit Test Assertion Logic (CALL)
Evaluate outcome
Release resources
Load Module to be unit tested
//AZURES DD DSN=…
• Stores test results in XML PDS member
22
• Testable logic routine
• Testable logic routine
• Testable logic routine
Agenda
Topics:
 Unit Testing Value Proposition,
zUnit Terms, Concepts and Vocabulary
 Demonstration
 Next Steps
23
Review - zUnit Test Case Development Wizard
RDz provides a wizard-driven process to create your Test Cases and Test
Fixture programs
24
Review - xUnit Vocabulary / zUnit Implementation
 Test Execution: JCL that invokes the Test Runner (a “supervisor program” or
“test harness” that runs your Test Suite)
 Test Runner: The zUnit “supervisor program” invoked by Test Execution, that:
 Reads a Test Configuration Test Suite
 Calls your Test Cases
 Test Suite: an XML configuration file that defines which Test Cases the Test Runner
should execute (Note - a Test Case == a Load Module)
 Test Case: An RDz-generated COBOL or PL/I program that:
 Is called by the Test Runner
 Uses a Test Fixture to Call your Subroutine and write test outcomes to a Test Results file
 Test Fixture: A set of generated programs nested inside your Test Case. Test Fixtures
run Tests that utilize Assertion logic to Call your Subroutine, and evaluate the result.
 Subroutine: The target call-able program to be Unit Tested
 Test Results: An output dataset that describes the results of each test run
in xUnit standard (XML) file format
 Assertion: A simple pass/fail predicate used to test one part of your Subroutine’s logic
** Test Execution, Test Runner, Test Suite, Test Case, Test Fixture, Assertion == xUnit vocabulary
25
26
Links - to additional Unit Testing content
http://lisacrispin.com/wp-content/uploads/2011/11/Agile-Testing-Quadrants.png
http://1.bp.blogspot.com/-RQLtpTssY_o/UZ9CYqzflqI/AAAAAAAAAyo/9kIx6aGwSaU/s1600/TestingTrianglePished.png
http://en.wikipedia.org/wiki/White-box_testing
http://en.wikipedia.org/wiki/Black-box_testing
http://en.wikipedia.org/wiki/XUnit
http://en.wikipedia.org/wiki/Test-driven_development
27