Software Development Conference 2007
IBM RATIONAL TOOLS
FOR TESTING
Presentation: Vítek Linduška
Demo: Václav Novotný
Software Development Conference 2007
Agenda
 Manual Tester
 Functional Tester
 Performance Tester
 ClearQuest
2
Software Development Conference 2007
Manual Tester
3
Software Development Conference 2007
Why Teams use Manual Testing?
Recruitment of subject-matter
experts and end-users to test
Strict regulatory or process
requirements aren’t met with
automation tools
Poor or non-existent test
automation-ready interfaces
4
Software Development Conference 2007
Challenges of Manual Testing
 Labor Intensive
 Too much time is spent creating
and maintaining manual test
documentation
“More time is spent
maintaining test
documentation than on
executing tests.”
“We struggle to
document even a
minimum set of tests.”
5
Software Development Conference 2007
Challenges of Manual Testing
 Labor Intensive
 Too much time is spent creating
and maintaining manual test
documentation
“Our results are
unreliable due to
human error.”
 Unreliable
 Inadvertent human-error often
skews test results
“Our tests aren’t
detailed enough for the
testers to consistently
execute them.”
6
Software Development Conference 2007
Challenge 1: Labor Intensive
 Challenge
 Creating and maintaining manual tests is labor intensive and difficult
with tools such as Microsoft Word or Excel.
 Resolution
 Implement a solution that
promotes the best practice of
modular tests which can share
common content
 Leverage existing Word or
Excel test documents to quickstart test development
Share
test
content
7
Software Development Conference 2007
Challenge 2: Unreliable
 Challenge
 Human error often skews tests results
 Resolution
 Assist testers with the tasks most
prone to error – data entry and
data verification
 Build manual tests that are clear,
concise and easy for the tester to
follow
8
Software Development Conference 2007
Leverage Existing Test Documents
Import from Word or Excel
Microsoft Word
documents
 Jump-start adoption of Rational Manual
Tester by leveraging existing assets
Rational Manual
Tester test
 Bulk import manual tests documented in
Word or Excel
Microsoft Excel
spreadsheets
9
Software Development Conference 2007
Assisted Data Entry and Data Verification
Reduces human error
 Store critical data used during the test as part of the test instructions
Test execution window
10
Software Development Conference 2007
Assisted Data Entry and Data Verification
Reduces human error
 Store critical data used during the test as part of the test instructions
 Automate data entry during test execution to speed entry and reduce errors
Test execution window
Application under test
11
Software Development Conference 2007
Assisted Data Entry and Data Verification
Reduces human error
 Store critical data used during the test as part of the test instructions
 Automate data entry during test execution to speed entry and reduce errors
 Automate data validation during test execution to ensure accurate results
12
Software Development Conference 2007
Enables Shared Content Across Multiple Tests
Reduces test maintenance and ensure consistency
 Create commonly used test statements
once
 Reuse statements across multiple tests
through simple drag and drop
 Maintenance of shared content is
efficient with single-point updating of
shared content
 Ensures consistency and reduces test
maintenance
13
Software Development Conference 2007
On-Screen Prompted Test Execution
Immediate and accurate results collection
 On-screen prompting of test steps
 Immediately record test results and attach images or files for clarity
 Tool will assist with data entry and data verification
 Results recorded in permanent log
14
Software Development Conference 2007
Spread-Sheet Ready Test Results
Use with any 3rd party reporting tool
 Results easily exported to
common file format
 Import into any 3rd party tool
for evaluation and
consolidation with other test
data
15
Software Development Conference 2007
Functional Tester
16
Software Development Conference 2007
Challenge of Manual Testing with Short Test Cycles
Tests
Rerun the
first tests
and...
Manual testing
can’t keep pace!
...the new
tests...
...take
more
time
Time
Build 1
Build 2
Build 3, 4, 5 …
 Manual testing alone can’t keep pace with the
rapid development of applications
 Quality will be sacrificed to keep on schedule
Software Development Conference 2007
Challenges of Functional Testing
 Controlling cost of test automation
 Inexperienced testers face steep
learning curve for tools
“Our testers are not
experienced in building
automated tests.”
 Test maintenance outpaces testing
“Application UI
changes are always
breaking our test
scripts.”
18
Software Development Conference 2007
Challenges of Functional Testing
 Control cost of test automation
 Inexperienced testers face steep
learning curve for tools
“Our script language
limits our ability to
overcome some
automation obstacles.”
 Test maintenance grows with
project
 Testing Tools Lack Flexibility
 Power users hands are tied by tool
limitations
“I need one tool for
both my expert and
novice testers.”
19
Software Development Conference 2007
Challenges of Functional Testing
 Control cost of test automation
 Inexperienced testers face steep
learning curve for tools
“How do I keep my
team in connected with
the project team?”
 Test maintenance grows with
project
 Testing Tools Lack Flexibility
 Power users hands are tied by tool
limitations
 Test Team Integration
 Managing script changes with
parallel development
 Sharing test assets with the
larger software team
“We’re geographically
distributed and we’re
worried about parallel
script development .”
20
Software Development Conference 2007
Challenge 1: Controlling the cost of Automation
 Challenge
 Overcome the complexity of test automation to ensure teams spend more
time on high value quality assessment rather than on low return test
maintenance.
 Resolution
 Provide technologies to enable
novice testers to craft complex
tests with no programming
 Data driven wizard
 Dynamic data verification
 Reduce script maintenance
 Wizard driven global updates to
centralized object map
21
Software Development Conference 2007
Challenge 2: Tools Lack Flexibility
 Challenge
 Fulfill both standard test needs as well as expert
requirements to ensure usage limited only by the imagination of the project
team.
 Resolution
 Provide two test script
development languages
 Java
 Visual Basic .NET
 Give professional testers the
flexibility to overcome testing
challenges in the language of
their choice
22
Software Development Conference 2007
Challenge 3: Test Team Integration
 Challenge
 Accommodate the requirements of both dedicated,
independent QA teams as well as integrated project teams to
ensure high operational integrity.
 Resolution
 Provide version control
support in the box with
ClearCase LT
 Provide integrations for
standard version control
features
 Check In/Out
 Compare with
Previous Versions
 Show change history
23
Software Development Conference 2007
Reduce Test Script Maintenance
Reliable Playback with Script Assure
Version 1.0
Version 2.0
Tester
Sees
No User
Intervention
Required
With
Tool
TM
Sees
ScriptAssure
Determines
Match
24
Software Development Conference 2007
Performance Tester
25
Software Development Conference 2007
Challenges of Performance Testing
 Complex Tools & Complex Task
 Tool complexity and lack of
experience intimidates many first
time users
“We don’t have a lot of
experience building
good load tests.”
“Most load test tools
are much more
complicated than we
require.”
26
Software Development Conference 2007
Challenges of Performance Testing
 Complex Tools & Complex Task
 Tool complexity and lack of
experience intimidates many first
time users
“We don’t have a good
understanding of what
exactly is occurring
during a load test.”
 Testing Tools Lack Flexibility
 Power users hands are tied by tool
limitations
“We have highly
technical performance
testers that need to
customize their tests.”
27
Software Development Conference 2007
Challenge 1: Complex Tools & Complex Task
 Challenge
 Tool complexity and lack of experience intimidates many first time
users
 Resolution
 Remove the need for
programming
 Tests are represented in a tree
view as a sequential flow web
pages
 User interface enables
 Data pooling
 Data correlation
 Looping
 Conditional events
28
Software Development Conference 2007
Challenge 2: Tools Lack Flexibility
 Challenge
Power users hands are tied by tool limitations
 Resolution
 Tests may call custom Java code
provided by the user
 Offers flexibility for power users
to perform complex tasks
 Expose behind the scenes
details which are hidden from
novice users by default
 Provide the flexibility to
overcome any testing obstacles
29
Software Development Conference 2007
Performance Problem Identification During Test
Page Performance Report
Shows average response time
per page.
Highest bar = Performance Problem
 Performance Testing finds bottlenecks
 Next logical question is Why?
 Root Cause Analysis provides to tools to answer this question
30
Software Development Conference 2007
Performance Problem Identification During Test
 Performance Testing finds bottlenecks
 Next logical question is Why?
 Root Cause Analysis provides to tools to answer this question
31
Software Development Conference 2007
Response Time Breakdown
Response data broken down
by tier and by transaction
component (JDBC, JSP,
Servlet, etc…)

Feature:
 Breakdown page response times into composite element response time

Benefit:
 Breakdown data drills down to highlight slowest components of a given page.
Breakdown into tiers and components is a unique competitive advantage for RPT.
32
Software Development Conference 2007
Deep Diagnostic Data
Providing the details you need to solve the performance problem mystery
Execution Statistics
Shows response time for
every method
UML Sequence diagram
showing method calls between
classes with timing
information. User can drill
down to source code.
33
Software Development Conference 2007
ClearQuest
34
Software Development Conference 2007
Test Management with ClearQuest
Eclipse, Windows and Web Clients for test project tracking
 Eclipse Based Test Management
 Joins Functional and Performance
Tester to complete transition of testing
tools to eclipse platform
 Native and web clients also available
 Key Functional Areas
 Test Planning

Create test plans, test cases and
configured test cases
 Test Execution

Create suites or run individual
test cases
 Test Reporting

Extensive query and reporting
facilities to measure test progress
35
Software Development Conference 2007
Lifecycle Traceability
Query one source for all aspects of project progress
Requirements
Tests
Defects
 Query to find
 Test results associated to requirements
 Defects associated to test results
 Defects associated to tests associated to requirements
 Traceability from requirements to tests and defects closes the loop
on the software development cycle
36
Software Development Conference 2007
Lifecycle Traceability
Query one source for all aspects of project progress
Leverage existing reports
or
Create new reports with Crystal Reports
To provide the information you need for
decision making
Test Verdicts
Test Case
Trending
37
Software Development Conference 2007
ClearQuest Web Interface
Distributed access for test planning and reporting
 Instant access for remote users
 Access for alternative platforms
 No additional licenses required
 Perfect for test planning and
reporting
38
Software Development Conference 2007
ClearQuest MultiSite
Replication and synchronization for geographically distributed teams
 Automatic replication and
synchronization of ClearQuest
repositories
Access current information,
regardless of location
`
Automatic recovery of
repositories in the event of
system failure maintains data
integrity
Transmission of only
incremental changes for
improved network performance
39
Software Development Conference 2007
Centralized User Management
LDAP authentication for common user id support
Reduce administration with
centralized user management –
maintain only one list of user ids
LDAP authentication supports
common user ids for ClearQuest
and RequisitePro
Rational
ClearQuest
Authorization
LDAP
LDAP 3.0
Compliant Server
Rational
RequisitePro
Authorization
Rational
ClearQuest
Rational
RequisitePro
Rational ClearQuest User DB
Logon Dialog
40
Software Development Conference 2007
Compliance and Control
Using Audit Trails and eSignatures to track change
 Audit Trails can track changes to
individual records
 Provides immutable change log of all
record change activity
 e-Signature requirement can be added
to any event on any artifact
 Will record who changed a record, when
and how the record was changed
 e-Signature provides immutable evidence
of process completion
e-Signature Approval
Requirement
Test Case
Test
Execution
Validate
Results
Approve
41
Software Development Conference 2007
Configure and Enforce Your Process
Customize appearance and behavior of all forms and processes
 Customize everything
 Modify forms
 Configure State Transition
 Add fields and control behavior
 Extend your system with custom
coded hooks
 Extend functionality using
pre-defined Packages
 eSignatures
 Audit Trail
 Many, many more…
42
Software Development Conference 2007
Extensible Test Ecosystem
Integrated with Rational, open source and 3rd party test automation tools
 Collect and report on results from
Rational test execution engines
 Rational Manual Tester
 Rational Functional Tester
 Rational Performance Tester
 Collect and report on results from Eclipse
Test and Performance Tools Project
 jUnit tests
 Manual tests
 URL tests
 Open API Extensibility
 SPI Dynamics QAInspect
43
Software Development Conference 2007
Rational Software Quality Solutions
BUSINESS
SOFTWARE QUALITY SOLUTIONS
Requirements
Test
Change
Defects
Rational
RequisitePro
Rational
ClearQuest
Rational
ClearQuest
Rational
ClearQuest
Test Automation
Functional Test
Developer Test
Rational Functional Tester Plus
Rational PurifyPlus
Rational Test
RealTime
Performance Test
Automated
Rational Functional
Tester
Manual
Rational
Performance Tester
OPERATOINS
DEVELOPMENT
Test and Change Management
Rational Manual
Tester
Rational Robot
Quality Metrics
Project Dashboards
Detailed Test Results
Quality Reports
44
Software Development Conference 2007
WAKE UP, IT’S
DEMO TIME!
45