1
ASTQB Webinar Series
ISTQB and Test Automation
How ISTQB Certification Prepares you
for Automated Testing
Presented by
ANDREW L POLLNER
Vice President, ASTQB
Chair, ISTQB Expert Level Test Automation Working Group
Introduction
ISTQB and ASTQB
Importance of Automation in Testing
Automation Topics in ISTQB Syllabi
Business Outcomes
Expert Level Syllabi
Copyright © 2014 ASTQB
Slide 3
WORLD-WIDE COVERAGE
ASTQB


Copyright © 2014 ASTQB
Countries covered by Member Boards
Countries covered by Exam Providers
Slide 4
Goals for Automation in Testing
o Fast execution of tests
o Repeatable process
o Increased functional coverage
o Lots of variations and permutations
o Automatic reporting of errors
o Low maintenance requirements
o Make test team more productive
o Stay on time and on budget
Copyright © 2014 ASTQB
Slide 5
Body of Knowledge (BOK)
What Documents Cover Automation?
o Foundation Level
o Advanced Level Test Analyst
o Advanced Level Technical Test Analyst
o Advanced Level Test Manager
o Expert Level
o Glossary
6
Copyright © 2014 ASTQB
Slide 6
Foundation Level
Chapter 6 – Tool Support for Testing:
o Types of Test Tools
o Effective use of Tools
o Introducing a Tool into an Organization
Copyright © 2014 ASTQB
Slide 7
Advanced Level Test Analyst
Chapter 7 – Test Tools:
o Test Data Preparation Tools
o Automated Test Execution Tools
o Test Automation Implementation
o Improving the Success of the Automation Effort
o Keyword-Driven Automation
o Causes for Failures of the Automation Effort
Copyright © 2014 ASTQB
Slide 8
Advanced Level Technical Test Analyst
Chapter 6 – Test Tools and Automation:
o Integration and Information Interchange
Between Tools
o Defining the Test Automation Project
o Selecting the Automation Approach
o Modeling Business Processes for Automation
o Specific Test Tools
o Tools for Web-Based Testing
Copyright © 2014 ASTQB
Slide 9
Advanced Level Test Manager
Chapter 6 – Test Tools and Automation:
o Tool Selection
o Selection Process
o Return on Investment (ROI)
o Tool Lifecycle
o Tool Metrics
Copyright © 2014 ASTQB
Slide 10
Expert Level Test Automation
Two Modules:
o Test Automation Engineering
o Test Automation Management
Copyright © 2014 ASTQB
Slide 11
Business Outcomes – TAE
o Contribute to the development of a plan to integrate
automated testing within the testing process
o Evaluate tools and technology for automation best fit to
each project and organization
o Create an approach and methodology for building a test
automation architecture
o Design and develop new or modified test automation
solutions that meet the business needs
o Create automated test reporting and metrics collection
o Manage and optimize testing assets to facilitate
maintainability and address evolving test systems
Copyright © 2014 ASTQB
Slide 12
Business Outcomes – TAM
o Develop a test automation policy and strategy appropriate for
the organization
o Create a business case which outlines the costs and benefits
to be expected from introducing and sustaining test
automation
o Develop a plan to integrate automated testing within the
testing process
o Oversee the evaluation of tools and technology for
automation best fit to each project and organization
o Manage the test automation resources to implement and
sustain an effective test automation program
o Define and gather metrics to effectively report progress of
the test automation program
o Apply findings to improve the test automation process
Copyright © 2014 ASTQB
Slide 13
Test Automation Engineering (TAE)
In Scope
oTasks performed by a test
automation engineer
oDesigning, developing and
maintaining concrete test
automation solutions
oConcepts, methods, tools, and
processes
Out of Scope
oTest Management
oAutomation of non-functional
tests
oTeaching of software
engineering methods and
programming
oTeaching of software
oAutomating dynamic functional technologies
tests
oSelection of software testing
products and services
Topics in TAE
o
o
o
o
o
o
Introduction and Objectives for Test Automation
Preparing for Test Automation
The Generic Test Automation Architecture
Deployment Risks and Contingencies
Test Automation Reporting Metrics
Transitioning Manual Testing to an Automated
Environment
o Verifying the TAS
o Continuous Improvement
Copyright © 2014 ASTQB
Slide 15
SDLC processes for SUT and TAS
Copyright © 2014 ASTQB
Slide 16
K LEVELS: Syllabus & Exam
Questions are classified according to the cognitive level, the Klevel (also known as level of knowledge):
K1 – Remember (recognize a term or concept)
K2 – Understand (able to explain a concept)
K3 – Apply (select correct application of concept or technique)
K4 – Analyze (can distinguish between facts and inferences for deeper
understanding)
K5 – Evaluate (can make judgments based on criteria and standards)
K6 – Create (can put elements together to form a coherent or functional
whole)
The number of questions for each topic is related to the length
of the topic in the syllabus
Copyright © 2014 ASTQB
Slide 17
Learning Objectives (Chap 1-2)
1.1 Purpose of Test Automation
ELTA-E-1.1.1 (K2) Explain the objectives, advantages
and limitations of test automation
2.1 Test Automation Requirements for the SUT and
its Context
ELTA-E-2.1.1 (K4) Analyze a system under test to
determine the optimal automation solution
Copyright © 2014 ASTQB
Slide 18
Learning Objectives (Chap 3-4)
3.2 TAA Design
ELTA-E-3.2.1 (K4) Design the appropriate TAA for a given
project
3.3 TAS Development
ELTA-E-3.3.1 (K6) Construct a purpose-built TAA based on
the generic TAA (gTAA) using predefined components
4.3 Technical Risk Assessment and Mitigation Strategies
ELTA-E-4.3.1 (K6) Devise mitigation strategies to address
technical risks, including framework approach, that could
lead to failure of the test automation project
Copyright © 2014 ASTQB
Slide 19
Generic Test Automation Architecture
Copyright © 2014 ASTQB
Slide 20
Learning Objectives (Chap 5-6)
5.3 Logging of the TAS and the SUT
ELTA-E-5.3.1 (K4) Implement logging of test
executions and test results (of both TAA and SUT
data)
6.1 Criteria for Automation
ELTA-E-6.1.1 (K4) Analyze factors relevant to
determining the appropriate criteria for suitability of
tests for automation
Copyright © 2014 ASTQB
Slide 21
Learning Objectives (Chap 7-8)
7.1 Verifying Automated Test Environment
Components
ELTA-E-7.1.1 (K3) Apply validity checks to the
automated test environment and test tool setup
8.2 Planning the Implementation of Test Automation
Improvement
ELTA-E-8.2.1 (K4) Analyze the test environment
components, including tools, in order to understand
where consolidation and updates need to be made
following a given set of test environment changes
Copyright © 2014 ASTQB
Slide 22
Test Automation Management
Tentative Topics:
o Test Automation Policy
o Test Automation Strategy
o Managing Test Automation
o Test Automation Resources
Copyright © 2014 ASTQB
Slide 23
ISTQB LEVELS AND SYLLABI
Copyright © 2014 ASTQB
Slide 24
Syllabus Released
Exam in Progress
Syllabus in Progress
Certification Prerequisite
ISTQB CERTIFICATES ISSUED
Copyright © 2014 ASTQB
Slide 26
Conclusion
o Test Automation is more similar to software
development than to testing
o ISTQB syllabi provide a wealth of information
on Test Automation topics and a forthcoming
certification exam
o Certification criteria:
o 7 Years of practical testing experience
o Foundation Level Certification
o Advanced Level TTA Certification (for TAE)
Copyright © 2014 ASTQB
Slide 27
Questions?
Copyright © 2014 ASTQB
Slide 28