The Marriage of ET and Agile
Kevin Malley
Oct 171th, 2013
Preamble
We aren’t “playing around”
It wasn’t found by accident
My Goal:
Help you understand and explain exploratory testing as a
thoughtful, purposeful approach whose results stand up under
scrutiny. And why it’s needed in an Agile environment
#TQ2013
2
Agenda
•
•
•
•
•
•
•
•
Agile & ET comparison
Why ET
What we need
Accountable ET
Agile testing & CSI
Other forms
Immediate & future benefits
Conclusion
#TQ2013
3
Agile & Exploratory Testing (ET) in
One Slide
• Getting the right stuff done at the right time
with near immediate value to the customer
#TQ2013
4
Each Produces:
•
•
•
•
Rapid feedback
Emphasizes responding to change
Requires learning
Are collaborative
Jon Bach pronounced them “married” at the
Agile 2010 conference
#TQ2013
5
Agile Expanded
Agile Manifesto provides the guiding principles for
agile development.
• Individuals and interactions….Over process and tools
• Working software………………..Over comprehensive
documentation
• Customer collaboration………..Over contract negotiation
• Responding to change………….Over following a plan
#TQ2013
6
Agile Method’s Don’t Matter
#TQ2013
7
ET Expanded
• Exploratory Testing is simultaneous test design, test
execution, and learning.
- Broad consensus from the participants of the Exploratory
Testing Research Summit
• “Test Driven Testing – using feedback from the last
test to inform the next.”
-Elisabeth Hendrickson, Quality Engineer, Pivotal Labs
#TQ2013
8
Why Use ET
• It’s fast
• The focus is on finding bugs
• It’s useful for augmenting other forms of testing
• It’s fun!
Unscripted doesn’t mean unprepared.
It’s about enabling choice, not constraining it.
-- Cem Kaner, Professor of Software Engineering, Florida
Institute of Technology
#TQ2013
9
Exercise (ebay Search)
• What is the most expensive item on ebay?
• What is trending on ebay?
• How many items are there on ebay?
• What is the most viewed item on ebay?
• What is the all-time, most popular item on ebay?
#TQ2013
10
What We Need Is
Decreasing Risk
Improving Quality
#TQ2013
11
What We Need in Words
A constantly evolving set of tests that exercise the
software in new ways (new combinations of features
and data) so that we get our choice of:
• Broader coverage of the infinite space of
possibilities, adapting as we recognize new classes
of possibilities
• Sharper focus on risks or issues that we decide are
of critical interest today
For THAT we do
Exploratory Testing
#TQ2013
12
Accountable ET
• Pure Exploratory Testing is difficult to manage.
• Need a process that is light enough to allow the
flexibility of ET, but allows you to track and report on
testing.
• A number of ways to manage session based testing.
#TQ2013
13
Accountable ET - Structure
The Session:
• Time Box
• Reviewable Result
• Debriefing
#TQ2013
14
Agile Testing is a lot Like CSI
• A lot is happening in a short amount of
time
• Things change rapidly
• Many tools, procedures, sources of
evidence.
• Tools and procedures don’t define an
investigation or its goals.
• There is too much evidence to test, tools
are often expensive, so investigators must
exercise judgment.
• The investigator must pick what to study,
and how, in order to reveal the most
needed information
How effective do you think they would be
if they only followed a script?
#TQ2013
15
Why Not Other Forms of Testing?
The Freedom Scale
They have their place but knowing when is key
#TQ2013
16
Other Forms - Scripted Testing
Automation
• Do early – factor it into the development
• Do consider ROI – not everything is worth the
cost
• Throw away automation is ok
Manual Scripts
• Useful where automation is too expensive
• Useful where a defined sequence must be
executed
#TQ2013
17
Immediate & Future Benefits
Use ET to provide rapid feedback to:
• Developers – Test Early
• Customer
• Production Support Team
• Use to form the purpose of the next test
• Use the session notes to build scripted test
cases
#TQ2013
18
Conclusion
• Type of Agile method used is not important
• ET is a cost effective adjustable tool with rapid
response
• Session based testing enables ET to stand up
under scrutiny
• Learn from the last test or sprint and apply what
worked to the next test or sprint and tweak what
didn’t.
• ET & Agile need to have an open marriage with
Automation
#TQ2013
19
Questions
#TQ2013
20
Contact Info
Kevin Malley :
kmalley@blackberry.com
twitter: @mxz905
#TQ2013
21
Appendix
Links
•
•
•
•
Cem Kaner http://www.kaner.com
James Bach http://www.satisfice.com
Michael Bolton http://www.developsense.com
Rapid Software Testing
http://www.satisfice.com/info_rst.shtml or
http://www.developsense.com
• Ministry of Testing (A project by the Software
Testing Club) – Everything Exploratory Testing
Related
http://www.ministryoftesting.com/resources/explor
atory-testing/
#TQ2013
22