What went Wrong?
Sprint 6 Customer Review
Ray Arell
Sr. Engineering Manager, Intel
Corporation
ray.arell@intel.com
1
2
Career user of the waterfall product
life cycle…
Prior owner of the platform level
waterfall corporate specification…
Contributor to a number of other
waterfall standards…
Even spoke at conferences
about my brainchild the
“Framework of Quality”!
3

My
team
Moore's Law states that the number of
transistors on a chip doubles about
every two years.
©2005 Intel Corporation
Flexibility
Customer
No!
Exploration
No!
Development
Planning
No!
Production
3-6 Months
“The basic framework described in the waterfall model is risky and
invites failure.”–Winston Royce, creator of the original waterfall model
7

8
Focuses on delivering high
customer value with every release
Work culture that promotes:
Teamwork
Just enough process to get stuff done
Frequent customer feedback
High level of empowerment
Welcomes change and allows the
product to evolve to meet the
customer’s needs
9
Cowboy Coding
Ad hoc processes focused on doing
things people want to do vs. need to do.
Wagile
Doing short waterfall delivery
and calling it Agile
10
11
12
Scrum project framework puts three major constraints on
testing
Products are delivered on a fixed cadence and cannot be pushed out
All features need to be working and meet the acceptance criteria
No features are shipped to the customer if it is not tested, repaired,
and retested
User stories/features by design are expected to evolve
Details and acceptance criteria in the backlog will evolve over time
May be deferred until the maximum amount of information is available
Development of the product itself may fill in the gaps
Customers may shift priorities
They are the customer after all!
13
Evolve vs. cold turkey
A small co-located team can move faster
Train your team prior to starting!
Scrum Master certified, Product Owner and team trained
Major paradigm shift for everybody
14
"A paradigm shifting without a clutch.“
--Dilbert, 25 Aug 1995
15
Agile
frAgile
ScrumFalls
1
5
10
Sprint
16
15
20
Effect on the Team
Role of management
Job titles and trust
Shifting to self-managed
The role of validation
Perception of micromanagement
Effect on the Customer
Timid
Level of involvement
Misconceptions of the process
Fine Tuning the Process
Getting “Done” defined correctly
Interfacing with non-scrum teams
Cross-site/geo communications
How Validation/QA should fit
17
Focus of the Stand-up
Inform, Commit to Peers, Ask for Help
What Went Wrong...
PO’s acting like
administrators
Accountability
Embarrassment
Scrum Master
needing to be a
stronger gate keeper
18
Stress, Anger, and Fear
Test strategy needed to change
Working integrated
Testing, debug, and retest faster
Dealing with requirement changes
19
Employee Burnout
Scrum
Waterfall
Ambivalent
17%
50%
33%
Source: Internal team survey
“Chaos in the world brings uneasiness, but it
also allows the opportunity for creativity and
growth.”
-- Tom Barrett
21
Management
22
23
24
Edward
Application Engineer
“There is no ‘One Size Fits All’ with Independent Software
Vendors (ISVs). Every ISV has a different environment,
architecture and customer needs.”
Ed has been working in this role for 4 years. He was a SW Engineer before that
for 10 years. His work focuses on the implementation of AMT capabilities. Ed’s
primary role is assisting ISV engineers in implementing specific features by
customizing a solution for their given environment. Much of his day is spent
troubleshooting issues and writing new code to test. Usually, Ed travels to the
ISV and spends time face to face working with their implementation team. Given
the economic climate, he has made changes to the way he interacts with his
customers. Most of his interaction with ISVs is done over the phone and via
email.
Goals:
• Simplify integration for ISV partners
• Solve issues fast
• Demonstrate AMT value
Obstacles:
• Each ISV requires custom solution
• Troubleshooting issues remotely
• Translating code to meet ISV needs
Values:
• Good customer relationships
• Flexible architecture
• Good documentation
Design Implications:
• Design should demonstrate how a feature
could be integrated--represent believable user
experience
• Vanilla design = palatable for all potential
customers
• Design should avoid appearing as a competing
product

27
Team Collaboration
Scrum
Waterfall
Ambivalent
0%
5%
95%
Source: Internal team survey
Team Cooperation
Scrum
Waterfall
Ambivalent
8%
15%
77%
Source: Internal team survey
Challenging Work Environment
Scrum
Waterfall
Ambivalent
0%
10%
90%
Source: Internal team survey
Overall Satisfaction
Scrum
Waterfall
Ambivalent
10%
25%
65%
Source: Internal team survey
“…So I was driving home and contemplating 9 months
of pregnancy, or 40 weeks. And I thought, well that
would only be 20 2 week sprints, or 10 4 week
sprints!” – one of my employees
32
You need to iterate your processes just
like your product
Don’t fall into Wagile or let cowboy
coding take over
Focus your test effort on weeding out
things that would create a bad user
experience with your product
Don’t stress on the change
33
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Various Authors, Exploratory Testing, Wikipedia
Various Authors, Test Strategy, Wikipedia
Various Authors, Scrum (development), Wikipedia
Various Authors, Session-based testing, Wikipedia
The Scrum Alliance, http://www.Scrumalliance.org/
Ray Arell, Change-Based Test Management, (ISBN: 0971786127)
James Bach, Heuristic Risk-Based Testing, STQE 11/99
James Bach, Risk and Requirements-Based Testing, Computer, June 1999
Ingrid Ottevanger, A Risk-Based Test Strategy, StarEast 2000
Bret Pettichord, The role of information in Risk Based testing, StarEast 2001
James Bach, Risk-Based Testing Troubleshooter, Paper Draft
Erik Petersen, Smarter Testing with the 80:20 Rule, StarWest 2002
Anne Campbell, Using Risk Analysis in Testing, StarEast 2000
Paul Gerrard and Neil Thompson, Risk-Based E-Business Testing
Gregory T Daich, Defining a Software Testing Strategy
Jim Highsmith, Agile Project Management
Ruku Tekchandani, Building a Effective Test Strategy
John Pruitt and Tamara Adlin, The Persona Lifecycle
Pettichord, Kaner, Bach, Lessons Learned in Software Testing, on-line
Jonathan Bach, Session-Based Test Management , http://www.satisfice.com/articles/sbtm.pdf
Daniel Pink, Drive the Surprising Truth about What Motivates Us, (eISBN: 97811001152140)