Improve Testing, Improve
Software
Paul Gerrard
Gerrard Consulting
1 Old Forge Close
Maidenhead
Berkshire
SL6 2RD UK
e: paul@gerrardconsulting.com
w: http://gerrardconsulting.com
t: 01628 639173
Slide 1
How to improve…
I want to improve my (insert any activity here)
 _______ people improvement
 _______ organisation improvement
 _______ process improvement




Changing people (like me) and
organisation (like my company) is so
hard – let’s not even think about it
The delusion of ‘best practice’




There are no “practice” Olympics to determine
the best
There is no consensus about which practices are
best, unless consensus means “people I respect
also say they like it”
There are practices that are more likely to be
considered good and useful than others, within a
certain community and assuming a certain
context
Good practice is not a matter of popularity. It’s a
matter of skill and context.
Derived from “No Best Practices”, James Bach, www.satisfice.com
The delusion of quality
management
(a la ISO 9001)

Google search for

ISO 9001 in a nutshell

“ISO 9001” – 14,500,000
“ISO 9001 Training” – 13,100
“ISO 9001improves quality” – 1
“Document what you do”
“Do what you document”
“Do what you like”
ISO 9001 accreditation without reference to
the garbage that is produced by the
documented processes.
The delusion of process models
(e.g. CMM)

Google search
- “CMM” – 7,260,000
- “CMM Training” – 14,300
- “CMM improves quality” – “1-6 of about 7”

Typical CMM user…
- CMM level 3 and proud of it
- Overall s/w quality, time to deliver is slipping
- Changes to be made are obvious
- But few can be adopted – they can’t change because
-
they’ll lose their CMM status
What’s going on here?
People need process?



A big problem with process is it becomes all
encompassing
Process folk sell process and cast all things in terms of it,
forgetting that people who are smart, succeed in spite of
process, not because of it
It could be argued, that less smart people need process
- (By less smart, we're talking about people who need so much
structure and enforced discipline they can only operate in the
military, or in prison probably)


Is our industry really staffed by such people?
Do we really want production-line workers?
Physics quotes…
“I believe that a scientist looking at nonscientific
problems is just as dumb as the next guy”
“It doesn't matter how beautiful your theory is, it doesn't
matter how smart you are. If it doesn't agree with
experiment, it's wrong”
Richard P. Feynman
Process quotes
“I believe that a process consultant looking at nonprocess problems is just as dumb as the next guy”
“It doesn't matter how beautiful your process model is, it
doesn't matter how smart you are. If it doesn't agree with
reality, it's wrong”
me
Section 3 – Example Findings (rapidly
growing software house)
3.1
3.2
3.3
3.4
3.5
3.6
3.7
3.8
3.9
3.10
Some Perceptions
Product Quality
Customer Management
Organisation and Methodology
Planning and Scheduling
Product and Release Management
Development
Developer Testing
System Testing
Support
Perceptions (3 of 15)
1.
“No one can test”. There is a perception that no-one in the company is
testing well enough to stabilise and improve the quality of the product. The
support/test team are split between support and testing and support
always takes priority. The team aren’t ‘career testers’ or focused on
criticising and ‘breaking’ the product and haven’t had any formal testing
training. Developers do not perform thorough unit testing. Requirements
are not reviewed.
2.
“No one is responsible for quality”. Although one could say everyone is
responsible for quality, no one owns it because all groups are under
pressure to compromise and see no way of resisting that pressure. No one
owns quality because they don’t have authority to say no.
3.
There has been a reluctance to implement a more structured process
because of past experience. When a dedicated QA manager was
recruited, they found it difficult to implement even basic processes.
Probably their approach was to write processes and assume they could
implement themselves. This negative experience discouraged attempts to
try alternatives so practices are largely unchanged.
Need to adopt changes based on
findings, not idealised models


‘Whole-Process’ assessments not just testing
Recommendations aren’t just testing-related:

Could be a change in
requirements/development/CM…
Could be a change in attitude, leadership, policy
Could be a change in organisation
Could be a change in emphasis
Could be an agile approach
Could be a novel approach
Could be a change in personnel
None of these changes are promoted by current
testing models, but are almost always required.
Summary
You have to treat every change project as
unique
 You need to understand how things are
 But you also need to understand the
reasons WHY they are
 You must listen to practitioners and
managers

-
To hear their ideas for improvement
To align/augment ideas with the known
constraints
To refine the vision to be something