Martyn Thomas
Page 1
Finance Today 8
Extreme Hubris
Pride and Prejudice
It is a truth universally acknowledged, that requirements change. There have always been
software developers who take this to mean that there is no point in trying to gain any
deep understanding of the requirements at the start of any project, because the work will
be wasted when the requirements change. A little thought shows that such views are
superficial and should be dismissed but recently, under the twin banners of “agility” and
“extreme programming”, these absurd ideas have gained a significant following. Of
course, the fact that an idea is shared by a large group of people does not make it
sensible and worth adopting: if it did we would all be wearing platform shoes and flared
trousers, or tattoos and body-piercings.
To see why the ideas are misguided, consider a typical system development. You start
with some broad requirement, for example for a replacement account management
system, and go through the usual steps. You have discussions with department managers
and end-users. You call in consultants and they construct dozens of use-cases. You issue
an invitation for expressions of interest, interview suppliers, then adopt some of their
suggestions as part of your planning. Finally, confident that you know what you want and
what you can get, you ask for bids and place a contract against your revised requirements.
These requirements look firm for a month or two, then they start to soften. The
developers ask detailed questions that reveal some ambiguities, and it turns out that the
supplier’s solution doesn’t quite provide what you really need. You can change the way
you work to meet some of the difficulties, but others require software changes. Once you
start testing the new system with real data, another level of problems emerges, as the real
world turns out to be far more complex than anyone had realised. Detailed interactions
emerge between the old ways of working and the extra features that the new system
provides. Again, the software has to be changed or some of the hoped-for benefits have
to be dropped; by now you have raised a few dozen changes to the contracted
requirements, and the delivery date for the project has slipped and the changes have
raised the cost well above the original budget.
It turns out that, surprisingly often, these difficulties are so severe that the project has to
be cancelled. Mostly the project continues, but ends up taking about twice as long and
costing about twice as much as originally agreed. (International surveys suggest that
around 25% of projects are cancelled and that the rest cost around twice their original
budget, with requirements changes being a major factor). Finally, with heroic effort, the
project goes live. This has been another typical IT system development and no great
success: the original cost-benefit analysis has been weakened from both sides; the costs
have increased and the benefits have reduced. Quite possibly the project would never
have been authorised if the true costs and benefits had been understood.
What went wrong? The first thing to notice is that, in this brief example, the external
requirements did not actually change. In the real world there would be an additional layer
of difficulty, for example if the Government changes the tax rules or the FSA changes
the reporting requirements after the contract has been placed. Furthermore, once the end
users of the system have some experience using it, there will be requests for
modifications and enhancements. These are real changes to the requirements, mostly they
could not be forecast with enough accuracy to take them into account. Usually, however,
Martyn Thomas
Page 2
Finance Today 8
Extreme Hubris
the changes that really hurt the budget are things that were known or could have been
discovered before the development contract was signed.
Persuasion
The software industry frequently behaves like a fashion business, with crazes taking hold
on the basis of novelty, an attractive name, and a few memorable slogans. The current
fashion for “extreme programming” is one such craze that combines several valuable
(but largely unoriginal) approaches to software development with several that are original
(but unhelpful or actively dangerous).
Among the valuable approaches are:

involving a representative of the users throughout the project

breaking the development into many small steps

frequently rebuilding the complete system to make sure that there are no integration
problems

the extensive use of regression testing
whereas the dangerous practices include

the use of stories, metaphors and test cases in place of specifications

the belief that software source-code can be self-documenting [why do they think it
earned the name “code”?]

the idea that there should be no written documentation beyond the source code
listings

“group ownership” of the software – anyone can change any part of the system
whenever they believe that they can improve it [which ensures that the person who
really understands the complexities of a particularly difficult area has to defend it
constantly].

the prohibition on writing any software that is not actually needed to run today’s test
cases [an attitude rather similar to the one that led to the huge waste of the Year 2000
remediation task – which would have been just as expensive even if it had been called
“re-factoring”].
Software requirements are very complex, because software systems have to handle an
enormous range of possible situations by following predetermined rules — unlike the
humans they so often replace, software systems cannot be expected “to use their
common sense”. Software also has to handle the complexity of interfacing with other
systems, accepting historic data, and providing an acceptable user interface. So software
is complex – very complex – and this complexity has to behave in a way the users find
acceptable.
People are poor at handling complexity. Psychological experiments have shown people
can usually only remember and work with six to eight things accurately at any time.
When the number of interrelated items gets larger than this, people make mistakes. So, to
avoid the problem, professionals in every discipline have discovered the power of
abstraction.
Abstraction means focusing on the important issues – not by being vague, but by
creating a new way of describing the world you are talking about where you can ignore
Martyn Thomas
Page 3
Finance Today 8
Extreme Hubris
irrelevant detail but be absolutely precise about the things that matter. Architects do not
specify buildings by listing the position of every brick, nor do they do it through stories
and metaphors. They build informal models, of course, to give an impression of the final
construction, but the specification that is shared with the civil engineers is abstract and
precise – precise enough that it can be analysed for feasibility, strength and safety.
For exactly these reasons, computer systems need precise, abstract specifications that can
be analysed for contradictions and omissions, and for feasibility and safety. You cannot
analyse the “metaphors” and “stories” of extreme programmers, to determine the safety
or security or usability properties that matter in the real world. Their techniques are
exploratory, at best; they are useful in elucidating the user’s needs and expectations but
they are not precise enough to be analysed for key properties.
So, to replace the slogans of the extreme programmers, here are a few principles that are
shared with other professionals that build complex systems:

Abstraction is the only tool we have available to master complexity.

Don’t be vague: anything you know about the requirements must be written down as
precisely as possible.

Without precise requirements, you cannot assess the impact of proposed changes.

Implementation documents are far too detailed to be the place where architectural or
design decisions are recorded.
To these, we can add some principles that are unique to software, because of its extreme
complexity and the digital, discontinuous nature of its behaviour.

Testing only exercises a very tiny proportion of possible paths through the system.
Therefore, the fact that a system has passed a set of tests – however large and
expensive they may be – tells you very little about how the system will behave in
service.

If it matters how the system will behave in all situations, it is necessary to show that it
has certain invariant properties (for example, that there cannot be any buffer
overflows at runtime). This can only be done by rigorous analysis, not by testing.

The main costs of current generation software development arise because it is so
expensive to find and remove the many errors that are introduced early in the
project. Strong software engineering methods avoid almost all of these errors,
yielding software that is far higher quality and far cheaper to maintain, but which
costs no more to develop.
The problems that cause great overruns in systems development, or that lead to major
problems in service, have not historically been programming problems or late changes in
the environment. In the great majority of cases they have been failures to understand and
analyse the requirements as they existed at the start of the project, or flawed decisions
about the appropriate system architecture or technical methods to be used. In a phrase,
they have been failures of engineering.
Computer systems are very complex artefacts and, like all such artefacts, their
development is an engineering task. Anyone who believes that they can reliably develop
such systems without really analysing the requirements, and solely by being “agile” and
embracing change simply does not understand the nature of the profession they have
chosen to enter. The slogans of extreme programming are extreme hubris and, as was
well understood by the ancient Greeks, where Hubris goes, Nemesis is never far behind.
Martyn Thomas
Page 4
Finance Today 8
Extreme Hubris
Martyn Thomas is a founder of the software company Praxis and Visiting Professor of Software
Engineering at Oxford University. He now works as a consultant and expert witness in complex cases.
Contact him on martyn@thomas-associates.co.uk if you want references to any papers that provide
evidence for the views expressed above.
1642 WORDS.