Improvisation in Small Software Organizations:
Implications for Software Process Improvement
Tore Dybå
SINTEF Telecom and Informatics and
Norwegian University of Science and Technology
(e-mail: Tore.Dyba@informatics.sintef.no)
1
Abstract
Traditional ways of developing software and improving the software process relies on
theories that assumes that the environment is predictable. However, for most small software organizations the environment is constantly changing and often unpredictable. This
problem has remained largely unnoticed in current debates on software process improvement, which has often been confined to discussions regarding the implementation of “best
practice” models.
In this essay, we propose that improvisation, and the jazz metaphor, can give valuable insights into the relationship between action and learning in small software-intensive
organizations. Specifically, we describe the challenges of balancing the refinement of the
existing skill base, with the experimentation of new ideas in order to find alternatives that
improve on old ones.
Keywords: Improvisation, Software Process Improvement, Small Organizations.
1
SINTEF is The Foundation of Scientific and Industrial Research at the Norwegian Institute of
Technology. Mail address: N-7465 Trondheim, Norway. Tel +47 73 59 30 00, Fax +47 73 53 25 86
–1–
Introduction
We are witnessing an explosive growth in the size and complexity of the problems addressed by software. We also experience complex interactions between software organizations and their environment, as well as the implications of rapid changes in technology.
Only by recognizing and actively dealing with this complexity and turbulence, can small
software organizations expect to survive.
There is an important distinction between acknowledging these complex interactions and being able to understand or control them. Unfortunately, there seems to be a
common belief among both researchers and managers within the software community that
success depends upon the ability to predict changes in the environment, and develop rational plans to cope with the change. Predictability, however, is a property of simple systems. Reality is different; the environment is not such a simple system – on the contrary, it
is highly complex – it is not predictable, it is not entirely knowable, and it is definitely not
controllable by the software manager or the software organization. The sooner we admit
this to ourselves, the sooner we can develop more useful models for improving the process
of software development.
To most small software organizations, their environment is more like the croquet
game in Alice in Wonderland – a game where everything is constantly changing around the
player. In that game, the croquet balls were live hedgehogs, the mallets live flamingoes,
and the arches were card soldiers. The players all played at once, without waiting for turns,
and fighting for the hedgehogs that constantly crawled away. Just as Alice tried to hit the
ball, her flamingo would twist itself round and lift its face in another direction. Besides,
there was generally a ridge or a furrow wherever she wanted to send the hedgehog to, and
the card soldiers were always getting up and walking to other parts of the playground
(Carroll, 1982).
As in the case of Alice, we argue that small organizations facing environmental turbulence, requires an improvement approach that recognizes (1) the need for a dramatically
shorter time-frame between planning and action; (2) that the planning of an action does not
provide all the details of its implementation; and finally (3) that creativity is necessary to
make sense of the environment. We consider improvisation as such an improvement approach to better understand the relationship between action and learning in small software
organizations. For that reason, we contend that small software organizations need to become more improvisational in order to survive in an increasingly turbulent and complex
environment.
The Nature of Improvisation
The notion of improvisation occurs in several contexts such as music, theatre, teaching,
therapy and sports. Recently, improvisation has also been suggested as a mindset for organizational analysis, and the jazz metaphor has been introduced as a source of under-
–2–
standing the processes of creativity and innovation, and of interpreting complex and
changing environments (Barrett, 1998).
Improvisation deals with the unforeseen, and involves continual experimentation
with new possibilities in order to create innovative and improved solutions outside current
plans and routines. The explorative nature of improvisation necessarily involves a potential
for failure, leading to the popular misconception that improvisation is only of a spontaneous, intuitive nature that happens among untutored geniuses or in immature organizations.
However, organizational improvisation does not emerge from thin air. Instead, it involves
and partly depends on the exploitation of prior routines and knowledge. Berliner (1994)
makes this point clear when he asserts that “Improvisation involves reworking precomposed material and designs in relation to unanticipated ideas conceived, shaped, and transformed under the special conditions of performance, thereby adding unique features to
every creation.” (p. 241).
Generally, there are different levels of improvisation, ranging from interpretations
or minor adjustments to an already existing model, via embellishment and variation, to
more extreme forms of improvisation where the original models are radically altered (Berliner, 1994). One of the most famous examples of extreme improvisation in modern history
is the NASA team of astronauts and ground crew that had to devise a solution to rescue
APOLLO 13.
Improvisation activities that are closer to interpretation and minor adjustments depend on the models they start with, while extreme improvisation depends more heavily on
past experience and memory. Berliner (1994) provides a description of the role experience
and knowledge plays in jazz improvisation when he describes good jazz improvisers as
having large “vocabularies”, “repertory storehouses”, and a “reservoir of techniques”.
Hence, improvisation involves a mix of exploiting previously learned lessons and exploring the contingencies of the current setting. This mix, however, points to a core paradox of
improvisation, which is also at the heart of SPI: Too much reliance on previously learned
patterns tends to limit the explorative behavior necessary for improvisation; while, on the
other hand, too much risk-taking leads to fruitless experimentation and repeated failures.
Balancing the exploitation of past lessons and the exploration of new possibilities is, as we
shall see, a function of both organizational size and environmental conditions.
Improvisation in Software Development
Today, the dominant perspective on software development is rooted in the rationalistic
paradigm, promoting a product-line approach to software development with a standardized,
controllable, and predictable software engineering process. From this perspective, the
software literature advocates discipline, and the replacement of routinized human labor
with mechanical routines and factory-like automation. As a result, a large part of the SPI
community has promoted a rational, “best practice” approach to SPI. The Software Engineering Institute has advocated the use of statistical process control (SPC) techniques, spe-
–3–
cifically control charts, in recent technical reports and proposed changes for the CMM to
explicitly support the use of rigorous statistical techniques.
However, software development is not manufacturing. It is not a mechanical process with strong causal models appropriate for a passive improvement approach based on
SPC. On the contrary, software development is largely an intellectual and sociological activity, requiring the ability to deal with change above the ability to achieve stability (Ould,
1996). Therefore, it’s time we distance ourselves from the assumptions underlying the rationalistic, linear model of software engineering, and admit that reality for most small
software organizations is a non-deterministic, multi-directional flux that involves constant
negotiation and renegotiation among and between the social groups shaping the software.
This does not mean that we abandon discipline altogether. On the contrary, what is
needed is to balance the “odd couple” of discipline and creativity in software development
(Glass, 1995). This balance can be challenging, since losing sight of the creative, designintense nature of software work leads to stifling rigidity, while losing sight of the need for
discipline leads to chaos. In software, as in jazz, discipline is, in fact, important to enable
creativity (Humphrey, 1997). Rigorous practice and mastery of tools and techniques are, in
other words, prerequisites to improvise on a theme in the course of action.
We see improvisation as an effective way of joining discipline and creativity to
further progress in the fields of software development and SPI. A distinct characteristic of
software development, as with all improvisational processes, is the fact that the results
cannot be specified completely at the outset of the work process. This means that the
planned software products can only be determined in outline. Hence, improvisation differs
from the rational approach in that there is no detailed plan and, furthermore, in that the
planning and execution of an action converge in time. This last point is particularly important since time to market often is what the competition is all about.
Improvisation in software development leads to an emphasis on how the developers
interpret and make sense of the environment, and on how they make choices in an open
situation, where there is more than one possibility. Such choices are made in selecting the
aspects we consider relevant for modeling, in making available modes of interaction with
the computer, in determining the system’s architecture and in the way we use the technical
resources for implementing the system. Moreover, we make choices in anticipating how
the computer will be embedded in its use context and in creating facilities and constraints
for users and other concerned parties. And finally, we make choices in how we conduct the
development process itself.
Only a small part of these choices are made explicit in terms of pre-decided plans.
More often they are implied by the course of action we take, or as Schön (1983) argues:
“our knowing is in our action.” (p. 49, italics in original). Furthermore, each practitioner
treats his or her case as unique, and consequently cannot deal with it by applying standard
theories or techniques. Hence, the practitioner “must construct an understanding of the
situation as he or she finds it.” (Schön, 1983, p. 129). Also, our choices are constrained by
our interactions with others. When seen in these terms, the task of software development
clearly involves a large portion of improvisation, and it should thus be clear that both so-
–4–
cial context and technological content are essential to a proper understanding of software
development.
As in the case of the jazz band, close and sustained interaction between professionals stimulates their creativity in such a way that the team performs better than the individuals could do alone (Humphrey, 1997). Hence, the best teams, whether in jazz or software,
are those that are able to honor the individualism of its members and at the same time act
as a unit.
The foregoing discussion has several important implications for SPI. However, before we dwell on these implications, we should first realize the consequences of blindly
adhering to “best practice” models for small software organizations.
Current Practices
Basically, there are two ways in which a software organization can improve their process
capability using “best practice” models such as the CMM (Florac and Carleton, 1999): (1)
it can increase the average performance (i.e. the mean of the performance distribution), or
(2) it can reduce the variance in performance (i.e. increase predictability), or a combination
of both. Increased average performance is a general feature of experiential learning, and it
is clearly beneficial for competitive advantage. Reduced variance, however, is not necessarily an advantage. In fact, for most small software companies, competition can turn reduced variance into a major disadvantage.
In order to examine the “best practice” approach to improvement and its consequences for competitiveness in small software organizations, we consider a simple model
devised by March (1991). The model assumes that survival is based on comparative performance within a group of competing organizations. Furthermore, each single performance is a draw from a performance distribution specific to a particular organization. The
mean of the distribution reflects the organization’s ability level and the variance reflects the
organization’s reliability.
For small software organizations, performance samples are also small. Hence, relative position does not depend on ability alone but is a consequence of ability and reliability
(Levinthal and March, 1993). Furthermore, the competitive environment of most small
software organizations is such that only the best survive. In such an environment, survival
depends on having an extreme performance draw. Thus, improving average ability helps
relatively little and increasing reliability (reducing variance) can be detrimental to survival.
In the extreme case where the organization faces only one competitor, increases in
average performance always pays off, whereas changes in variance have no effect on competitive advantage. However, when there are several competitors, increases in either the
mean or the variance have a positive effect. As the number of competitors increases, the
contribution of the variance to competitive advantage increases. Ultimately, as the number
of competitors goes to infinity, the mean performance becomes irrelevant (March, 1991).
The argument behind CMM’s improvement approach is that as software processes
are standardized and techniques are learned, the time required to accomplish development
–5–
Frequency
tasks will be reduced, productivity and the quality of task performance will be increased
together with the reliability of task performance (Paulk et al., 1995). March’s (1991) model
implies that if the increase in reliability comes as a consequence of reducing the left-hand
tail of the performance distribution, the likelihood of finishing last among several competitors is reduced, without changing the likelihood of finishing first. However, if process
improvement has the effect of reducing the right-hand tail of the distribution, it may easily
decrease the chance of being best among the competitors despite increases in average performance.
An improvement strategy that simultaneously increases average performance and
its reliability is, therefore, not a guarantee of competitive advantage (see Figure 1). The
consequences of such a strategy is that it helps in competition to avoid relatively low positions, whereas it has a negative effect in competition where finishing near the top is important. Thus, the price of reliability is a smaller chance of primacy.
Failures
Successes
Performance
M1
M2
Figure 1 Impact of ability and reliability on performance.
If our main goal is to increase the competitive advantage of small software organizations, it is time to move away from the model-based, “one-size-fits-all” thinking of the
1990s. Instead, we should proceed to improvement strategies that focus on learning from
our success to increase the right-hand tail of the performance distributions while at the
same time reducing the left-hand tail by learning from our failures. As we have already
argued, improvisation is thus, a potential approach for such strategies to succeed.
Implications for Software Process Improvement
Having discussed the concept of improvisation and the consequences of CMM-based improvement, we now turn to the implications for SPI in small software organizations. There
are many challenges for an improvisational approach to SPI to succeed. Two of the most
important challenges are, however, (1) to sustain exploration and not fall into the trap of
–6–
devoting all SPI resources to experiential learning and standardization of development processes, and (2) to value the lessons of failing.
Balancing exploitation and exploration
Software organizations can engage in two broad kinds of improvement strategies. They can
engage in exploitation – the adoption and use of existing knowledge and experiences, and
they can engage in exploration – the search for new knowledge, either through imitation or
innovation (March, 1991). A basic problem is to engage in enough exploitation to ensure
short-term results and, concurrently, to engage in exploration to ensure long-term survival
(Levinthal and March, 1993). A software organization that specializes in exploitation will
eventually become better and better to use an increasingly obsolete technology, while an
organization that specializes in exploration never will realize the advantages of its discoveries.
Exploitation
Exploration

Refinement, routinizing and elaboration of
existing ideas, paradigms, technologies, processes, strategies and knowledge.

Experimentation with new ideas, paradigms,
technologies, processes, strategies and
knowledge in order to find alternatives that
improve on old ones.

Provides incremental returns on knowledge
and low risk of failure.

Provides uncertain but potentially high returns on knowledge and carries significant
risk of failure.

Requires personnel who are skilled in existing technologies.

Requires personnel who are skilled in
emerging or innovative technologies.

Can generate short-term improvement results.

Often requires a long time horizon to generate improved results.
As we have already argued, improvisation requires both exploitation and exploration. Determining what the appropriate balance should be is a complicated dynamic that involves
considerations of both organizational size and environmental factors. Since exploitation
generally generates clearer, earlier, and closer feedback than exploration, the most common
situation is one in which exploitation tends to drive out exploration (Levinthal and March,
1993). To make improvisation possible for organizations operating in increasingly more
complex and turbulent environments, it is, therefore of vital importance that they are able
to increase exploration while at the same time sustaining exploitation.
We studied the balance between exploitation and exploration in a survey of 95
software departments of 55 Norwegian software companies. Specifically, we studied how
organizational size and environmental turbulence affected this balance. Effects of organizational size were compared by contrasting large organizations with small organizations. The
groups were defined such that large organizations consisted of the upper third (>200 developers) and small organizations of the lower third (<30 developers) of the distribution.
Similarly, the effects of environmental turbulence were examined by contrasting the upper
and lower third of the operationalized environment distribution.
–7–
Results showed that small software organizations kept the same level of exploitation both in stable and turbulent environments. However, they engaged in significantly
(p<0.05) more exploration in turbulent environments than they did in stable environments.
It is important to note that the increased level of exploration did not drive out exploitation.
In other words, we found support for the proposition that increased environmental turbulence requires increased levels of improvisation.
Similar to the small organizations, large software organizations did not differ significantly in their level of exploitation between stable and turbulent environments. In contrast to the small organizations, however, increased turbulence did not lead to increased
levels of exploration. On the contrary, the larger organizations seemed to lower their levels
of exploration during turbulent environments. However, this difference was not significant.
It is interesting to note that, as there was no difference in the level of exploitation
between small and large organizations regardless of the environment, there was a marked
difference in the level of exploration (see Figure 2). The results showed that small software
organizations engage in significantly (p<0.001) more exploration in turbulent environments than large software organization.
EXPLOITATION:
EXPLORATION:
19
30
18
29
17
28
16
27
15
26
14
25
Stable
ENVIRONMENT
Turbulent
SW ORGANIZATION:
Stable
Small (<30)
ENVIRONMENT
Turbulent
Large (>200)
Figure 2 Improvement strategy vs. organizational size and environmental turbulence.
These results supports the assertion that small software organizations in turbulent environments, require improvement strategies that are more closely aligned with explorative
behavior, while at the same time promoting the exploitation of past experiences. As already
discussed, this is at the heart of an improvement strategy based on improvisation.
Most software managers agree that changes in their competitive environment are
fast and increasingly unpredictable. However, managers of large software organizations
still rely on learning from experience to prepare for the future rather than exploring new
possibilities. They tend to generate the same responses even when the stimuli have
changed. They keep doing what they do well, rather than risk failure. There could be many
reasons for this. One explanation, which is also a direct consequence of using improvement
models such as the CMM, is the institutionalized routines. In stable situations, such routi–8–
nization can become an effective way of developing software, but it can also drive out the
exploration of new alternative routines. The deeper these routines are grounded in the organizational culture, the more difficult they are to change and more easily they turn into an
obstacle to improvement.
Improvisation involves, as we have seen, an organizational climate that supports an
explorative risk-taking behavior. An inherent part of such risk-taking is failure and, consequently, we should learn to value the benefits of failure in order to reduce the left-hand tail
of the performance distribution.
Learning through failure
Inherent in the rationalistic approach to software development is to consider failure as unacceptable. This is consistent with the goal of promoting stability and short-term performance, as is the case in exploitation. In this situation, success provides an excellent
foundation for increased reliability. Hence, success tends to encourage the maintenance of
the status quo – “if it ain’t broke, don’t fix it”. However, the absence of failure experiences
can result in decreased organizational competence when faced with changing and turbulent
environments. Therefore, it is important to note that improvisation requires tolerance for
failure, and that failure is an essential prerequisite both for learning and for challenging the
status quo.
In software development, as in jazz, we must be able to turn unexpected problems
and failure into learning opportunities (Abdel-Hamid and Madnick, 1990). Rather than
treat failure as unacceptable and stigmatizing, we should distinguish between failures that
result from carelessness and failures that are a result of intelligent efforts to experiment
outside existing patterns (Barrett, 1998). In this way, we can draw attention to potential
problems and stimulate the search for creative and innovative solutions when the environmental factors are changing. The paradox is that you have to experience failure in order to
have success.
Summary and Conclusions
The rationalistic approach to software development and SPI does not satisfy the challenges
facing most small software organizations. However, the jazz metaphor teaches us to look
beyond routines, formalization, and success to better understand the relationship between
action and learning. Consequently, we have suggested that improvisation has the potential
to improve the quality of software development, and thus, that improvisation and process
improvement are closely aligned.
Successful improvisation is difficult because it requires both modes of learning –
exploitation and exploration. Therefore, we hope that this essay can trigger a fruitful debate
on how small organizations can balance these modes, and better learn to improvise in order
to cope with the constantly changing and often unpredictable environment.
–9–
References
ABDEL-HAMID, T.K. AND MADNICK, S.E. (1990) The Elusive Silver Lining: How We Fail
to Learn from Software Development Failures, Sloan Management Review, Vol. 32,
No. 1, Fall, pp. 39-48.
BARRETT, F.J. (1998) Creativity and Improvisation in Jazz and Organizations: Implications
for Organizational Learning, Organization Science, Vol. 9, No. 5, pp. 605-622.
BERLINER, P.F. (1994) Thinking in Jazz: The Infinite Art of Improvisation, Chicago, IL:
University of Chicago Press.
CARROLL, L. (1982) Alice’s Adventures in Wonderland; and, Through the Looking-Glass
and What Alice Found There, Oxford: Oxford University Press.
FLORAC, W.A. AND CARLETON, A.D. (1999) Measuring the Software Process: Statistical
Process Control for Software Process Improvement, Reading, Massachusetts:
Addison-Wesley.
GLASS, R.L. (1995) Software Creativity, Englewood Cliffs, New Jersey: Prentice Hall.
HUMPHREY, W.S. (1997) Managing Technical People: Innovation, Teamwork, and the
Software Process, Reading, Massachusetts: Addison-Wesley.
LEVINTHAL, D.A. AND MARCH, J.G. (1993) The Myopia of Learning, Strategic Management Journal, Vol. 14, pp. 95-112.
MARCH, J.G. (1991) Exploration and Exploitation in Organizational Learning, Organization Science, Vol. 2, No. 1, pp. 71-87.
OULD, M.A. (1996) CMM and ISO 9001, Software Process – Improvement and Practice,
Vol. 2, pp. 281-289.
PAULK, M.C., WEBER, C.V., CURTIS, B. AND CHRISSIS, M.B. (1995) The Capability Maturity
Model: Guidelines for Improving the Software Process. Reading, Massachusetts:
Addison-Wesley.
SCHÖN, D.A. (1983) The Reflective Practitioner: How Professionals Think in Action, Basic
Book.
– 10 –