Agile Development
Running head: Agile Development- a Positive Development in Software Quality
AGILE DEVELOPMENT - A POSITIVE DEVELOPMENT IN SOFTWARE QUALITY
Ed Wisniowski
University of St. Francis
1
Agile Development
Abstract: Software development has a low success rate of 28% as of 2004.
Something needs to be done to improve the success of software projects. The advent of
Agile software development methodology appears to be latest attempt to try and improve
this process. In this paper, Ed Wisniowski compares more traditional software
development methodologies with Agile. He also outlines the challenges to Agile in the
workplace and the benefits of adopting it.
2
Agile Development
3
Agile Development- a Positive Development in Software Quality
Computer software runs in every major business. With the advent of e-commerce,
companies are rushing headlong into custom software development to tie their websites into their
customer relationship management software and Enterprise Resource Planning software. This
technology is making businesses faster, smarter and more responsive.
With all this emphasis on using technology to solve complex business problems to make
the business run faster and smarter we should be able to develop software systems successfully.
The dirty secret of the internet age is that software design and development is the least efficient
type of construction with a rate of failure which would be unacceptable in any other industry.
According to Frank Hayes from Computer world, larger projects and increasing scope creep on
projects are causing projects to fail at a rate of 18% while challenged projects which are over
budget, late, or lacking desired features is a whopping 51%. This means that counting the
margin for error only 28% of software projects succeed. (Hayes, 2004).
These numbers are confirmed by the Standish Group Survey which said that project
completion between 1994 and 2004 ranges between 16% and 34%. (Hibbs, Jewett, Sullivan,
2009). Delivery rates like this would be unacceptable in other industries and it costs businesses
billions of dollars. No other activity has this level of failure in business. According to the
Standish Group if current trends continue software projects will reach a success rate of 50%
sometime after 2020. (Hibbs, Jewett, Sullivan, 2009).
Confronted with this challenge of getting software projects done, on time and on budget
technology leaders have turned to project management techniques to try and get software
projects moving in a forward direction. Unfortunately, this has lead to more problems as project
management techniques have clashed with organizational cultures. Also, the use of the Software
Agile Development
4
Development Life Cycle or waterfall technique of managing projects have been seen as
inflexible and unable to deal with the rapidly changing demands of business. (Braun, 2005).
Something needs to be done if business leaders are to develop more confidence in
technology. One of the growing trends in the software development industry is called Agile
development and it is becoming more popular. In this paper, I will attempt to explain Agile
software development and compare it to the previous project methodologies Software
Development Life Cycle and Ad-Hoc programming. Additionally, I will then illustrate how the
use of Agile has improved quality and increased rate of return for businesses which use this
technique.
Building a bridge
Many people liken building software to a construction project. Kevin Matheny a Senior
E-Business Architect at Best Buy says that this is a wrong assumption. According to Matheny,
each day he takes the Wakota Bridge in South St. Paul Minnesota to work. It has been under
repair since 2002 and has a projected finish date of 2010. In the six years since the start of the
project, the river has not gotten wider, the structural strength of concrete has not changed and the
force of gravity has not increased. (Matheny, 2008).
Since 2002, YouTube, Facebook and MySpace did not exist and only 12% of American
households had a broadband connection. According to Matheny, “The internet we have now is
not the one we has six years ago. It is not even the one we had a year ago.” This means that
software projects and web projects, in particular, need to be quick and relevant or they are
doomed to failure. (Matheny, 2008).
So if the conditions consistently change and there is need for rapid design, deployment,
and development of software products how do technology professionals build a bridge between
Agile Development
5
the expectations of business leaders and the realities of a project. This is where project
methodologies come into play. By developing a system of design and development, it is possible
to manage change and keep projects from getting out of hand. The two primary techniques for
doing this were called the Ad-Hoc development and the Software Development Life Cycle or
waterfall technique.
The Ad-hoc approach to software development is grounded in common sense. A
developer, usually working by themselves, directly interacts with the end user community and
then writes software based on the demands of those users. If the software requires a new feature
or has a bug then quick work is done to patch the software. . This approach is fine for small and
simple software projects with few interdependencies with other projects but can quickly lead to
problems. (Braun, 2009).
Ad-Hoc development projects have a high failure rate. Since the developer answers to
numerous people, there is a lack of sponsorship for the project. Schedules and costs are
unpredictable as requirements change and the project drifts on the whims of the users. Finally,
Ad-Hoc projects have a high potential for “surprise factors” where upon implementation risks
overlooked during the development cycle come out and threaten to scuttle the project. (Braun,
2009).
The Ad-Hoc approach with all of its risks often creates projects which “…end up being
suboptimal, expensive or even impossible to maintain at an organizational level.” (Braun, 2009).
This means a software developer often creates tools which are deeply disappointing to the user
community.
This has led to the development of the Software Development Life Cycle or SDLC
approach to projects. This more formalized approach attempts to avoid the problems of Ad-Hoc
Agile Development
6
development by creating more structure to the process. In many respects SDLC, looks like
traditional project management and it has six steps. The first step is called ideation where
someone requests a feature for software; in this example, the ability to calculate sales tax on an
order. A user or sponsor suggests the feature to a developer or project manager and it moves on
to the next step. (Braun, 2009). From here the developer and project manager do a feasibility
study to see if there is enough time and budget to add the feature to the software. If the project
does not make economic sense then it is halted. Otherwise, it moves to the next step known a
definition where the developers attempt to “…define the intent of the project by eliciting,
understanding, and documenting what the particular capability is intended to accomplish in the
form of high-level and detailed requirements.” (Braun, 2009).
The definition phase can best be described as a huge game of 20 questions attempting to
try and narrow down exactly what the user community wants. For instance, is their one sales tax
ideation
feasibility
requirements
design
The Software
Development Life
Cycle or the Waterfall
method of project
management.
deployment
maintenance
Agile Development
7
rate or do the rates differ by state or county? Additionally, is sales tax applied before or after
discounts? These questions need to be answered before a single line of code is written.
Once the definition phase or requirements gathering phase is complete then the
developers enter the design phase where they actually start writing code to make the feature a
reality. As the design phase ends the project enters the deployment phase where the software is
tested and released to the public. If the software is acceptable it enters what is called a
maintenance phase where bugs are corrected when discovered. (Braun, 2009).
The Software Development Life Cycle methodology or waterfall technique of project
management has one major drawback and that is lessons learned during the progression of the
project don’t substantially revise the direction of the project. Countless project managers have
followed the directions of the project plan and delivered what the customer asked for only to
have the customer become disappointed when the final product is revealed. Additionally, market
conditions may have changed since requirements gathering and the final product does not
address these changes. (Braun, 2009).
Agile Fills the Gap
With the problems inherent with Ad-Hoc development and SDLC project approaches
something had to change or the failure rate of projects would increase. A group of software
project managers and developers got together at the Snowbird Ski Lodge in Utah between
February 11th and February 13th 2001 to discuss what could be done to improve software
projects. They combined numerous experimental project management methods such as:
Pragmatic programming, SCRUM, and Extreme Programming into a synthesis which became
known as Agile Project management. (Highsmith, 2001).
Agile Development
8
This new approach even has a manifesto which outlines the major tenants of Agile
Project management. These tenants are as follows:
“Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on
the right, we value the items on the left more.”
(Highsmith, 2001)
This seems rather touchy-feely and free form but it quickly evolved into a process which
has better been able to meet the demands of software consumers. Instead of projects having a
beginning, middle and end; a project is an ongoing living piece of code which changes over time
through a series of iterations. (Hibbs,
Jewett, Sullivan, 2009).
For instance, if we were going
Idea
to use an Agile approach to add sales
tax feature to an online shopping cart
we would skip the feasibility and
Debrief
Develop
This is an
example of how
a developer
does an agile
sprint.
requirements gathering portion of the
project and start building the sales tax
feature based on the information we
have. Once we have done the
Deploy
Document
development we document what we
Test
have done for future reference. We
then test the code to make sure it is
Agile Development
9
free from bugs. This is what Shingo Shigeo would call Poka-yoke or mistake proofing. We
would then deploy the code to a test or production environment and finally we would debrief
with the users to find out what changes or improvements need to be done. Once debriefing is
complete, we start all over again at the idea phase and go through the cycle again.
This circular process is called a “sprint” and the each sprint will yield a new iteration of
software. If there is a change to the requirements they can be folded into the next iteration.
According to Dave West from Forrester Research, “Aglie is perfect when you’re not sure what
you’re getting into, if you run into a big roadblock you’re only derailed on average a week’s
worth of work.” (West, 2009).
The sprint process does not exist in a vacuum. When setting up an Agile process the IT
team must work intimately with the user community. What they do is create what is known as a
prioritized feature list or a scrum backlog of all features they want in their software. These
features are clumped together into Iterations. Then developers begin a sprint to incorporate
these features into a software product. During the sprint the developers have a stand up meeting
or Scrum to review what they have done over the last 24 hours, what they are planning to do and
Agile Development
10
what risks may get in the way of their completing their tasks. Finally, software is delivered to
the user in a rapid fashion. (Hibbs, Jewett, Sullivan, 2009).
This Agile process may look like chaos but there is a method to the madness.
Requirements are not gathered up front, they become a just in time process and when
requirements change, corrections can easily be folded into the next iteration. The Information
technology professional is always building the software that the user is requesting. In addition,
planning is reduced according to demands of the project. Now long requirements and feasibility
phases fall away as the IT team builds software which meets the needs of the consumer. (West,
2009).
Agile Challenges and Results
Approximately one third of all companies use Agile methodologies to build software. In
addition, companies who use Agile report they build software faster and at a higher quality level.
(West, 2009). IBM has used Agile with project teams as large as 500 to 600 developers. It was
also the software methodology used for the creation of Lotus Sametime 7.5 Suite which
consisted of a team of 200 developers. (Ambler, 2008).
There are challenges to implementing an Agile project management approach in a
company. Many of these challenges are cultural and organizational. Hierarchical organizations
struggle with agile development’s quick iterative review cycles and decision making. (West,
2009). It also requires a shift from command-and-control management styles to leadership-andcollaboration styles. This means there needs to be the correct blend of autonomy and
cooperation between the development team and the user community. (Nerur, Mahapatra,
Mangalaraj, 2005).
Agile Development
11
Employees need to build trust with one another, and this could take an enormous amount
of time and energy. Also organizations have pursued the goal of creating optimized and
repeatable processes, this desire for stability represents a hurdle to adopting agile development
because Agile tends to thrive in environments of uncertainly and high variability. (Nerur,
Mahapatra, Mangalaraj, 2005).
Finally, Agile will not work unless you have the correct people on your technology team
to make it a reality. This means that it is hard to find staff. According to research, there is little
evidence to suggest that agile principles will work in the absence of competent and aboveaverage people. Not only must your development team be above average developers they must
be able to work well with others. What this can do in an organization is make it difficult to fill
job openings and create feeling of elitism among the technology staff which could interfere with
teamwork. (Nerur, Mahapatra, Mangalaraj, 2005).
Fortunately, companies which try to implement lean processes and reduce waste are the
ones most receptive to Agile project methodologies. As companies strive to be more efficient
they are turning to Agile with greater frequency. (West, 2009).
If the challenges to adopting an Agile approach are addressed there are numerous benefits
for the firm. According to Scott Ambler, Agile teams are producing greater quality software, are
more productive, enjoy greater stakeholder satisfaction and are doing it at lower cost that AdHoc or SDLC teams. Furthermore, iterations of one to four weeks from Agile deliver more
satisfaction to stakeholders than SDLC approaches which may take longer. (Ambler, 2008).
It also appears that adopting Agile methods is a very low risk activity which can be
applied to small projects and then integrated with the other projects at the firm. The most telling
statistics are the project success rates for projects which use the Agile method. According to
Agile Development
12
Ambler, surveys show that software projects have a success rate of 83% for collocated
development teams. Non-collocated teams experience a success rate of 72% and distributed
teams such as those that involve off shore developers experience a 60% success rate. (Ambler,
2008).
This is a significant improvement over the success rate of 28% discussed by Frank Hayes
in Computer World (Hayes, 2004). Looking at this evidence, the benefits of moving to Agile
development methodology outweigh the challenges. Agile also seems to have a natural partner
with lean manufacturing and business practices. (West, 2009).
In conclusion, if software development is to have the same success as other portions of
the business then as business leaders we need to abandon other approaches which are not
working. It appears that Ad-Hoc development and SDLC are not working as they should and
this means we will have to change direction. Agile, though new, appears to be filling a need
which makes software development faster, better quality, and more adaptive. This makes Agile
more than a technology fad but a real solution to a growing business problem.
Agile Development
13
REFERENCES
Aken, A. (2008, September). CHUNK: An Agile Approach to the Software Development Life
Cycle. Journal of Internet Commerce, 7(3), 313-338. Retrieved July 13, 2009,
doi:10.1080/15332860802250385
Ambler, Scott W. (2008, July). Agile and Large Teams. Dr. Dobb's Journal, 33(7), 60-62.
Retrieved July 13, 2009, from Sciences Module. (Document ID: 1499634871).
Ambler, Scott W. (2008, June). Has Agile Peaked? Dr. Dobb's Journal, 33(6), 52-54. Retrieved
July 13, 2009, from Sciences Module. (Document ID: 1503449301).
Ambler, Scott W. (2007, October). The Discipline of Agile. Dr. Dobb's Journal, 32(10), 60-62.
Retrieved July 13, 2009, from Sciences Module. (Document ID: 1337627691).
Ambler, Scott W. (2006, October). Imperfectly Agile: You Too Can Be Agile! Dr. Dobb's
Journal, 31(10), 82-84. Retrieved July 13, 2009, from Sciences Module. (Document
ID: 1137729931).
Braun, Ellen. Lean/Agile Methods for Web Site Development Online; Sep/Oct2005, Vol. 29
Issue 5, p58-60, 3p. Retrieved July 13, 2009 from
http://search.ebscohost.com/login.aspx?direct=true&db=aph&AN=17980378&site=ehost
-live
Hayes, F. (2004, November 8). Chaos Is Back. Computerworld, 38(45), 70-70. Retrieved August
2, 2009, from Business Source Elite database.
Hibbs, Curt; Jewett, Steve; Sullivan, Mike (2009) The Art of Lean Software Development.
Sebastopol, CA. O’Reilly Media.
Highsmith, J. (2001). The Agile Manifesto. Retrieved August 9, 2009 from
http://agilemanifesto.org/
Jørgensen, M., & Moløkken-Østvold, K. (2006, April). How large are software cost overruns? A
review of the 1994 CHAOS report. Information & Software Technology, 48(4), 297-301.
Retrieved August 2, 2009, doi:10.1016/j.infsof.2005.07.002
Levent Gurses. (2006, December). 10 Mistakes in Transitioning to Agile: Slow down the
transition in order to go fast. Dr. Dobb's Journal, 31(12), 40,42-43. Retrieved July 13,
2009, from Sciences Module. (Document ID: 1165894961).
Agile Development
14
Matheny, K. (2008, December 29). Agile Software Development: Bridges to the Future.
Business Week Online, Retrieved July 13, 2009, from Academic Search Premier
database.
Mearian, L. (2004, May 31). KILLING Time ON IT PROJECTS. Computerworld, 38(22), 3636. Retrieved July 13, 2009, from Academic Search Premier database.
Nerur, S; Mahapatra, RK; Mangalaraj, G. (May 2005) Challenges of Migrating to Agile
Methodologies. Communications of the ACM May 2005/ Vol. 48, No 5, 73-78.
Retrieved July 13, 2009 from University of St. Francis Database.
Shigeo Shingo (1988). Non-Stock Production: The Shingo System for Continuous Improvement.
Cambridge, MA. Productivity Press.
West, D. (2009, April). Agile Processes Go Lean. InformationWeek. (1228), 32,34,38-40.
Retrieved July 13, 2009, from Sciences Module. (Document ID: 1700361451).