INNOV-9: Adventures in Development
Methodologies
there’s no subtitle like a good subtitle
Gus Björklund
Wizard, Progress Software Corporation
Adobe’s (old) Development Process
PhotoShop development
 Waterfall method
•
•
•
•
•
•
•
2
Specify desired features, set ship date
Work on features to “feature complete” date
Fix bugs
Ship beta
Revise features based on beta tests
Fix more bugs
Release
INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
the Klingon way
What is this talk of 'release'? Klingons do not make
software 'releases'. Our software 'escapes' leaving a
bloody trail of designers and quality assurance
people in its wake.
Our users will know fear and cower before our
software! Ship it! Ship it and let them flee like the
dogs they are!
3
INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Adobe’s results
PhotoShop development
 Death-march to get features in by date
• work nights and weekends as deadline looms
 Lots of bugs at feature complete
 Fix worst bugs before beta ship
 No time to revise features based on feedback
 Death-march to get enough bugs fixed by
final freeze date
• work nights and weekends as deadline looms
 Buggy product
reference: blogs.adobe.com
4
INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
5
INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
This Just In …
“The digital signs along the 98 B-line between
downtown Vancouver and Richmond are supposed to
let people waiting at the bus stop know when the next
bus will arrive.
The signs, which are linked to a GPS system on
the buses, haven't been working for the past week,
freezing up and requiring frequent reboots.
...
Siemens has basically thrown up its hands and
say they can't make it work.”
CBC News, March 8, 2007
6
INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Topics
History
Failures
Agile Methods
Results
More Important Things
Call to Action
7
INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
A Bit of History
“Those who fail to learn the lessons
history teaches are doomed to repeat the
mistakes of the past.”
8
INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Early days of programming: “Cut and Try”
 code some
 test some
 fix
 code some more
 test, fix
 lather, rinse, repeat
9
INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Programs got bigger.
Chaos ensued
10 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
COBOL was invented.
Business people could create their own
applications and programmers were no longer
needed.
order was restored.
11 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Fred Brooks: “The Mythical Man Month”
12 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Harlan Mills: Top-Down Programming
13 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Edsger Dijkstra: “Goto considered harmful”
14 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
1970: Winston Royce: The “Waterfall Model”
which became US DoD STD-2167
and other standards
15 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
The “waterfall model”
Requirements
Analysis
Design
Coding
Integration
Test / Debug
Installation
Maintenance
16 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Order was restored.
17 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Digression: A Survey
 Who uses a formal development process?
 Who uses waterfall?
 Who uses DOD-2167?
 Who uses something else?
18 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Back to our regularly scheduled program
19 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Order was restored.
20 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Waterfall assumes that
 A reasonably well defined set of requirements



if we take the time to understand them
Change will be small and manageable
Integration will go well
The schedule can be met
21 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Waterfall assumes that
 A reasonably well defined set of requirements



if we take the time to understand them
Change will be small and manageable
Integration will go well
The schedule can be met
22 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Development “Failures”










1960-1964: OS/360
1981-1994: US Air traffic control system
1995: Denver airport baggage handling system
2000-2005: MS SQL Server 2005
1988: Dbase IV
1994: Progress Version 7
2004: Sainsbury PLC supply chain system
2007: TurboTax online tax filing
2001-2007: Vista
Many, many more.
23 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Facts
 $ 89 Billion spent on cancelled software


projects
$ 59 Billion more on cost overruns
Of the challenged or cancelled projects, the
average project
• was 189 % over budget
• was 222 % behind schedule
• had 61 % of planned features delivered
That was in 1994. The situation has not improved.
source: Standish Group CHAOS Chronicles
24 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Why Is Software Development So Hard?
25 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Common Causes
1. Unrealistic or unarticulated
project goals
7. Use of immature technology
2. Inaccurate estimates of
needed resources
8. Inability to handle the project’s
complexity
3. Badly defined system
requirements
9. Sloppy development practices
4. Poor reporting of project’s
status
10. Poor project management
5. Unmanaged risks
11. Stakeholder politics
6. Poor communication among
customers, developers, and
users
12. Commercial pressures
“Why Software Fails”, Robert N. Charette. IEEE Spectrum
26 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
How Can We Make It Less Hard?
27 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
QUIZ:
What is the 1 most important thing
we need in developing software?
28 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
How do we avoid failures
like the ones that we all know about?
29 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Agile development methods to the rescue!
30 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
The Manifesto for Agile Software Development,
17 anarchists agree
by Kent Beck, Mike Beedle, Arie van Bennekum,
Alistair Cockburn, Ward Cunningham, Martin
Fowler, James Grenning, Jim Highsmith, Andrew
Hunt, Ron Jeffries, Jon Kern, Brian Marick,
Robert C. Martin, Steve Mellor, Ken Schwaber,
Jeff Sutherland, Dave Thomas
Feb 13, 2001
31 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Agile manifesto: We follow the following
principles:












Our highest priority is to satisfy the customer through early and continuous
delivery of valuable software.
Welcome changing requirements, even late in development. Agile processes
harness change for the customer's competitive advantage.
Deliver working software frequently, from a couple of weeks to a couple of
months, with a preference to the shorter timescale.
Business people and developers work together daily throughout the project.
Build projects around motivated individuals. Give them the environment and
support they need, and trust them to get the job done.
The most efficient and effective method of conveying information to and within a
development team is face-to-face conversation.
Working software is the primary measure of progress.
Agile processes promote sustainable development. The sponsors, developers
and users should be able to maintain a constant pace indefinitely.
Continuous attention to technical excellence and good design enhances agility.
Simplicity—the art of maximizing the amount of work not done—is essential.
The best architectures, requirements and designs emerge from self-organizing
teams.
At regular intervals, the team reflects on how to become more effective, then
tunes and adjusts its behavior accordingly.
32 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Agile Manifesto - values
We are uncovering better ways of developing software by doing it and
helping others do it. Through this work we have come to value:
 Individuals and interactions over processes



and tools.
Working software over comprehensive
documentation.
Customer collaboration over contract
negotiation.
Responding to change over following a plan.
That is, while we value the items on the right, we value the items on the left
more.
33 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
How are agile methods different?
 Emphasizes direct communication
 Focuses on working software
 Raises the level of skill
34 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Agile approaches
 Chief programmer teams
 XP
 Scrum
 Gus’s method
35 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Chief programmer teams
An old idea that shouldn’t be forgotten
 Team consists of
• Chief programmer
• Backup chief programmer
• Librarian
•
•
•
•
Programmers
Project administrator
Toolsmith
Testers
F.T. Baker, IBM, late 60’s
36 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Agile approaches
 Chief programmer teams
 XP
 Scrum
 Gus’s method
37 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Extreme Programming (XP)
XP Values:
 Communication
 Simplicity
 Feedback
 Courage
 Respect
38 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
XP Summarised


Team of 5 to 10 work at one location with customer
representation
Development occurs in frequent iterations
• each is releasable
• each delivers more functionality

Requirements specified as “user stories”
• each is a chunk of functionality user needs


Programmers work in pairs
Requirements, architecture, and design emerge over
the course of the project
39 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
XP Summarised
 In an iteration
• Write automated tests first
• Coding, by pairs of programmers
– follow strict coding standards
– complete when
 all tests pass
 programmers can’t think of any more needed tests
• Refactor
• Demo
40 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
XP works best for
 New or prototype technology
• requirements change frequently
 Research projects
• goal is knowledge gained, not product
 Smaller projects
41 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Agile approaches
 Chief programmer teams
 XP
 Scrum
 Gus’s method
42 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Scrum Summary
 Timebox everything
 Work from “backlogs”
• Product backlog
• Sprint backlog
 Short cycles
 Small teams
43 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Scrum: Teams
 Team of 5 to 9 people
 One member serves as “Scrum Master” to
•
•
•
•
Facilitate meetings
Manage outside interference
Record decisions, track action items
Keep everyone informed
44 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Scrum: phases
 Review release plans
 Distribution, review and adjustment of product



standards
“Sprints” lasting about 4 weeks
Repeat as many times as necessary
Closure
45 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Product backlog
ID
Name
Imp
Est
How to demo
Notes
1
Deposit
30
5
Log in, open deposit page,
deposit €10, go to my balance
page and check that it has
increased by €10.
Need UML
sequence
diagram. No
need to worry
about
encryption for
now.
2
See your own
transaction
summary
10
9
Log in, click on “transactions”.
Do a deposit. Go back to
transactions, verify the new
transaction shows up
Use paging to
avoid large
queries.
Design similar
to view users
page
Source: “Scrum and XP from the Trenches”, Henrik Kniberg
46 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Scrum: sprints
 Plan
 Develop
 Wrap
 Review
 Adjust
47 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Scrum: Team meetings
 Daily, always at same time
 15 minutes
 3 questions only, by Scrum master
• What have you done since last meeting?
• What impedes your work?
• What will you do by next meeting?
 Follow-up meetings held after, if needed
48 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Scrum: advantages








Product becomes series of manageable chunks
Progress made, even if requirements not stable
Everything visible to everyone
Communication improves
Success shared along the way
Customers see how product actually works
Strong customer relationships develop
Culture created where everyone expects project to
succeed
49 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Agile approaches
 Chief programmer teams
 XP
 Scrum
 Gus’s method
50 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Gus’ Method
 Think
 Think more
 Think still more
 Write code
 Test, fix
 Write code again, test
 Write code again, test
 Doc
Not recommended for general use. Works for me.
51 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Adobe’s Recent Experience
with a new development process
52 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Adobe: the new process
PhotoShop CS3
 Incremental development
• Work on a few features at a time, finish them
before moving on
 Any dev with 20 bugs can’t work on features
 Only complete features can be merged
 Ready to ship within short notice
• some features may not be there
• everything that is there is complete and works
 More beta cycles
53 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Adobe: results with new dev process
PhotoShop CS3
 Higher quality
• Fewer bugs overall
• Fewer bugs in mid cycle
• Much more useful (and earlier) beta cycles
 More predictable dev cycles
 Can demo product at almost any stage
 No more nights and weekends
54 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
The question I know
you are all wanting to ask:
55 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Does this stuff actually work?
 Yes. Agile methods succeed at
• Fidelity, VA Software, Medtronic, Adobe, IBM,
Sapient, Yahoo, SAP, BMC Software,
Symantec, Verizon, Microsoft, others
 Results can be excellent
 Agile scales
 Agile methods will NOT solve every problem.
56 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Agile Works
Standish Group:
16 percent of waterfall projects succeed
41 percent of agile projects succeed !!!
(2006 Chaos report)
Forrester:
Agile works.
17% of companies using agile.
(2006 Agile Adoption Survey)
57 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Agile Development Myths
 No process
 No documentation
 Only works for small projects
 Only works for small teams
 Chaotic
58 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Agile Development Myths
 No process
 No documentation
 Only works for small projects
 Only works for small teams
 Chaotic
59 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
There’s something else to consider …
"A great lathe operator commands several
times the wage of an average lathe operator,
but a great writer of software code is worth
10,000 times the price of an average
software writer.”
Bill Gates
60 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Methodology/process isn’t everything
People are the most important thing
 Huge difference between best and worst
• 10 x and more variance in productivity
• 10 x and more variance in quality
 Motivation
 Experience
 Work environment
• Top performers have quiet work area and few
interruptions
http://www.stevemcconnell.com/ieeesoftware/eic14.htm
61 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Call to Action
 Try something on a small scale first
• Share your stories
 A given methodology may not suit your or



your development teams
Adapt to suit your needs
Adapt and adopt, based on your results
Read this:
“Scrum and XP from the Trenches”, Henrik Kniberg
www.crisp.se/henrik.kniberg/ScrumAndXpFromTheTrenches.pdf
62 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Homework



Books
• “The Mythical Man Month”, Fred Brooks
• “Scaling Software Agility”, Dean Leffingwell
• “Agile Software Development”, Alistair Cockburn
Paper
• “Chief Programmer Team Management of Production
Programming”, F.T. Baker, IBM Systems Journal, Vol 11,
No 1, 1972
Wikipedia articles (www.wikipedia.org)
•
•
•
•
Agile software development
Waterfall Method
Extreme Programming
Software development process
63 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation
Answers
64 INNOV-9: Adventures in Development Methodologies
© 2007 Progress Software Corporation