University of Southern California
Center for Systems and Software Engineering
History of Software Engineering
Barry Boehm
CS 510
Fall 2013
University of Southern California
Center for Systems and Software Engineering
A Hegelian View of Software Engineering
Evolution
Theses
Engineer
Software
like
Hardware
Many defects
Formality,
Waterfall
Software
Value-Add
PlanDriven
Software
Maturity
Models
Compliance
COTS
Integrated
Sw-Systems
Engineering
Soft
SysE
Process Overhead
Syntheses
Scalability,
Risk Mgmt.
Productivity;
Reuse;
Objects;
Peopleware
Risk Mgmt.
Domain Engr.
Risk-Based
Agile/Plan
-Driven
Hybrids;
Model-Driven
Development
Value-Based
Methods;
Collaboration;
Global
Development;
Enterprise
Architectures
Autonomy; BioComputing
Scalability
Prototyping
Antitheses
Software
Differences,
Engineer
Shortages
Software
as Craft
1950's
1960's
Global
Systems
of
Systems
Time to Market,
Rapid Change
1970's
1980's
©USC-CSSE
1990's
Agile
Methods
2000's
2010's
2
University of Southern California
Center for Systems and Software Engineering
The Z4 computer, as seen in a
German museum (in Munich).
http://royal.pingdom.com/2009/12/11/retro-delight-gallery-of-early-computers-1940s-1960s/
©USC-CSSE
3
University of Southern California
Center for Systems and Software Engineering
1950’s Thesis: Engineer Software Like Hardware
• Hardware-oriented:
– Software applications: airplanes, bridges, circuits
– Economics: Boehm supervisor, 1955
• “We’re paying $600/hour for that computer, and
$2/hour for you, and I want you to act accordingly.”
– Professional Societies: Association for Computing
Machinery, IEEE Computer Society
– Software Processes: SAGE (Semi-Automated Ground
Environment)
• 1 MLOC air defense system, real-time, user-intensive
• Successful development of highly unprecedented
system
• Hardware-oriented waterfall-type process
©USC-CSSE
4
University of Southern California
Center for Systems and Software Engineering
WITCH : Wolverhampton Instrument for Teaching
Computation from Harwell, UK
•2.5-ton machine, first constructed in the 1950s as part of an atomic research program
•“All together, the machine can store 90 numbers. The closest analogy is a man with a pocket calculator,”
http://pctechmag.com/2012/11/61-year-old-computer-springs-back-to-life/
©USC-CSSE
5
University of Southern California
Center for Systems and Software Engineering
OPERATIONAL PLAN
MACHINE SPECIFICATIONS
The SAGE
Software
Development
Process
- (Benington, 1956)
“We were
successful
because we
were all
engineers”.
OPERATIONAL SPECIFICATIONS
PROGRAM SPECIFICATIONS
CODING SPECIFICATIONS
CODING
PARAMETER TESTING (SPECIFICATIONS)
ASSEMBLY TESTING (SPECIFICATIONS)
SHAKEDOWN
SYSTEM EVALUATION
©USC-CSSE
6
University of Southern California
Center for Systems and Software Engineering
1960’s Antithesis: Software Is Not Like Hardware
- Four Brooks factors plus two
• Invisibility: like the Emperor’s Magic Cloth
• Complexity: Royce, “for a $5M procurement, need
a 30-page spec for hardware, and a 1500-page
spec for software”
• Conformity: executed by computers, not people
• Changeability: up to a point, then becomes difficult
• Doesn’t wear out: different reliability, maintenance
phenomena
• Unconstrained: can program antigravity, time
travel, interpenetration, …
©USC-CSSE
7
University of Southern California
Center for Systems and Software Engineering
1960’s Antithesis: Software Crafting
• Flexible materials, frequent changes as above
• SW demand exceeded supply of engineers
– Music, history, art majors
– Counter culture: question authority
– Cowboy programmers as heroes
– Code-and-fix process
– Hacker culture (Levy, 1984)
• Collective code ownership
• Free software, data, computing access
• Judge programmers by the elegance of their
code
©USC-CSSE
8
University of Southern California
Center for Systems and Software Engineering
1960’s Progress and Problems
• Better infrastructure: OS, compilers, utilities
• Computer Science Departments
• Product Families: OS-360, CAD/CAM,
math/statistics libraries
• Some large successes: Apollo, ESS, BofA check
processing
• Problems: 1968, 1969 NATO Reports
– Failure of most large systems
– Unmaintainable spaghetti code
– Unreliable, undiagnosable systems
©USC-CSSE
9
University of Southern California
Center for Systems and Software Engineering
Software
Development Process
Model, c. 1960.
http://www.ics.uci.edu/~wscacchi/Presentations/Process/Informatix-Process-Lecture.ppt
University of Southern California
Center for Systems and Software Engineering
H200 (1962)
The Honeywell 200 and its successors were introduced to compete with affordable commercial computers
from IBM (specifically the IBM 1401)
http://royal.pingdom.com/2009/12/11/retro-delight-gallery-of-early-computers-1940s-1960s/
©USC-CSSE
11
University of Southern California
Center for Systems and Software Engineering
1970’s Antithesis: Formal and Waterfall Approaches
• Structured Methods
– Structured programming (Bohm-Jacopini: GO TO
unnecessary)
• Formal programming calculus: Dijkstra, Hoare,
Floyd
• Formalized Top-Down SP: Mills, Baker
• Waterfall Methods
– Code and fix too expensive (100:1 for large systems)
– Precede code by design (De Marco SD, Jackson
JSD/JSP)
– Precede design by requirements (PSL/PSA, SA,
SREM)
©USC-CSSE
12
University of Southern California
Center for Systems and Software Engineering
Software
Development Process
Model, c. 1970.
http://www.ics.uci.edu/~wscacchi/Presenta
tions/Process/Informatix-ProcessLecture.ppt
University of Southern California
Center for Systems and Software Engineering
Increase in Software Cost-to-fix vs. Phase (1976)
1000
Relative cost to fix defect
Larger Software Projects
500
IBM-SSD
200
GTE
•
100
50
•
80%
•
Median (TRW Survey)
20%
•
SAFEGUARD
20
•
10
•
5
2
Smaller Software Projects
•
1
Requirements
Design
Code
Development
test
Acceptance
test
Operation
Phase in Which defect was fixed
©USC-CSSE
14
University of Southern California
Center for Systems and Software Engineering
1970’s: Problems with Formal Methods
• Successful for small, critical programs
• Largest proven programs around 10 KSLOC
• Proofs show presence of defects, not absence
– Defects in specification, proofs happen
• Scalability of programmer community
– Techniques require math expertise, $500/SLOC
– Average coder in 1975 survey:
• 2 years of college, SW experience
• Familiar with 2 languages, applications
• Sloppy, inflexible, in over his head, and
undermanaged
©USC-CSSE
15
University of Southern California
Center for Systems and Software Engineering
University of Southern California
Center for Systems and Software Engineering
1970’s Computer
Cable television,
Cell phones,
Personal computers
http://www.pophistorydig.com/?tag=apple-computer-1970s
©USC-CSSE
17
University of Southern California
Center for Systems and Software Engineering
A Hegelian View of Software Engineering Evolution
Theses
Engineer
Software
like
Hardware
Many defects
Formality,
Waterfall
Software
Value-Add
PlanDriven
Software
Maturity
Models
Compliance
COTS
Integrated
Sw-Systems
Engineering
Soft
SysE
Process Overhead
Syntheses
Scalability,
Risk Mgmt.
Productivity;
Reuse;
Objects;
Peopleware
Risk Mgmt.
Domain Engr.
Risk-Based
Agile/Plan
-Driven
Hybrids;
Model-Driven
Development
Value-Based
Methods;
Collaboration;
Global
Development;
Enterprise
Architectures
Autonomy; BioComputing
Scalability
Prototyping
Antitheses
Software
Differences,
Engineer
Shortages
Software
as Craft
1950's
1960's
Global
Systems
of
Systems
Time to Market,
Rapid Change
1970's
1980's
©USC-CSSE
1990's
Agile
Methods
2000's
2010's
18
University of Southern California
Center for Systems and Software Engineering
Reuse and Object Orientation
•
•
•
•
1950’s: Math routines, utilities
1960’s: McIlroy component marketplace, Simula – 67
1970’s: Abstract data types, Parnas program families
1980’s: Smalltalk, Eiffel, C++, OO methods, reuse
libraries
• 1990’s: Domain engineering, product lines, UML,
pub-sub architectures
• 2000’s: Model driven development, service oriented
architectures
©USC-CSSE
19
University of Southern California
Center for Systems and Software Engineering
HP Product Line Reuse Investment and Payoff
©USC-CSSE
20
University of Southern California
Center for Systems and Software Engineering
People: The Most Important Factor
- SW engineering is of the people, by the people, and for the people
• 1970’s: Weinberg Psychology of Computer Programming
• 1980’s: COCOMO factor-of-10, Scandinavian
Participatory Design, DeMarco-Lister Peopleware
• 1990’s – 2000’s: Importance emphasized in both Agile and
CMM cultures
– Individuals and interactions over process and tools
– People CMM, Personal Software Process
• Overall migration from Reductionism toward
Postmodernism (Toulmin)
– Universal towards Local
– General towards Particular
– Timeless towards Timely
– Written towards Oral
©USC-CSSE
21
University of Southern California
Center for Systems and Software Engineering
Dual 1990’s – Early 2000’s Antithesis:
- Maturity Models and Agile Methods
• Predictability and Control: Maturity Models
– Reliance on explicit documented knowledge
– Heavyweight but verifiable, scalable
• Time to Market and Rapid Change: Agile
Methods
– Reliance on interpersonal tacit knowledge
– Lightweight, adaptable, not very scalable
©USC-CSSE
22
University of Southern California
Center for Systems and Software Engineering
Agile and Plan-Driven Home Grounds:
Five Critical Decision Factors
• Size, Criticality, Dynamism, Personnel, Culture
Personnel
(% Level 1B) (% Level 2&3)
40
15
30
20
20
25
10
30
0
35
Criticality
(Loss due to impact of defects)
a: Many Lives
a b
b: Single Life
c: Essential Funds
d: Discretionary Funds
e: Comfort
c
d
e
Dynamism
(% Requirements – change/month)
1.0 0.3
3.0
30 10
3
10
30
100
300
90
70
50
30
10
Size
(# of personnel)
Culture
(% thriving on chaos vs. order)
©USC-CSSE
23
University of Southern California
Center for Systems and Software Engineering
COTS: The Future Is Here
• Escalate COTS priorities for research, staffing, education
– Software is not “all about programming” anymore
– New processes required
CBA Growth Trend in USC e-Services Projects
80
70
Percentage
60
*
50
40
30
20
10
0
1997
1998
1999
2000
2001
2002
Year
• CBA: COTS-Based Application
* Standish Group CHAOS 2000 (54%)
©USC-CSSE
24
University of Southern California
Center for Systems and Software Engineering
NDI/NCS Growth Trends
70%
57%
60%
50%
Percentage
USC e-services project data
shows increase in number of
projects using NCS from 19% in
2006 to 57% in 2009
50%
40%
35%
30%
20%
19%
10%
0%
Fa'06 - Sp'07
Fa'07 - Sp'08
Fa'08 - Sp'09
Fa'09 - Sp'10
Year
Programmableweb.com :
-3 new Mashups listed /day
-Total 4436 Mashups listed
[Programmableweb.com
accessed 11/09/09]
11/19/2009
Supannika Koolmanojwong _Qualifying
Exam
25
University of Southern California
Center for Systems and Software Engineering
What does a SISOS look like?
- Network-Centric Air Traffic Control
©USC-CSSE
26
University of Southern California
Center for Systems and Software Engineering
Integrated Enterprise Architectures
Federal Enterprise
Architectural Framework (FEAF)
DOD Architectural
Framework (DODAF)
Zachman
Framework
©USC-CSSE
27
University of Southern California
Center for Systems and Software Engineering
History of Agile Methods
• Particularly in 1990s, some developers reacted against
traditional “heavyweight” software development
processes.
• New methods were being developed and tested,
– e.g. extreme programming, SCRUM, Feature-driven development
– Generally termed “light” processes
• “Representatives” from several of these methods got
together in Utah in 2001
–
–
–
–
Settled on term “Agile” as a way to describe these methods
Called themselves the “Agile Alliance”
Developed a “manifesto” and a statement of “principles”
Focuses on common themes in these alternative methodologies
http://courses.cs.tamu.edu/choe/10fall/315/lectures/slide15.pdf
©USC-CSSE
28
University of Southern California
Center for Systems and Software Engineering
Spiral Family of Models
Spiral Model
1988
WinWin Spiral
1994
Anchor Point Milestones
Spiral/RUP compatibility
1996
MBASE
1999
Spiral, MBASE
variants and invariants
2001
LeanMBASE
2005
Where do OC&A’s come from?
Where are phases and milestones ?
How to avoid model clashes?
What is really required and optional ?
How to make the process more lean and agile?
• How can spiral be mapped onto system
acquisition phases and milestones?
• How can hardware, software and human
factors be integrated?
Incremental Commitment Model
Incremental Commitment Spiral Model
9/30/2011
(C) 2011 USC-CSSE
2007
2010
29
University of Southern California
Center for Systems and Software Engineering
Spiral Model (1988)
Waterfall model
-Focus on front load elaboration
Spiral model
-Risk-driven
-Complete a round by review
-Round 0- Feasibility Study
-Round 1- Concepts of Operations
-Round 2- Top level Reqm Spec
http://csse.usc.edu/csse/TECHRPTS/1988/usccse88-500/usccse88-500.pdf
9/30/2011
(C) 2011 USC-CSSE
30
University of Southern California
Center for Systems and Software Engineering
WinWin Spiral Model (1994)
Use the Theory W (win-win) approach to converge on a system's next level
objectives, constraints and alternatives.
http://csse.usc.edu/csse/TECHRPTS/1995/usccse95-509/usccse95-509.pdf
9/30/2011
(C) 2011 USC-CSSE
31
University of Southern California
Center for Systems and Software Engineering
Anchor Point Milestones (1996)
•Lack of intermediate milestones
– Anchor Points:
LCO, LCA, IOC
– Concurrent-engineering spirals between anchor points
http://csse.usc.edu/csse/TECHRPTS/1995/usccse95-507/usccse95-507.pdf
9/30/2011
(C) 2011 USC-CSSE
32
University of Southern California
Center for Systems and Software Engineering
Spiral/RUP compatibility
9/30/2011
(C) 2011 USC-CSSE
33
University of Southern California
Center for Systems and Software Engineering
Model-Based (System) Architecting and Software Engineering
(MBASE)
9/30/2011
(C) 2011 USC-CSSE
34
University of Southern California
Center for Systems and Software Engineering
The Incremental Commitment Model
6 Key Principles:
Commitment and
accountability
Incremental growth of
system definition and
stakeholder commitment
Concurrent engineering and
Iterative development
cycles
Success-critical stakeholder
satisficing
Risk-based activity levels
and milestones
9/30/2011
(C) 2011 USC-CSSE
35
University of Southern California
Center for Systems and Software Engineering
ICSM: The Incremental Commitment Spiral Model
Cumulative Level of Understanding, Product and Process
Detail (Risk-Driven)
Concurrent
Engineering of
Products and
Processes
OPERATION2
DEVELOPMENT3
FOUNDATIONS4
OPERATION1
DEVELOPMENT2
FOUNDATIONS3
DEVELOPMENT1
FOUNDATIONS2
FOUNDATIONS
RISK-BASED
STAKEHOLDER
COMMITMENT
REVIEW
POINTS:
VALUATION
EXPLORATION
6
5
4
3
2
1
Opportunities to
proceed, skip
phases
backtrack, or
terminate
Risk-Based Decisions
Evidence-Based Review Content
- A first-class deliverable
- Independent expert review
- Shortfalls are uncertainties and risks
Acceptable
Negligible
Risk
Too High,
Unaddressable
High, but
Addressable
9/30/2011
(C) 2011 USC-CSSE
1
Exploration Commitment Review
2
Valuation Commitment Review
3
Foundations Commitment Review
4
Development Commitment Review
5
Operations1 and Development2
Commitment Review
6
Operations2 and Development3
Commitment Review
36
University of Southern California
Center for Systems and Software Engineering
Paradigm shifts
Ref: Mary Shaw – History of Software Engineering http://www.dagstuhl.de/Reports/96/9635.pdf
©USC-CSSE
37
University of Southern California
Center for Systems and Software Engineering
The Important People Can Be
Used to Structure the History
• A good place to start is with « prizes » for individuals:
– Association for Computing Machinery
• ACM Turing Prize Winners: like the Nobel prize for Computing
• ACM SIGSOFT (Special Interest Group Software Engineering)
Outstanding Research Award
• ACM Software System Award Recipients
– Institute of Electrical and Electronics Engineers
• IEEE Technical Council on Software Engineering (Harlan D Mills
Award)
• IEEE Fellows
– American Academy of Arts and Sciences Fellows
Ref: J Paul Gibson http://www-public.it-sudparis.eu/~gibson/Teaching/CSC7003/L4-HistorySoftEng.pdf
©USC-CSSE
38
University of Southern California
Center for Systems and Software Engineering
ACM Turing Prize Winners: like
the Nobel prize for Computing
1966 A.J. Perlis
1972 E.W. Dijkstra
1974 Donald E. Knuth
1977 John Backus
1978 Robert W. Floyd
1979 Kenneth E. Iverson
1980 C. Antony R. Hoare
1984 Niklaus Wirth
1986 John Hopcroft
1991 Robin Milner
1995 Manuel Blum
1996 Amir Pnueli
1995 Manuel Blum
1996 Amir Pnueli
1999 Frederick P. Brooks
2001 Ole-Johan Dahl KristenNygaard
2003 Alan Kay
2005 Peter Naur
2006 Frances E Allen
2007 Edmund M Clarke E Allen
Emerson Joseph Sifakis
2008 Barbara H Liskov
Ref: J Paul Gibson http://www-public.it-sudparis.eu/~gibson/Teaching/CSC7003/L4-HistorySoftEng.pdf
©USC-CSSE
39
University of Southern California
Center for Systems and Software Engineering
ACM SIGSOFT Outstanding Research Award
This award is presented to an individual who has made significant and
lasting research contributions to the theory or practice of software
engineering
2012 Lori Clarke
http://www.sigsoft.org/awards/outResAwd.htm
©USC-CSSE
40
University of Southern California
Center for Systems and Software Engineering
http://www.computer.org/portal/web/awards/harlan
©USC-CSSE
41
University of Southern California
Center for Systems and Software Engineering
©USC-CSSE
Ref: J Paul Gibson http://www-public.it-sudparis.eu/~gibson/Teaching/CSC7003/L4-HistorySoftEng.pdf
42
University of Southern California
Center for Systems and Software Engineering
©USC-CSSE
Ref: J Paul Gibson http://www-public.it-sudparis.eu/~gibson/Teaching/CSC7003/L4-HistorySoftEng.pdf
43
University of Southern California
Center for Systems and Software Engineering
©USC-CSSE
Ref: J Paul Gibson http://www-public.it-sudparis.eu/~gibson/Teaching/CSC7003/L4-HistorySoftEng.pdf
44
University of Southern California
Center for Systems and Software Engineering
©USC-CSSE
Ref: J Paul Gibson http://www-public.it-sudparis.eu/~gibson/Teaching/CSC7003/L4-HistorySoftEng.pdf
45
University of Southern California
Center for Systems and Software Engineering
©USC-CSSE
Ref: J Paul Gibson http://www-public.it-sudparis.eu/~gibson/Teaching/CSC7003/L4-HistorySoftEng.pdf
46
University of Southern California
Center for Systems and Software Engineering
©USC-CSSE
Ref: J Paul Gibson http://www-public.it-sudparis.eu/~gibson/Teaching/CSC7003/L4-HistorySoftEng.pdf
47
University of Southern California
Center for Systems and Software Engineering
©USC-CSSE
Ref: J Paul Gibson http://www-public.it-sudparis.eu/~gibson/Teaching/CSC7003/L4-HistorySoftEng.pdf
48
University of Southern California
Center for Systems and Software Engineering
©USC-CSSE
Ref: J Paul Gibson http://www-public.it-sudparis.eu/~gibson/Teaching/CSC7003/L4-HistorySoftEng.pdf
49