Version 2
Strategic Engineering
Designing Systems for an Uncertain Future
Change Propagation Analysis in Complex Systems
October 7, 2008
Olivier L. de Weck, Ph.D.
deweck@mit.edu
Associate Professor of Aeronautics and Astronautics
and Engineering Systems
© Olivier de Weck, Oct 2008
Page 1
Strategic Engineering – “the big picture”
technology
http://strategic.mit.edu
markets
regulations
flexibility
real options
commonality
platforms
System
Architecture
concept
Design for
Changeability
Temporal
Dimension
Integrated Modeling
and Simulation
performance, cost, risk
changes
Design for
Commonality
Spatial
Dimension
standardization
Multidisciplinary
Design Optimization
uncertainty
variety
“optimal” design
more than one variant of the
at t=to+Dt requirements change x* at t=to
system is needed: x1*, x2, … xn
and x* is no longer optimal
© Olivier de Weck, Oct 2008
Page 2
F/A-18 Center Barrel Section
Y453
Y470.5
Y488
Wing
Attachment
74A324001
© Olivier de Weck, August 2008
Page 3
F/A-18 Complex System Change
F/A-18 System Level Drawing
Fuselage
Stiffened
Original
Change
Flight Control
Software Changed
Manufacturing
Processes
Changed
Center of Gravity
Shifted
Gross Takeoff
Weight
Increased
© Olivier de Weck, August 2008
Page 4
Change Propagation Analysis
in Complex Systems
Problem
Addressed
Understanding change propagation patterns in large technical
projects involving hardware, software and human operators
Scientific
Contribution
Developed procedure for data-mining of a large change request
database (9 years, 41,500 changes) and analyzing change
patterns (“motifs”) as well as classification of system
components with a Change Propagation Index (CPI)
Outcome, Impact
Applied to a large USAF Radar System project at Raytheon.
Identified areas that are likely candidates for flexibility infusion
Giffin M., de Weck O., Bounova G., Keller R., Eckert C., Clarkson J., “Change Propagation Analysis in
Complex Technical Systems”, DETC2007-34652, ASME 2007 Design Engineering Technical
Conferences, DETC2007-34871, Las Vegas, NV, September 4-7, 2007
In Press: ASME Journal of Mechanical Design
Sponsor: Raytheon Integrated Defense Systems
© Olivier de Weck, Oct 2008
Page 5
System Description

Complex Sensor System




46 Areas (“Subsystems”)




Complex sensor system,
complex hardware, software,
human operators
Derivative of earlier system
9 Year development
Hardware
Software
Program Documentation
System Map (graph)

Interconnections between
areas
© Olivier de Weck, Oct 2008
Page 6
Data Set
Typical Change Request

Change Request Database




technical, managerial,
procedural
track parent, child, siblings by
areas with unique ID number
chronologically numbered IDs
Data Mining Procedure





Export from DBMS to text file
Written into MySQL database
with Perl scripts
Equivalent to a MS Word
document with 120,000 pages
Sorting, Filtering, Anonymizing
Write simplified change request
format (see right side)
ID Number
12345
Date Created
Date Last Updated
06-MAR-Y5
10-JAN-Y6
Area Affected
19
Change Magnitude
3
Parent ID
8648
Children ID(s)
15678, 16789
Sibling ID(s)
9728
Submitter
eng231
Assignees
eng008 eng231
eng018
Associated Individuals
Admin_001
Engineer_271
Stage Originated,
Defect Reason
[blank], [blank]
Severity
[blank]
Completed?
1
© Olivier de Weck, August 2008
Page 7
Change Networks

Apply Graph Theory to extract
networks of connected changes


parent-child changes
sibling changes

Most changes are only loosely
connected
 2-10 related changes

Some large networks emerged

Question: do these networks emerge
from a single initial change?
(rank)
(connected
changes)
1
2579
2
424
3
170
4
87
5
64
© Olivier de Weck, August 2008
Page 8
Change Propagation Network
Network plot of largest
change network in the
dataset, with 2579
associated change
requests.
© Olivier de Weck, August 2008
Page 9
Mapping Changes to affected subsystem areas
30143
27585
28187 28213
28122
28007
28166 30344
28153 27027
28695
23942
28567
28788
28790
28878
23945
28528 27656
30465
28531
23024
28009 28186 26331
23729
28067
23922
32289
29826
29353
26333
30148
28428
23992
2716924980
29538 30614
29547
28846 29399
27627
28821
29711
27023
30771
28529
30126
23821
29226
29731
30548
30466
30501
31471
23925
24781
29227
23831
28696
25481
8000 22850
31973
24927
25476
29744
System Network Map
28162 30503
25053
24659
25515
28601
31972
27952
32645
31966
24926
25463
12156
22946
31235
27592
31967
13320 26117
Change Propagation Network
© Olivier de Weck, August 2008
Page 10
Change Propagation Index (CPI)
change propagation probability

Classify each area
 Absorber, Carrier, Multiplier
instigating area
pij 
 cij ( parent )   cij ( sibling )
Ctot ( j )
total
completed
changes
in Area j
DDSM Change
Propagation Frequency
Area
1
2
3
4
5
6
1
0.4843
0.0011
0.0136
0.0057
0.0125
0.0023
2
0.0061
0.0000
0.0000
0.0030
0.0000
0.0000
3
0.0173
0.0000
0.1053
0.0050
0.0012
0.0000
4
0.0224
0.0000
0.0112
0.0449
0.0000
0.0000
5
0.0137
0.0000
0.0000
0.0000
0.1262
0.0000
6
0.0417
0.0000
0.0000
0.0000
0.0000
0.0833
A change in Area 1 caused
changes in Area 6 with a
frequency of 4.17%.
Cout (i ) 
receiving
area
Cin (i ) 
N
  pij  Ctot ( j ) 
j 1
N
  p ji  Ctot (i ) 
j 1
CPI (i ) 
Cout (i )  Cin (i )
Cout (i )  Cin (i )
-1 <= CPI <= +1
© Olivier de Weck, August 2008
Page 11
System Area Classification
CPI Spectrum

Areas found to be strong multipliers







16: hardware performance evaluation
25: hardware functional evaluation
5: core data processing logic
32: system evaluation tools
19: common software services
3: graphical user interface (GUI)
Areas found to be perfect reflectors




27, 41: look like perfect absorbers
but actually zero changes implemented
despite numerous changes proposed
= perfect reflectors
© Olivier de Weck, August 2008
Page 12
Change Request Generation
Discovered new change
pattern: “inverted ripple”
Change Requests Written per Month
1500
system
integration
and test
1200
bug
fixes
900
subsystem
design
93
89
81
69
65
61
57
53
49
45
41
37
33
29
25
21
17
13
9
5
0
77
300
73
component
design
85
major milestones
or management
changes
600
1
Number Written
[Eckert, Clarkson 2004]
Month
© Olivier de Weck, August 2008
Page 13
Insights

Inverse relationship between change magnitude and frequency of occurrence


Many change requests are never implemented


Large changes are infrequent, small ones are ubiquitous
Some are rejected, others are ignored (~ 50%)
Changes may form complex networks over time.


Most are small (<10 changes), a few large ones exist (beware of these !)
Change networks form through coalescence and not necessarily through multistep causal change propagation

Changes can propagate between areas that are not direct neighbors in the
system DSM (not shown here, but we found this is so)

Subsystems can be classified as:




Multipliers CPI > ~0.3
Carriers -0.1<CPI<1.0
Absorbers CPI<-0.3


Reflectors of Change CRI>CAI
Acceptors of Change CAI>CRI
Analysis of change database revealed that



Real world change processes more complex than expected
Industry data tends to be “noisy”
Potential for deriving change impact and likelihood for future projects
© Olivier de Weck, August 2008
Page 14
Future Work

Change Prediction:



Data Processing:


Analyze effects of staffing on changes and components
Patterns based on which personnel/organization work on the changes?
Contractual:


Standardize methods for recording and processing data, tracing large change
networks in greater depth- attempt to reconstruct logic
Staffing and Organization:



How good are our predictions regarding actual versus planned effort?
How can change propagation patterns observed on past projects be
leveraged for future design decisions (e.g. modularity, flexibility)
Can change propagation be used to write better prime and sub-contracts?
Statistical:

Are there critical numbers for change propagation? Limits on the number of
propagation steps? .
CMI-Sponsored Workshop on Engineering Change
MIT Endicott House, October 30-31, 2008
~ 12 firms from various industries (aerospace, auto, printing, construction)
© Olivier de Weck, August 2008
Page 15
Cambridge-MIT-Institute (CMI)
Engineering Change Twin
Workshops
Trinity Hall College, UK
University of Cambridge
April 7-8, 2008
MIT Endicott House, USA
October 29-31, 2008
Reasons for Change






Problems discovered during production and operations in the field
such as retrofits, recalls ….(melioration)
Customization of product variants for different customers and market
segments (globalization)
Infusion of new technologies during product refreshes or major
“block” upgrades (innovation)
Cost reduction Initiatives, response to new features introduced by
other firms (competition)
New government regulations (e.g. fuel economy standards, no lead
in electronics …(compliance)
Others ….?
Workshop Goals
Obtain multi-faceted industry perspective on state-of-the art in
engineering change practice
 Present academic perspective and recent research advances to
industry
 Establish a research agenda for the next 5 years
 Put in place basis for Special Issue of RED*
 Stimulate interest in follow-up collaboration
 Establish user community for advanced engineering change methods
and tools

* Research in Engineering Design (RED) Journal
Invited Companies
 US
 UK









Rolls Royce (A/C Engines)*
Perkins (Diesel)*
Volvo (Trucks, Engines)*
BAE Systems (Defense)*
Bosch (Auto Supplier)*
BMW (Cars)*
BP (Oil & Gas)*
MAN Roland (Printing
Systems)
Arup (Construction)













*attended April 2008
Xerox (Printing Systems)
Ford, GM (Cars and Trucks)
Agusta Westland (Helicopters)
Boeing (Aircraft)
General Mills (Food)
Fluor (Construction)
Mack (Highway Trucks)
Gerber (Textile Machines)
NASA (Spacecraft)
Raytheon (Defense Systems)
Ventana Systems (S/W)
Aberdeen Group
United Technologies Corp.
Strategic Engineering

Strategic Engineering is the process of designing
systems and products in a way that deliberately accounts
for customization and future uncertainties such that their
lifecycle value is maximized.
© Olivier de Weck, August 2008
Page 20