Enabling Systems Thinking to
Accelerate the Development of
Senior Systems Engineers
INCOSE Presentation
February 2007
Heidi L. Davidz
Acknowledgment of
Research Support
* Introduction *
Methods
Results
Implications
Conclusion
• Doctoral Committee
–
–
–
–
–
Professor Deborah Nightingale (chair)
Professor Tom Allen
Dr. Joel Cutcher-Gershenfeld
Dr. Eric Rebentisch
Dr. Donna Rhodes
• Research Sponsored by the Lean Aerospace
Initiative (LAI)
– Additional Reader: Professor John Carroll
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 2
* Introduction *
Methods
Results
Implications
Conclusion
Agenda
•
•
•
•
•
Introduction
Research Methods
Results
Implications
Conclusion
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 3
Enabling Systems Thinking to Accelerate the
Development of Senior Systems Engineers
Heidi Davidz
Advisor: Professor Deborah Nightingale
Methods
Motivation
• Increasing interest in systems thinking
• Data needed on systems thinking development
Job
rotations
Systems
work roles
Innate
traits
Training
classes
What are the
mechanisms that
develop systems
thinking in engineers?
Results
University
programs
1. Literature Review
2. Pilot Interviews
3. Field Study with Interviews & Surveys
• 205 Participants, 10 Companies
• Expert Panelists, Sr. Systems Engineers,
Inductive
Sr. Technical Specialists & Jr. Systems
Engineers
Exploratory
4. Blue Chip Interviews
5. Data Analysis
6. Theory Synthesis
How do senior systems
engineers develop?
• Even though systems thinking definitions
diverge, there is consensus on primary
mechanisms that enable or obstruct
systems thinking development in engineers
• Enabling mechanisms include experiential
learning, certain individual characteristics,
supportive environment
• Developed a framework and conceptual
illustration for systems thinking
•
•
•
•
•
•
Implications
Identified implications for government,
industry, and academia
Highlighted inconsistencies between policy
& effective mechanisms
Need to evolve intervention maturity
Government should set enabling policy
Industry should utilize primary mechanisms
Academia should continue studying how
systems thinking actually develops
* Introduction *
Motivation
Methods
Results
Implications
Conclusion
• Increasing complexity of engineering systems and the
corresponding need for systems professionals
• Importance of systems engineering, demonstrated in
policy mandates
• Importance of systems engineering workforce issues,
also shown in policy documents
• Data needed on systems thinking development in
order to know which methods are most effective in
developing systems thinking in engineers
Need for DATA on Systems Thinking Development
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 5
Key Research
Questions
* Introduction *
Methods
Results
Implications
Conclusion
1. What are enablers, barriers, and
precursors to the development of
systems thinking in engineers?
2. How do senior systems engineers
develop?
3. What are the mechanisms that develop
systems thinking in engineers?
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 6
Literature Review
and Existing Theory
Introduction
* Methods *
Results
Implications
Conclusion
• Broad literature found on “systems thinking”
– Lack of a central, ongoing discussion
– Systems thinking literature found in disparate fields and
journals, from systems dynamics to systems engineering
to general philosophy
• Very limited literature on “systems thinking
development” and mechanisms for development
• Heavy dependence on heuristics of how systems
thinking develops
Scant Literature on Systems Thinking Development
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 7
Research
Methods
Literature
review
Field
Study
Identified POC to work
with others to identify
Expert Panelists
(N=37)
• Completed
survey and
interview
• Identified
subjects for
3 follow-on
groups
Pilot
Interviews
(N=12)
(a) Contacted
Company
• 10 companies
participated
• Primarily U.S.
aerospace companies
1. Senior
2. Senior
Systems
Technical
Engineers
Specialists
(N=62)
(N=53)
(Total of 205 interviews and 188 surveys)
Exploratory
Inductive
Theory
Synthesis
February 15, 2007
(c) Expert
Panelists
(b) Point-of-contact
Data Analysis
Using QSR N6,
SPSS, MS Excel
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
3. Junior
Systems
Engineers
(N=53)
(d) Follow-On Subjects
• Completed interview
• Completed survey
Additional
Interviews
with “Blue
Chip”
Proven
Experts
(N=2)
Slide 8
Participating Companies
Company
Site
System Context

The Aerospace
Corporation
Systems Engineering in Chantilly, VA &
Los Angeles, CA
FFRDC - Global Positioning System (GPS), Air Force Satellite
Communications (AFSATCOM) System, etc.1

BMW
Systems Architects at BMW Group in
Munich, Germany
Commercial - Manufacturer of premium automobiles and
motorcycles2

Boeing
Boeing Commercial Airplanes,
Engineering Liaison group in Renton
and Everett, Washington
Contractor - Commercial jetliner manufacturer3

Booz Allen
Hamilton
Systems group, multiple locations,
referred by a systems partner at
headquarters in McLean, VA
Consultant - Strategic management and technology consulting
firm to industry and government4

General
Dynamics
Sites 1 & 2
SE at General Dynamics Advanced
Information Systems in Bloomington,
MN and in Pittsfield, MA
Contractor - Provider of transformational mission solutions in
command, control, communications, computers, intelligence,
surveillance and reconnaissance (i.e. Future Combat Systems)5

MITRE
Systems Engineering in Bedford, MA &
McLean, VA
FFRDC - Global Information Grid, IRS enterprise modernization
program, etc.6

Northrop
Grumman
Airborne Ground Surveillance & Battle
Management Systems, Integrated
Systems, Melbourne, FL, SE
Contractor - E-8C Joint Surveillance Targeting Attack Radar
System (Joint STARS), Cyber Warfare Integration Network
(CWIN), etc.7

Pratt &
Whitney
SE in East Hartford, CT
Contractor - Design, manufacture, and support of turbine engines8

Sikorsky
February 15, 2007SE in Stratford, CT
Contractor
Design and build advanced helicopters Slide
for 9
© Massachusetts Institute
of -Technology
9
commercial, industrial and military use
Heidi Davidz, hdavidz@alum.mit.edu
Introduction
Coding in QSR N6
* Methods *
Results
Implications
Conclusion
Q: How to make sense of 205 interviews each with a 4-5
page transcript?
A: Use “content analysis” to categorize key ideas and
thoughts from the interview
– This categorization process is called “coding”
– The resulting categories are called “nodes”
– The nodes were recorded and organized in a
qualitative data management tool called QSR N610
– Nodes were organized in hierarchies, with Level 2
as a sub-node of Level 1
Content Analysis Performed Using QSR N6 Tool
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 10
Content Analysis:
From Raw Data to Nodes
Question
Introduction
* Methods *
Results
Implications
Conclusion
Response
• Coded As:
– Level 1 Node - “Experience”
– Level 2 Node - “Job/opportunity to see
systems view”
• Individual lines of ~1000 pages of
transcripts coded in this way
• Yield of 908 nodes
Node Hierarchies Organize Conceptual Patterns
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 11
Screen
Shot
Additional
Data Analysis
Introduction
* Methods *
Results
Implications
Conclusion
• Interview data exported from QSR N6 to MS Excel
to determine top interview responses
• Interview data exported from QSR N6 to SPSS to
run statistical tests
• Results reported at both Level 1 and Level 2 of
the node hierarchy to address aggregation bias
• SPSS used to analyze survey data
• Manual content analysis performed on pilot
interviews and blue chip interviews
Utilized Multiple Data Exploration Methods
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 12
Underlying
Research Result
Introduction
Methods
* Results *
Implications
Conclusion
Even though systems
thinking definitions diverge,
there is consensus on primary
mechanisms that enable or
obstruct systems thinking
development in engineers
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 13
Consensus on Primary
Enabling Mechanisms
•
Introduction
Methods
* Results *
Implications
Conclusion
There is consensus on primary
mechanisms that enable systems
thinking development in engineers
1. Experiential learning
2. Individual characteristics
3. Supportive environment
Data Show Consensus
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 14
Solving the Puzzle
Introduction
Methods
* Results *
Implications
Conclusion
Q: How can people agree on mechanisms that enable
systems thinking when their definitions of systems
thinking do not agree?
A: Though the articulation of the systems thinking
definitions diverge, there are common themes:
(a) Functions and behaviors at the contextual edge
(b) Interactions of elements and how large scale things relate
The primary mechanisms cited enable and encourage
(a) Translation across contextual edges
(b) Consideration of interactions
(c) Higher impact learning
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 15
Divergent Systems
Thinking Definitions
Introduction
Methods
* Results *
Implications
Conclusion
• Expert Panelists and follow-on subjects
were asked:
– “How do you define systems thinking?”
– Considering a given systems thinking definition, what
aspects do you agree or disagree with and why
• 205 interviews, 205 unique definitions
• Data show that when people refer to the
phrase “systems thinking” they are often
not articulating the same concept
Interview
Questions
Systems Thinking Definitions Diverge
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 16
Example Systems
Thinking Definitions
Introduction
Methods
* Results *
Implications
Conclusion
•
•
•
•
“Big picture”
“Interactions”
“Worrying about everything”
“System thinking is the ability to think about a system or system architecture
holistically, considering the design elements, complexities, the “ilities”, the
context that product or system will be used in, etc.”
• “You have to think extremely broadly. You can’t focus on a specific aspect.
Think from the application of what a product is. Think from what the customer
wants explicitly. Be able to think in all the areas that are related to that device.
It’s broad and deep thinking. If you can’t do both, then you shouldn’t do
systems stuff. You must be organized. Think without boundaries at the start.
If you think that your job is the requirements, then you are a clerk, not a
systems engineer.”
• “Connecting lots of dissimilar disciplines and weighing trade offs between
them…”
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 17
More
Definitions
Reconciling Systems
Thinking Definitions
Introduction
Methods
* Results *
Implications
Conclusion
• Synthesis of the definitions from the field study and the definitions
in the literature yielded an original framework of systems thinking
• Five foundational elements:
1.
2.
3.
4.
5.
COMPONENTIAL - What types of things are considered
RELATIONAL - Interconnections, interactions, and interdependencies
both within the system of interest and between the system of interest
and other systems
CONTEXTUAL – The nested and embedded nature of systems
DYNAMIC – Links system in time to future and past, includes feedback,
uncertainty, risk, and the “ilities”
MODAL – Aids to understand and comprehend system
• Systems thinking is utilizing modal elements to consider the
componential, relational, contextual, and dynamic elements of the
system of interest.
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 18
Conceptual Illustration
of Systems Thinking
Introduction
Methods
* Results *
Implications
Conclusion
Tools &
Methods
Types of
Thinking
Models &
Simulations
Processes &
Frameworks
© 2005 Andreas Davidz, Elizabeth Davidz,
Heidi Davidz. All rights reserved. Used with permission.
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 19
Introduction
Methods
Coding Results
* Results *
Implications
Conclusion
“Systems Thinking Mindset”
• This MUST be decomposed, since understandings can be
contradictory
• Before designing an intervention, know what you are trying to
produce
Process-Centered SE Traits
Detail oriented
Structured
Methodical
Analytical
System-of-Systems SE Traits
Not detail focused
Thinks out-of-the-box
Creative
Abstract thinking
Define the Goal then Design the Intervention
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 20
Difficulties with
Determining Strength of
Systems Thinking
Introduction
Methods
* Results *
Implications
Conclusion
• Divergent systems thinking definitions are
problematic since strength of systems
thinking is determined by observation and
subjective measure
• In addition, many of the respondents do not
know how strength of systems thinking is
determined in their organization
Systems Thinking Definitions Diverge
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 21
Divergent
Definitions
Determination of Strength
of Systems Thinking
Introduction
Methods
* Results *
Implications
Conclusion
How does your company determine if an
employee displays strong systems thinking?
Level
Difficulty
Observation & Subjective Measure
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 22
Subjective Determination of
Strength of Systems Thinking
Introduction
Methods
* Results *
Implications
Conclusion
How does your company determine if an
employee displays strong systems thinking?
Do not
know
Observation
& Subjective
Measures
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 23
Consensus on
Enablers
Introduction
Methods
* Results *
Implications
Conclusion
Even though systems
thinking definitions diverge,
there is consensus on
primary mechanisms that
enable or obstruct systems
thinking development in
engineers
Consensus on primary mechanisms that enable or
obstruct systems thinking development in engineers
1.
2.
3.
February 15, 2007
Experiential learning
Individual characteristics
Supportive environment
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 24
Experiential Learning
Develops Systems Thinking
Introduction
Methods
* Results *
Implications
Conclusion
Q: What were key steps in your life that developed your systems thinking abilities?
Remarkable Consensus for Data Solicitation Format
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 25
Experiential Learning – Inside and Outside Work
Top Ranked Categories Are All Experiential Learning
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 26
Experiential Learning
Develops Systems Thinking
Introduction
Methods
* Results *
Implications
Conclusion
Q: In your experience, what enablers or barriers have you seen to the
development of systems thinking in engineers?
Top Node Category for “Enablers” is Experiential Learning
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 27
Experiential Learning
Develops Systems Thinking
Introduction
Methods
* Results *
Implications
Conclusion
3 of 4 Top Node Categories Are Experiential Learning
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 28
Experiential Learning
Develops Systems Thinking
Introduction
Methods
* Results *
Implications
Conclusion
• Blue chip interviewees also support experiential learning
“When I was involved in the mid-60s, programs went from concept to operation in 3-5 years. In
a period of 15 years of experience, an engineer would work on 3-5 programs. They would work
up progressively to larger and larger responsibilities. There was a whittling down process so
that we could pick the systems engineer. There would be 3-5 programs with 4-5 segments
each, so we could pick the systems engineers for the new programs from this pool. We would
have 3 to 5 to 8 people to pick from, and we could pick the best.
We never had a problem with training, since this was provided by on-the-job training and
experience. We never thought about setting up training until the 2001 timeframe when we
thought about how to fix the problems in space acquisition…
The training was all on-the-job. We would have young guys work on a section of the program,
then they would move up to be in charge of a particular element, then they would work there for
4-5 years, then they would move to a subsystem level, then they would move up to be
responsible for a segment of the program. Each time, we could pick from 5-8 engineers to
move up to the position at that higher level.”
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 29
Consensus on
Enablers
Introduction
Methods
* Results *
Implications
Conclusion
Even though systems
thinking definitions diverge,
there is consensus on
primary mechanisms that
enable or obstruct systems
thinking development in
engineers
Consensus on primary mechanisms that enable or
obstruct systems thinking development in engineers
1.
2.
3.
February 15, 2007
Experiential learning
Individual characteristics
Supportive environment
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 30
Individual Characteristics
Enable Systems Thinking
Introduction
Methods
* Results *
Implications
Conclusion
Q: Are there certain individual characteristics or innate traits that seem to
predict the development of systems thinking? If so, what are they?
Personality is Top Node Category for Individual Characteristics
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 31
Individual Characteristics
Enable Systems Thinking
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Introduction
Methods
* Results *
Implications
Conclusion
Slide 32
Background on
NEO PI-R™
Results of NEO PI-R™
Personality Test12 from
One Company
Introduction
Methods
* Results *
Implications
Conclusion
Different Sample
High in:
• Openness to
Ideas
• Competence
Note: Junior Systems
Engineers add “Anxiety”
Low In:
• SelfConsciousness
• TenderMindedness
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 33
Explanation of
“Openness to Ideas”
Link to Interdisciplinary
Studies Project
Introduction
Methods
* Results *
Implications
Conclusion
• Results correlate to findings by the Interdisciplinary Studies Project at
Project Zero, Harvard Graduate School of Education led by Howard
Gardner and Veronica Boix-Mansilla13,14
• “At the individual intellectual level, the paper characterizes exemplary
interdisciplinary workers as embodying a disposition toward curiosity, risktaking, open mindedness and humility.”
• “Curiosity in multiple areas of knowledge was a mobilizing force for the
interdisciplinary workers in our study. Curiosity emerged implicitly in their
accounts of professional growth as well as explicitly as a driving force of
interdisciplinary work.”
• “Open-mindedness is the second trait repeatedly attributed to
interdisciplinary workers and collaborators.”
Outside Study Also Emphasizes Curiosity and Open-Mindedness
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 34
Consensus on
Enablers
Introduction
Methods
* Results *
Implications
Conclusion
Even though systems
thinking definitions diverge,
there is consensus on
primary mechanisms that
enable or obstruct systems
thinking development in
engineers
Consensus on primary mechanisms that enable or
obstruct systems thinking development in engineers
1.
2.
3.
February 15, 2007
Experiential learning
Individual characteristics
Supportive environment
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 35
Environment Affects
Systems Thinking
Introduction
Methods
* Results *
Implications
Conclusion
Q: In your experience, what enablers or barriers have you seen to the
development of systems thinking in engineers?
3 of 5 Top Barriers Are Environmental
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 36
Environment Affects
Systems Thinking
Introduction
Methods
* Results *
Implications
Conclusion
Organizations Shape These Node Categories
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 37
Introduction
Methods
Statistical Tests
* Results *
Implications
Conclusion
• Multiple statistical tests run to compare differences between groups
– Comparison of all classifications
– Comparison of Senior Systems Engineers to:
 The Expert Panelists
 The control group of Senior Technical Specialists
 The control group of Junior Systems Engineers
– Comparison of all companies
– Comparison of two opposing companies
• Results show that the differences between groups are not
significant most of the time
• The Senior Systems Engineers do not differ from the other
classifications for the majority of the top-ranked node categories
Differences Are Not Significant Most of the Time
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 38
Need for Systems
Opportunities
Introduction
Methods
Results
*Implications*
Conclusion
New United
States
Military
Aircraft
Programs by
Decade and
Career
Lengths of a
Typical
Engineer
(From Murman, Walton et
al. 2003, citing
Hernandez)15
Declining Opportunities for Experiential Learning
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 39
Inappropriate Emphasis
on Training16
Introduction
Methods
Results
*Implications*
Conclusion
Emphasis is on Training Not Experiential Learning
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 40
Intervention Maturity
Introduction
Methods
Results
*Implications*
Conclusion
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 41
Intervention Maturity
Introduction
Methods
Results
*Implications*
Conclusion
Systems Thinking Interventions Should Be Based
on Knowledge and Include Feedback Mechanisms
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 42
Implications
for Government
Introduction
Methods
Results
*Implications*
Conclusion
• Applications of Research for Government
1. INCENTIVES - Provide incentives to promote strong systems
thinking
2. POLICY - Adjust policies to emphasize experiential learning for
systems thinking development
3. ACQUISITION STRATEGY - Change acquisition strategy to
provide more programs and opportunities for engineers to
develop systems thinking
4. RESEARCH - Promote research on the mechanisms for effective
systems thinking development
5. SYSTEMS PROGRAMS - Encourage systems programs that
teach systems skills and systems thinking
Set Policy Environment to Enable Systems Thinking Development
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 43
Implications
for Industry
Introduction
Methods
Results
*Implications*
Conclusion
• Applications of Research for Industry
1. INTERVENTION STRUCTURE - Structure systems thinking
interventions to emphasize experiential learning
2. FILTER AND FOSTER - Filter and foster identified individual
characteristics in systems organizations
3. SUPPORTIVE ENVIRONMENT - Provide an environment
supportive to the development of systems thinking
4. COMMUNICATE ASSESSMENT - Clearly communicate how
strength of systems thinking is assessed
5. SYSTEMS PROGRAMS - Offer systems programs to teach
systems skills and systems thinking
Utilize the Primary Mechanisms That Enable Systems Thinking
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 44
Implications
for Academia
Introduction
Methods
Results
*Implications*
Conclusion
• Applications of Research for Academia
1. SYSTEMS PROGRAMS - Offer systems programs to teach
systems skills and systems thinking
2. FEEDBACK - Use feedback mechanisms to continually improve
systems programs and systems courses
3. EMPHASIZE EXPERIENCE - Structure programs and courses to
emphasize experiential learning
4. COURSE STRUCTURE - Structure courses and programs to
promote systems thinking by emphasizing context and knowledge
integration
5. RESEARCH - Continue research on the mechanisms for effective
systems thinking development
Continue Studying How Systems Thinking Actually Develops
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 45
Introduction
Methods
Results
Implications
Summary
* Conclusion *
• Exploratory and inductive study
• Field study with auxiliary interviews
• Result: Even though systems thinking definitions diverge, there
is consensus on primary mechanisms that enable or obstruct
systems thinking development in engineers
• Divergent systems thinking definitions reconciled with a systems thinking
framework, illustration and definition
• Highlights importance of experiential learning
• Development is enabled by individual characteristics such as openness to
ideas, curiosity, questioning, strong communication and interpersonal skills
• A supporting environment also enables development
• Implications for government, industry, and academia given
Rigorous Exploration of an Extensive Data Set to Discover
Effective Mechanisms to Develop Systems Thinking in Engineers
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 46
Enabling Rigor
in SE Research
Introduction
Methods
Results
Implications
* Conclusion *
• Many engineers are not familiar with
research methods applicable to studying
systems problems
• Ideas for enhancing academic rigor in
systems engineering research
1. SE Research Methods Tutorials
2. SE Research Methods Task Force
3. SE Research Crits
Ideas for Enabling SE Research Rigor
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 47
References
1)
2)
3)
4)
5)
6)
7)
8)
9)
10)
11)
12)
13)
14)
15)
16)
http://www.aero.org
http://www.bmw.com
http://www.boeing.com
http://www.boozallenhamilton.com
http://www.generaldynamics.com
http://www.mitre.org
http://www.northgrum.com
http://www.pratt-whitney.com
http://www.sikorsky.com
QSR N6 Student Mini-Manual, copyright by QSR International Pty. Ltd. Melbourne, Australia, March
2002.
Maier, M. W. and E. Rechtin, “The Art of Systems Architecting,” CRC Press LLC, 2002.
Costa, J., Paul T. and R. R. McCrae, “NEO PI-R Professional Manual, Revised NEO Personality
Inventory (NEO PI-R) and NEO Five-Factor Inventory (NEO-FFI),” Psychological Assessment
Resources, Inc., 1992.
http://www.pz.harvard.edu/interdisciplinary/research.html, 2006.
Mansilla, Veronica Boix, Dan Dillon, and Kaley Middlebrooks, “Building Bridges Across Disciplines:
Organizational and Individual Qualities of Exemplary Interdisciplinary Work,” Interdisciplinary Studies
Project, Project Zero, Harvard Graduate School of Education, 2000.
Murman, E., M. Walton, et al. "Challenges in the Better, Faster, Cheaper Era of Aeronautical Design,
Engineering and Manufacturing." Massachusetts Institute of Technology Engineering Systems Division
White Paper, paper to appear in The Aeronautical Journal, 2003, citing Hernandez.
Skalamera, R. J., “Implementing OSD Systems Engineering Policy,” 2004.
February 15, 2007
© Massachusetts Institute of Technology
Heidi Davidz, hdavidz@alum.mit.edu
Slide 48
Thank You!
Questions or Comments?