Collaborative Systems Thinking:
Uncovering the rules of team-level systems thinking
Caroline Twomey Lamb and Donna H. Rhodes
Massachusetts Institute of Technology
3rd Annual IEEE Systems Conference
March 26, 2009
seari.mit.edu
© 2008 Massachusetts Institute of Technology
Agenda
•
•
•
•
•
•
•
Research Question
Defining Systems Thinking
Motivation
Methodology
Results
Implications of Research
Conclusions and Future Work
3rd IEEE Systems Conference
March 24-26, 2009
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
2
Research Questions
1. What is collaborative systems thinking and how does it differ from
individual systems thinking?
2. What are the empirically generalized traits of systems thinking
teams within the context of the aerospace industry?
3. What observed mechanisms correlate with collaborative systems
thinking?
Objectives:
– To describe team-based, or collaborative, systems thinking through
an exploration of literature, interviews, and case studies.
– To propose an initial explanatory theory of collaborative
systems thinking, identifying those traits most closely
linked to collaborative systems thinking.
3rd IEEE Systems Conference
March 24-26, 2009
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
3
What is Systems Thinking?
A framework for systems with four basic ideas: emergence, hierarchy, communication
and control. Human activity concerns all four elements. Natural and designed systems
are dominated by emergence.
(Checkland 1999)
A method of placing the systems in its context and observing its role within the whole.
(Gharajedaghi 1999)
A skill to see the world as a complex system and understanding its interconnectedness.
(Sterman 2000)
A skill of thinking in terms of holism rather than reductionism.
(Ackoff 2004)
A method and framework for describing and understanding the interrelationships and
forces that shape system behavior.
(Senge 2006)
Systems thinking is utilizing modal elements to consider the componential,
relational, contextual, and dynamic elements of the system of interest.
(Davidz 2006)
Component
Complexity
Interrelationships
Context
Emergence
Wholes
R. Ackoff. Transforming the Systems Movement. Opening Speech at 3rd International Conference on Systems Thinking in Management, May 2004. Philadelphia, PA.
P. Checkland. Systems Thinking, Systems Practice, Soft Systems Methodology: A 30-year retrospective. John Wiley and Sons, West Sussex, England, 1999.
J. Gharajedaghi. Systems Thinking: Managing chaos and complexity. Butterworth-Heinemann, Burlington, MA, 1999.
P. Senge. The Fifth Discipline. Doubleday, New York, NY, 2006.
J. Sterman. Business Dynamics: Systems thinking and modeling for a complex world. McGraw-Hill, New York, NY, 2000.
H. Davidz. Enabling Systems Thinking to Accelerate the Development of Senior Systems Engineers. PhD thesis, Massachusetts Institute of Technology, Cambridge,
Massachusetts, 2006.
3rd IEEE Systems Conference
March 24-26, 2009
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
4
What is Systems Thinking?
Systems thinking is considering the system, its
components, interrelationships, context, and
dynamic behavior during engineering design
Component
Complexity
Interrelationships
3rd IEEE Systems Conference
March 24-26, 2009
Context
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
Emergence
Wholes
5
Motivation:
Fewer Systems Opportunities
Manned Fighter Program Starts by Decade
Manned Spacecraft Programs by Decade
50
6
5
4
3
2
1
?
0
1950
1960
1970
1980
1990
2000
2010
Number of Program Starts
Number of Programs
7
40
30
20
10
0
1950
Decade
V. Neal, C. Lewis, and F. Winter, Spaceflight. Macmillan,
New York, 1995.
3rd IEEE Systems Conference
March 24-26, 2009
1960
1970
1980
1990
2000
2010
Decade
E. Murman et al., Lean Enterprise Value: Insights from MIT’s
Lean Aerospace Initiative. Palgrave, New York, 2002.
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
6
Motivation:
A Workforce Nearing Retirement
25
Aerospace
Workforce
US Workforce
Percentage of Workforce
20
15
10
5
0
Under
20
20 to
24
25 to
29
30 to
34
35 to
39
40 to
44
45 to
49
50 to
54
55 to
59
60 to
64
65 to
69
70 and
Older
Five-Year Age Bands
D. Black, D. Hastings, and the Committee on Meeting the
Workforce Needs for the National Vision for Space
Exploration. Issues Affecting the Future of the U.S.
Space Science and Engineering Workforce: Interim
report, 2006.
3rd IEEE Systems Conference
March 24-26, 2009
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
7
Methodology:
Research Structure
Using multiple types of data facilitates triangulation (Robson 2002)
Different types of data illuminate different aspects of a phenomenon.
Interviews, surveys, qualitative and quantitative data needed to
describe collaborative systems thinking.
Phase 1: Literature Review
– Identify relevant constructs
– Establish framework for further
inquiry
Phase 2: Pilot Interviews
– Validation of framework
– Formulate a definition for
collaborative systems thinking
Phase 3: Case Studies
– Empirical data collection
– Basis for theory development
Phase 4: Validation Activity
– Test explanatory power of
theory on new cases
– Ensure results are generalizable
beyond initial cases
C. Robson, "Real World Research", Blackwell Publishing, Malden,
MA, 2002.
3rd IEEE Systems Conference
March 24-26, 2009
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
8
Methodology:
Grounded Theory
Grounded Theory is the ‘top half’ of the scientific
process (Glaser and Strauss 1967)
–
–
–
–
Method is exploratory in nature
Starts with a question
Execution is based in observation
Ends with an explanatory theory and set of
hypotheses
Empirical Generalizations
Theories
Observations
Predictions (Hypotheses)
3rd IEEE Systems Conference
March 24-26, 2009
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
B. Glaser and A. Strauss. The Discovery of Grounded Theory:
Strategies for qualitative research. Aldine Publishing Company,
Chicago, IL, 1967.
9
Methodology:
Grounded Theory
Grounded Theory is the ‘top half’ of the scientific
process (Glaser and Strauss 1967)
–
–
–
–
Method is exploratory in nature
Starts with a question
Execution is based in observation
Ends with an explanatory theory and set of
hypotheses
Copernicus’s Heliocentric
Cosmology (c1500)
3rd IEEE Systems Conference
March 24-26, 2009
Kepler’s Laws of
Planetary Motion (c1600)
Newton’s Laws of Gravity
(c1680)
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
Genetics and Modern
Medicine
10
Methodology: Qualitative Analysis
Descriptive Analysis
• Memo writing
– Definition: record of
researcher thoughts,
data interpretations
and questions
– Code, theoretical,
operation, diagrams
•
Coding
– Definition: breakdown
of textual data into
central ideas with goal
of creating relational
structure for
interpreting data and
explaining observations
– Open, axial, and
selective coding
•
Tools
– MaxQDA
3rd IEEE Systems Conference
March 24-26, 2009
K. Eisenhardt. Building Theories from Case Study Research. The Academy of Management Review,
14(4):532{550, 1989.
D. Krathwohl, editor. Methods of Educational and Social Science Research. Waveland Press, Inc., Long
Grove, IL, 1998.
C. Robson. Real World Research. Blackwell Publishing, Malden, MA, 2002.
R. Stebbins. Exploratory Research in the Social Sciences, volume 48 of Sage University Papers Series on
Qualitative Research Methods. Sage Publications, Thousand Oaks, CA, 2003.
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
11
Methodology: Qualitative Analysis
Descriptive Analysis
• Memo writing
– Definition: record of
researcher thoughts,
data interpretations
and questions
– Code, theoretical,
operation, diagrams
•
Coding
– Definition: breakdown
of textual data into
central ideas with goal
of creating relational
structure for
interpreting data and
explaining observations
– Open, axial, and
selective coding
•
Tools
– MaxQDA
3rd IEEE Systems Conference
March 24-26, 2009
K. Eisenhardt. Building Theories from Case Study Research. The Academy of Management Review,
14(4):532{550, 1989.
D. Krathwohl, editor. Methods of Educational and Social Science Research. Waveland Press, Inc., Long
Grove, IL, 1998.
C. Robson. Real World Research. Blackwell Publishing, Malden, MA, 2002.
R. Stebbins. Exploratory Research in the Social Sciences, volume 48 of Sage University Papers Series on
Qualitative Research Methods. Sage Publications, Thousand Oaks, CA, 2003.
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
12
Methodology: Qualitative Analysis
Descriptive Analysis
• Memo writing
Case 1
– Definition: record of
researcher thoughts,
data interpretations
and questions
– Code, theoretical,
operation, diagrams
•
Coding
– Definition: breakdown
of textual data into
central ideas with goal
of creating relational
structure for
interpreting data and
explaining observations
– Open, axial, and
selective coding
•
Tools
– MaxQDA
3rd IEEE Systems Conference
March 24-26, 2009
overlap
Case 2
Agreement
Disagreement
Seek
Exceptions
Seek
Exceptions
Better
Understanding
K. Eisenhardt. Building Theories from Case Study Research. The Academy of Management Review,
14(4):532{550, 1989.
D. Krathwohl, editor. Methods of Educational and Social Science Research. Waveland Press, Inc., Long
Grove, IL, 1998.
C. Robson. Real World Research. Blackwell Publishing, Malden, MA, 2002.
R. Stebbins. Exploratory Research in the Social Sciences, volume 48 of Sage University Papers Series on
Qualitative Research Methods. Sage Publications, Thousand Oaks, CA, 2003.
Better Action
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
13
Methodology: Quantitative Analysis
Descriptive Analysis
• Survey statistics
• ‘Distance’ between case studies
Equation governing ‘distance’ matrix
– Based on grouping teams
based on distance between
vectors describing the teams
– Used complete linkage method
Inferential Analysis
• Multivariate Regression
Analysis
– Identify regressor variables that
best explain team-reported
CST: A model explaining CST
observations
– Use new case studies and
model to validate chosen
variables are generalizable
3rd IEEE Systems Conference
March 24-26, 2009
Equations describing multivariate
regression
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
14
Defining Collaborative Systems
Thinking: Pilot Interview Results
• 8 Pilot Interviews
• Most universally cited concepts
– Definition
• Requires a holistic approach
• Involves producing a product
– Culture
• Creativity is an enabler
• A willingness to ask and answer
questions
– Process
• Provides a shared language and
taxonomy
• Bringing disciplines together early is
an enabler
– Team Traits
• Well-socialized experts enable CST
• High communication bandwidth
3rd IEEE Systems Conference
March 24-26, 2009
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
15
Defining Collaborative Systems
Thinking: Basis in Literature
Literature Concepts
Team thinking is valid concept based on
shared processing of information.
(Salas and Fiore 2004)
Team thinking is supported by interactions
that create pointers to knowledge held
within team. (Wegner 1986)
Creative environments/multiple perspective
support systems thinking.
(Thompson and Lordan 1999)
Normative design processes that utilize
divergent and convergent thinking are
superior for handling complexity.
PI Concepts
Central Concept
“A willingness to ask
and answer
questions”
Team Interaction
“Creativity is an
enabler”
Multiple Thinking
Styles / Design
Process
(Stempfle and Badke-Schaub 2002)
“Process provides a
shared language and
taxonomy”
Multiple design languages (e.g. sketching,
modeling, etc) are required to communicate
design knowledge. (Dym et al. 2005)
“High communication
bandwidth enables
CST”
Multiple
Communication
Media
Emphasis on end product a differentiator
“Involves producing a
between successful and failed product
product”
3rd IEEE Systems Conference
© 2009 Caroline Twomey Lamb
development
teams. (Dougherty 1990)
March 24-26, 2009
Massachusetts Institute of Technology
Importance of an
End Product
16
Collaborative Systems Thinking
Defined
Collaborative
Collaborativesystems
systemsthinking
thinkingisisan
anemergent
emergentbehavior
behaviorof
of
teams
teamsresulting
resultingfrom
fromthe
theinteractions
interactionsof
ofteam
teammembers
members
and
andutilizing
utilizingaavariety
varietyof
of thinking
thinkingstyles,
styles,design
design
processes,
processes,tools
toolsand
andcommunication
communicationmedia
mediato
toconsider
consider
the
thesystem,
system,its
itscomponents,
components,interrelationships,
interrelationships, context,
context,
and
anddynamics
dynamics toward
towardexecuting
executingsystems
systemsdesign.
design.
(Lamb,
(Lamb,2008)
2008)
C. Lamb Systems Thinking as an Emergent Team Property. IEEE
Systems Conference, Toronto, Canada, April 2008
3rd IEEE Systems Conference
March 24-26, 2009
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
17
Empirically Generalized Traits:
Team Structure
Collaborative Systems Thinking (CST) Teams have 3 Membership
Categories
• Strong systems leadership
– 2-3 individuals acting in coordinated manner
– Strong individual systems thinkers with complementary social and
technical skills
•
Developing systems professionals
– Functional background
– Demonstrated ability to ask questions outside their functional
background
– Convey functional information to team at correct level of detail
(translators of technical information)
•
Functional specialists
– Have concurrent membership in several teams
– Participation driven by when expertise is required
– Role on team changes with design stage
3rd IEEE Systems Conference
March 24-26, 2009
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
18
Empirically Generalized Traits:
Team Structure (2)
CST Rated 7.9/10
(Std 0.9)
CST Rated 5.6/10
(Std 2.1)
26
28
25
29
30
15
16
8
27
19
21
11
6
5
23
3
4
7
24
20
9
21
12
14
18
1
22
2
17
22
1
6
20
10
24
2
3
23
13
10
12
18
14
5
19
9
13
4
7
17
11
15
16
8
Based on team roster
3rd IEEE Systems Conference
March 24-26, 2009
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
Based on group org chart and 6
interviews
19
Empirically Generalized Traits:
Experience
Collaborative Sys te m s Think ing vs . Ye ars of Indus try
Expe rie nce
Collab o r ative Sys te m s T h in k in g vs Pas t Pr o g r am Expe r ie n ce
9
8.5
8
7.5
7
6.5
6
5.5
5
4.5
4
1
5-10
2
10-15
3
15-20
4
25-30
5
30-35
6
>35
M ode of Te am M e m be rs Ye ars of Indus try Expe rie nce
•
8
7.5
7
6.5
6
5.5
5
4.5
4
0
<5
•
8.5
(on 1-10 scale)
Reported Collaborative Systems Thinking Ability
Reported Collaborative Systems Thinking
Ability (on 1-10 scale)
9
0
1
2
3
4
5
6
+
M o d e of T e am M e m b e r Pas t Pr o g r am Exp e r ie n ce
Years of experience is an important indicator of collaborative
systems thinking
Past program experience is a better indicator of collaborative
systems thinking
3rd IEEE Systems Conference
March 24-26, 2009
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
20
Empirically Generalized Traits:
Culture
•
CST teams have more creative environments
–E.g. Challenging work, collaborative work environment, decision
freedom, access to resources. (Thompson and Lordan 1999)
Technical and social leadership are important on CST teams
Consensus decision making is more common on CST teams
•
•
–Collocation affects perceptions of how decisions are made within team
7
6
6
5
5
Primary Decision Making Mode
(1=consensus, 7=individual)
Primary Decision Making Mode
(1=consensus, 7=individual)
Perceptions of Team Decision Making (1—Consensus; 7—Individual)
7
4
3
4
3
2
2
1
1
0
0
A
B
C
D
E
3rd IEEE Systems Conference
March 24-26, 2009
F
G
H
I
J
A
B
C
D
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
21
Mechanisms Explaining CST
Observations
Concept
Correlation to CST
Relative Frequency of Consensus
Decision Making
0.89
Creativity: Realistic Schedule
0.79
Past Similar Program Experience
0.79
Team Environment: Trust in Team
Member Abilities
0.78
Creativity: Collaborative
Environment
0.65
Perceived Relevance of Standard
Process
0.60
Relative Frequency of Face-toFace Interactions
0.53
3rd IEEE Systems Conference
March 24-26, 2009
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
Self-Assessed CST
8.3 8.1
7.9
5.7
4.7
2
Higher
CST
Teams
Lower
CST
Teams
4
6
22
Recap: Research Questions
1. What is collaborative systems thinking and how does it differ
from individual systems thinking?
2. What are the empirically generalized traits of systems thinking
teams within the context of the aerospace industry?
3. What observed mechanisms correlate with collaborative systems
thinking?
Objectives:
– To describe team-based, or collaborative, systems thinking
through an exploration of literature, interviews, and case
studies.
– To propose an initial explanatory theory of collaborative
systems thinking, identifying those traits most closely
linked to collaborative systems thinking.
3rd IEEE Systems Conference
March 24-26, 2009
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
23
Implications for Industry
and Academia
1.
Program experiences are essential
–
–
–
2.
Industry should provide more opportunities to see and participate in
the entire system lifecycle
Example: Phaeton early career hire rotation program (Reiber et al. 2008)
Secondary Benefits: Shorter program cycles improve workforce
retention (Pieronek and Pieronek 2004)
The social and technical components of engineering need to be
balanced
–
–
–
Engineering coursework and promotion decisions overemphasize the
technical aspects of engineering
Teams should be formed on the basis of both technical contributions
and social roles
The best systems engineers and leaders balance the social and
technical needs and interactions within their programs (Griffin 2007;
Derro 2008)
R.R. Reiber, et al., “Bridging the Generation Gap: A Rapid Early Career Hire Training Program,” AIAA Space 2008
Conference, AIAA, Washington D.C., 2008.
C. Pieronek and T. Pieronek. Attracting Women to Careers in Aerospace Using Lessons Learned in Higher Education.
In Proc. AIAA Space 2004, San Diego, CA, September 2004.
M. Griffin. System Engineering and the “Two Cultures" of Engineering. Purdue University Boeing Lecture, March 2007.
M.E. Derro, “NASA Systems Engineering Behavior Study,” URL:www.nasa.gov/pdf/291039main_NASA_SE_Behavior_
Study_Final_11122008.pdf [cited November 20 2008].
3rd IEEE Systems Conference
March 24-26, 2009
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
24
Conclusions and Future Work
•
•
Collaborative systems thinking (CST) is distinct from individual
systems thinking
CST teams have characteristics that differentiate them from nonCST teams
–
–
–
–
–
•
Strong systems leadership is present (social and technical)
Teams have 3 consistent categories of team membership
Team membership have relatively more past program experience
Teams are using sketches, models and prototypes more frequently
Teams utilize more consensus decision making
Future Work
– Validation Case Studies
– Develop Simulation Activity to measure and/or foster CST
– Longitudinal Study
• Track relative performance of teams; quantify value of CST
• Role of CST in individual systems thinking development
3rd IEEE Systems Conference
March 24-26, 2009
© 2009 Caroline Twomey Lamb
Massachusetts Institute of Technology
25