LAI Research Seminar
S
Performing Collaborative, Distributed
Systems Engineering (CDSE)
Lessons Learned from CDSE
Enterprises
Darlene
D
l
A
A. Utt
Utter
February 28, 2007
Presentation Overview
•
•
•
•
•
•
•
•
•
•
•
Research Motivations
CDSE Definitions
Literature Search Overview
Research Questions
Research Method Overview
Case Study Overview
D t A
Data
Analysis
l i O
Overview
i
Research Limitations
Research Results
Conclusions
Proposed
p
Future Work
Darlene A. Utter
2
Presentation Overview
•
•
•
•
•
•
•
•
•
•
•
Research Motivations
CDSE Definitions
Literature Search Overview
Research Questions
Research Method Overview
Case Study Overview
D t A
Data
Analysis
l i O
Overview
i
Research Limitations
Research Results
Conclusions
Proposed
p
Future Work
Darlene A. Utter
3
Research Motivation
• Advances in information technology enables collaboration.
• Increasing system complexity requires collaboration.
• Increasingly limited resources necessitates collaboration.
Resulting in…
Collaborative, Distributed Systems Engineering (CDSE)
• Enterprises perform CDSE to remain competitive, because
they have to, and to overcome limited resources, but do they
know how to do it successfully?
– Past Research: Distributed design
g teams differ from traditional face-to-face
teams.
– Current Ongoing Research: Does not yet identify critical technical and social
methods and factors that enable teams to successfully handle the complexity
i
introduced
d
db
by di
distribution.
ib i
Darlene A. Utter
4
Motivation: Current CDSE Scenario
• “Because of the time difference, we have less than 2 hours of useful
collaborative work time across all sites.”
• “The
The network is always down and it takes hours to re
re-boot.
boot ”
• “I get kicked off of Sametime every 15 minutes – how can I attend a
virtual meeting?”
• “We
We spent 45 minutes discussing parameter ‘ABC’
ABC in a telecon
telecon,
before someone finally asked ‘what does ‘ABC’ mean to you?’”
• “ We never can trust ‘A’ to complete their work – so we wind up doing
their work anyway.”
y y
• “Their accents at ‘B’ are so bad, we can barely understand a word
they say, but we are too embarrassed to ask them to keep repeating
themselves.”
• “All off our system
t
interfaces
i t f
are ‘messed
‘
d up’.”
’”
• “We have no idea what they are really doing with all of those
proprietary agreements preventing us from seeing anything.”
Darlene A. Utter
5
Research Motivation: From Literature
• We currently lack defined successful CDSE practices.
((Hammond et al., 2001).
)
• “[T]he one major reason that virtual teams fail, when compared
with face to face teams, is because they do not change their
working environment and processes to accommodate distributed
collaboration.”
ll b ti ” (Li
(Lipnack
k and
d St
Stamps, 2000)
• “Everything that goes wrong with in-the-same-place teams also
plagues virtual teams – only worse.” (Lipnack and Stamps, 2000)
• Possible
P
ibl applicability
li bilit tto ffuture
t
engineering
i
i projects
j t (E
(Ex. S
SoS).
S)
• Be armed in advance of potential CDSE pitfalls and plan
accordingly. (Komi-Sirvio et al, 2005)
• CDSE iis customer-driven
t
di
and
d exemplified.
lifi d (Crisp
(C i and
d Chen,
Ch
2002)
• Use successful CDSE practices for competitive advantage.
(Harvey and Koubek
Koubek, 2000
2000.))
Darlene A. Utter
6
Presentation Overview
•
•
•
•
•
•
•
•
•
•
•
Research Motivations
CDSE Definitions
Literature Search Overview
Research Questions
Research Method Overview
Case Study Overview
D t A
Data
Analysis
l i O
Overview
i
Research Limitations
Research Results
Conclusions
Proposed
p
Future Work
Darlene A. Utter
7
Some CDSE Definitions…
•
•
•
•
•
•
Collaborative Teams: Individuals or groups of individuals who are working
together toward the same common goals. (Lipnack and Stamps, 2000)
Distributed/Dispersed Teams: Individuals in the same organization or on the
same project that work collaboratively from different geographic locations.
Traditional SE Environment: Collaboration in the aerospace and defense
industry are not new; however their form has shifted. Traditionally, a great
deal of money and resources went into engineer travels and co
co-location
location of
development teams.
CDSE: Collaborations between individuals or teams from within the same
company or from different companies performing SE activities from
geographically distributed locations with the same (or a set of) common
goals.
Virtual Teams: “A group of people who work interdependently with a shared
purpose across space, time, and organizational boundaries using
technology ” (Lipnack and Stamps
technology.
Stamps, 2000) CDSE teams are a type of virtual
team.
Successful: Satisfying or exceeding the customer’s expectations and being
as close to “on schedule” and “within budget” as possible. Not wasting
resources (personnel
(personnel, money
money, time
time, etc
etc.).
)
Darlene A. Utter
8
Presentation Overview
•
•
•
•
•
•
•
•
•
•
•
Research Motivations
CDSE Definitions
Literature Search Overview
Research Questions
Research Method Overview
Case Study Overview
D t A
Data
Analysis
l i O
Overview
i
Research Limitations
Research Results
Conclusions
Proposed
p
Future Work
Darlene A. Utter
9
Literature Search Overview
• Reviewed 60+ journal articles, papers, theses, books, and online
publications on a variety of CDSE-related topics.
• Topics Include: SE
SE, Collaboration Frameworks
Frameworks, Collaborative Decision
Making, Virtual Teaming, IT, Collaboration Tools, Distributed Software
Engineering and Manufacturing, Social and Cultural Factors in CDSE
g and Data
environment,, Collaboration Barriers and Benefits,, Knowledge
Management…
• Used literature search to define factors of interest and develop interview
questions.
Social Factors:
Cultural Differences
Career Development Advancement
Distributed Teamwork
Communications
Working w/ IT and tools
Project
j
Management
g
Darlene A. Utter
CDSE
Technical Factors:
SE Process/Architecture
Distributed Decision Making
Tools/Information Technology
Knowledge Management
Cost & Schedule
Product Impact
p
10
Literature Review Key Take-Aways
IN COLLABORATIVE, DISTRIBUTED ENVIRONMENTS…
• Good SE is necessary: A design mistake discovered in manufacturing costs ~100
times more than if it was discovered during design. (Harvey and Koubek, 2000)
• The ability to see teammates is not important. (Harris,
(
2001))
• Group decision making leads to lower quality decisions due to social pressure.
(Hammond et al, 2001)
general is
• In the absence of some channels of communication, communication in g
altered. (Hammond et al, 2001)
• Group communication has fewer messages with more task orientation and less
spontaneity. (Hammond et al, 2001)
g
of a leader is not as p
prominent. ((Hammond et al, 2001))
• The emergence
• Lack of knowledge results in re-work, no work, and mistaken assumptions of
leadership. (Komi-Sirvio et al, 2005)
• Significantly more effort is required for up-front planning in order to manage a project
y (Komi-Sirvio
(
et al,, 2005))
successfully.
• Trust is necessary – it enables open communication, inspires confidence in the final
product, and cooperation between teams. (Lipnack and Stamps, 2000)
• Leadership must provide their teams with a common vision, purpose and goals to
g the organizations
g
and create team identity.
y ((Lipnack
p
and Stamps,
p , 2000))
align
Darlene A. Utter
11
Presentation Overview
•
•
•
•
•
•
•
•
•
•
•
Research Motivations
CDSE Definitions
Literature Search Overview
Research Questions
Research Method Overview
Case Study Overview
D t A
Data
Analysis
l i O
Overview
i
Research Limitations
Research Results
Conclusions
Proposed
p
Future Work
Darlene A. Utter
12
Exploratory Research Questions
1.
How can distributed enterprises successfully collaborate to perform
systems engineering?
–
–
–
–
–
–
–
–
–
–
–
Use IT and Collaboration Tools
Schedule and Conduct Meetings
Communicate
Train Engineers
Overcome Social and Cultural Differences
Make Decisions
Adapt the Product
Overcome Issues and Barriers
Determine or Measure CDSE Benefits
Manage Knowledge and Data
Coordinate Processes
2.
What lessons can be learned and success factors developed from
enterprises currently performing CDSE?
3.
What are the key
y areas for future CDSE research based on exploratory
p
y
research?
Darlene A. Utter
13
Presentation Overview
•
•
•
•
•
•
•
•
•
•
•
Research Motivations
CDSE Definitions
Literature Search Overview
Research Questions
Research Method Overview
Case Study Overview
D t A
Data
Analysis
l i O
Overview
i
Research Limitations
Research Results
Conclusions
Proposed
p
Future Work
Darlene A. Utter
14
Research Methods Overview
• Literature search was used to identify key focus areas and Q’s for CDSE
research.
– Approximately
pp
y ~50 q
questions
• Interview questions were refined after completion of 5 pilot interviews
with SE experts (20+ years of SE experience).
– A separate questionnaire was created for SE leadership
• 2 case studies
t di were carried
i d outt att 2 US d
defense
f
companies.
i
• 21 semi-structured formal interviews were carried out with SE leaders,
systems engineers, and SE support staff (tools/IT/process experts).
• Participating companies and interviewees are anonymous.
• Each interview lasted from ~1-3 hours (Average 1.5 hrs)
• Interview Process:
–
–
–
–
Interviewee was given short overview of research.
Interviewee signed COUHES release forms.
Q&A: Interviewee responses were hand-written at time of interview.
Post interview, handwritten notes were transcribed into a typed transcript
((~4-10pages).
p g )
– Transcription was sent to interviewee via email for approval.
Darlene A. Utter
15
Presentation Overview
•
•
•
•
•
•
•
•
•
•
•
Research Motivations
CDSE Definitions
Literature Search Overview
Research Questions
Research Method Overview
Case Study Overview
D t A
Data
Analysis
l i O
Overview
i
Research Limitations
Research Results
Conclusions
Proposed
p
Future Work
Darlene A. Utter
16
Case Study Company Summaries
Company Information
Overall Company/Business Unit Size
Number of Company/Business Unit Locations
Program Information
Overall Program Size within Company
Projected Program Lifecycle Time
Program Customer
Project Type
Company Role in Program
SE Stage
g of Development
p
Number of Major Collaborating Companies
(Including Company)
Approximate # of Collaboration Sites
Collaboration Locations for CDSE Efforts
Time Zones Involved for CDSE Efforts
Darlene A. Utter
Company A
Company B
13,000+ people
18
1000+
5
Program A
Program B
1800+ people
12 years
US Government
Defense
Prime Mission Systems Integrator
g and Integration
g
Detailed Design
350+ people
10 years
US Government
Defense
Prime Contractor and System Integrator
Detailed Design
g
5
6
Multiple Companies in over 45 states
Multiple Companies in over 5 US states and
the UK
Massachusetts, California, New Jersey,
Washington D.C., New Hampshire, Florida, Massachusetts, California, Florida, Indiana,
Indiana, Maryland, Virginia, Colorado,
Minnesota, UK
Mississippi, New York, Minnesota
4: EST, CST, MST, PST
4: EST, CST, PST, WET
17
Presentation Overview
•
•
•
•
•
•
•
•
•
•
•
Research Motivations
CDSE Definitions
Literature Search Overview
Research Questions
Research Method Overview
Case Study Overview
D t A
Data
Analysis
l i O
Overview
i
Research Limitations
Research Results
Conclusions
Proposed
p
Future Work
Darlene A. Utter
18
Data Analysis
Data Analysis
Example
A
Company
Transcripts
OR
B
Collaboration
C
ll b ti
Situation and
Management
Collaboration
Tool Use
Knowledge,
K
l d
D
Data
t
and Decision
Management
SE Processes
and Practices
Description
Issue or
Barrier
Recommendation
Lesson
Learned
CDSE S
Social
i l
CDSE
and Cultural
Benefits and
Environment
Motivation
Interview Heading Topics
Success
Factor
Irrelevant
Interviewee Experience
p
Tool Training
Tool Access
Network
Reliability
y
Tool Versions
Learning
Curves
Classified
Data
Subtopic
Darlene A. Utter
19
Presentation Overview
•
•
•
•
•
•
•
•
•
•
•
Research Motivations
CDSE Definitions
Literature Search Overview
Research Questions
Research Method Overview
Case Study Overview
D t A
Data
Analysis
l i O
Overview
i
Research Limitations
Research Results
Conclusions
Proposed
p
Future Work
Darlene A. Utter
20
Results: CDSE vs. Traditional SE Differences
CDSE vs. Traditional SE Environment
1
2
3
4
5
6
7
8
9
10
11
There is a great deal more "up-front" work to coordinate SE efforts, teams, resources,
etc.
Communications in a CDSE environment are in general more difficult and facilitated by
the introduction of and reliance on collaboration tools.
CDSE meetings are more formal, thus there is less brainstorming and social
interactions amongst teams.
New and different processes are standardized
standardized, mandated and followed
followed.
There are additional obstacles and complexities: company proprietary data sharing,
corporate fire-walls, non-disclosure agreements, classified data transfer.
Untraditional organizational channels are used to enforce all developers to use the
agreed upon processes
processes.
Centrally collected raw data metrics are used to measure relative company
performance.
New and different SE management positions are created to coordinate efforts.
C ll b ti creates
Collaboration
t a ""one tteam"" or ""one goal"l" workk arrangement,t where
h allll
contractors are working toward the same final, integrated product.
It is more difficult to allocate or re-allocate resources as changes occur, since formal
contracts with schedules and resource allocations are typically done way in advance of
program execution.
ti
There are more discussions and more frequent interactions among teams.
Darlene A. Utter
A
B
A and B
X
X
X
X
X
X
X
X
X
X
X
21
Results: Collaboration Scenario Management
•
•
Issues and Barriers:
– Widespread process enforcement.
– Sustainable SE support structures.
structures
– Lack of previous experience – leads to underestimated efforts.
– Time differences.
– Time wasted on collaboration activities because tools/processes do not
support.
t
– Difficult to allocate or re-allocate resources upon change.
Success Factors and Lessons Learned:
– Contractually obligate following of SE processes and practices
practices.
– Additional up-front work necessary to facilitate SE collaboration.
– Determine and monitor key metrics.
– Require 100% meeting attendance.
– Create “middle management” positions to coordinate SE efforts between
sites.
– Rotate face-to-face meetings.
– Process improvement initiatives
initiatives.
– Have dedicated SE support resources.
Darlene A. Utter
22
Results: Collaboration Tool Use
•
•
Issues and Barriers:
– Not all sites use or have access to all tools they need.
– Tool use learning curves
curves.
– Lack of availability and access to classified data networks.
– There are significant delays to obtain accounts for tool access.
– Transfer of data between unclassified and classified networks.
– Tool use processes are ambiguous – artifacts are therefore not consistent.
Success Factors and Lessons Learned:
– Offer a wide variety of collaboration and development tool use training,
including online courses and classroom training.
training
– Create centralized databases for storage and access to tools.
– Use the simplest tools where possible.
– Determine which engineers need access to which tools in advance.
– Tailor tool training for specific applications.
– Limit the tool features to facilitate consistency of use.
– Establish processes and guidelines to determine how and when tools
should be updated.
updated
Darlene A. Utter
23
Results: Knowledge, Data, and Decision
Management
•
•
Issues and Barriers:
– Decisions are slower, hard to disseminate, record, and track.
– There is not enough time or resources to properly document all decisions
decisions,
design rationales, and tacit knowledge.
– Difficult to disseminate information, changes, and decisions in a timely
manner.
– Company proprietary data agreements and data classification differences
impede the free flow of information between teams.
– There are no tools currently in use to document meeting minutes, action
items and meeting attendance.
items,
attendance
Success Factors and Lessons Learned:
– Define team glossary/dictionary.
– Record and share meeting minutes
minutes.
– Capture program lessons and recommendations for future use.
– Use document standards and templates to improve consistency.
– Use accessibility tool features and attendance to provide accountability
accountability.
– Use the processes and tools in place to your advantage.
Darlene A. Utter
24
Results: SE Processes and Practices
•
•
Issues and Barriers:
– Product integrity may be negatively impacted due to lack of total system
visibility and different company priorities.
priorities
– Differences in company design philosophies complicates development.
– System architecture mirrors that of program and contract, and therefore
g advantage
g of common services.
mayy not be taking
Success Factors and Lessons Learned:
– Coordinate all aspects of SE product development with formal,
management enforced processes.
– Update the collaboration and product development tools to support SE
processes and facilitate their following.
– Use integrated modeling and simulation approaches to catch interface
issues/defects in phase and facilitate cross-company
cross company integration
integration.
– Be specific as early as possible.
– Ensure that SE development tools are widely available to all teams.
– Perform process improvement initiatives to improve product and team
performance.
Darlene A. Utter
25
Results: Social and Cultural Environment
•
•
Issues and Barriers:
– Tensions between competing companies.
– Multi
Multi-sensory
sensory engagement and constant “fire-fighting
fire fighting.”
– Trust – complicated by competitive environment and proprietary data issues,
and lack of opportunity to build relationships
– Different company management structures and decision making cultures make
it diffi
difficult
lt tto workk att engineer-engineer
i
i
llevel.
l
– Communication more difficult.
– Language and interpretation issues.
– Relationships are difficult to establish
establish.*
– More formal work environment with less brainstorming.
Success Factors and Lessons Learned:
– Have
a e tea
team glossary
g ossa y a
and
d ac
acronym
o y d
dictionary.
ct o a y
– Sponsor social events to foster relationship building.
– Use SE metrics to determine if problems exist at off-site locations.
– Offer training to foster relationships and improve tool comfort levels.
– Global team rules and team paraphernalia foster “one team” mentality.
Darlene A. Utter
26
Presentation Overview
•
•
•
•
•
•
•
•
•
•
•
Research Motivations
CDSE Definitions
Literature Search Overview
Research Questions
Research Method Overview
Case Study Overview
D t A
Data
Analysis
l i O
Overview
i
Research Limitations
Research Results
Conclusions
Proposed
p
Future Work
Darlene A. Utter
27
Conclusion: Overall CDSE Benefits
CDSE Benefits Provided by Interviewees
1
2
3
4
5
6
7
8
9
10
12
12
13
Less travel (or a lot less travel than there otherwise would have been without the addition of the collaboration
tools).
Time differences allow the east coast teams to work ``un-interrupted'' by the west coast teams during the
morning hours.
Different industrial and experiential backgrounds allows the program to take advantage of the expertise of the
national defense industry.
Team members get to experience diversity in many things: companies
companies, cultures
cultures, people
people, ideas
ideas, etc
etc.
Technology enables engineers to not have to travel, allowing the engineers to save time and remain with their
families. It also saves the company and the program time and money.
By having the collaboration tools in place, impromptu or emergency meetings can be called on short notice;
whereas in a traditional environment, an entire day of travel may have been needed to have a face-to-face
meeting
ti with
ith th
the customer,
t
etc.
t
Ability to have a challenging and rewarding job position.
This environment forces us to enforce the processes, standards, and documents.
There is a greater level of predictability (in people and products), since the processes are well-followed.
Because employees can still live where they want, there is a larger pool of applicants, and therefore the program
gets better qualified and happier engineers.
Get to work with customer as a partner.
Exposure to a broad range of information and personnel. The increased exposure leads to a great deal of
interactions (different processes, ideas, practices, cultures, etc.), which overall leads to improvements in SE.
Resources are more widelyy available and can be better allocated and allocated more quickly.
q
y
Darlene A. Utter
A
B
A and B
X
X
X
X
X
X
X
X
X
X
X
X
X
28
Conclusion: Overall Research Themes
CDSE T
Topic
i
Collaboration Situation and Management
Collaboration Situation and Management
Collaboration Tools
Collaboration Tools
Knowledge, Data, and Decision Management
Knowledge, Data, and Decision Management
Knowledge, Data, and Decision Management
Knowledge, Data, and Decision Management
SE Processes and Practices
SE Processes and Practices
SE Processes and Practices
Social and Cultural Environment
Social and Cultural Environment
Darlene A. Utter
Proposed
P
d Successful
S
f l CDSE Theme
Th
Need well-defined SE and program organizational structure, with additional middle-management to
coordinate efforts across and companies.
Need management buy-in and wide-spread enforcement of the processes.
Collaboration tools are critical ; the better the tools and the processes in place for their use, the
more "distributed''
distributed the work and the less resources that are wasted
wasted.
Widely available collaborative product development tools are also needed to support successful SE
and development.
Transference and sharing of classified data is an issue that affects almost all aspects of CDSE.
Need to dedicate resources.
Company
p yp
proprietary
p
y data development
p
and sharing
g affects almost all aspects
p
of CDSE, including
g
trust, product integration, system cohesiveness, and information dissemination. Need to dedicate
resources.
Better methods are needed to facilitate dissemination of information and decisions to teams. Need
to dedicate resources.
Tools and processes are needed for successful knowledge, data, and decision management
practices
ti
tto address
dd
many iissues.
Formal, contractually obligated, and publicized SE processes are needed to control all aspects of
systems development.
Widely available and platform independent system modeling and simulation tools are needed to
confine system defects in phase and facilitate system-wide integration.
System interfaces especially those that cross company boundaries
boundaries, are a problem area
area.
Recognize this issue, the importance of system interfaces, and dedicate resources early on to
better define and monitor interfaces.
Program kick-off face-to-face, and regularly scheduled face-to-face meetings are necessary to
build and maintain relationships and trust between teams.
Have team-building activities or social events - the CDSE social environment is believed by many
to be more formal, and almost all interviewees suggested team social events as a mechanism to
improve relationships.
29
Questions?
Darlene_A_Utter@raytheon.com
Darlene A. Utter
30
References (1)
Tiranee Achalakul, Booncharoen Sirinaovakul, and Nion Nuttaworakul. Virtual laboratory: A distributed
collaborative environment. Computer Applications in Engineering Education, 12(1):44 – 53, April 23
2004.
g D. Ballentine,, Angelique
g q Lee,, and Carole Townsley.
y Collaborative technology
gy tools for
Jack D. Becker,, Roger
virtual teaming. In Proceedings of the 5th Americas Conference on Information Systems, Milwaukee,
WI., 1999.
Linda Candy and David Harris. Evaluating model data exchange between systems engineering tools. Systems
Engineering, 4(1):13–23, February 13 2001.
John Cleveland. Toyota’s other system - this one for product development. Automotive Design and Production,
(F b
(February
2006)
2006):18–22,
18 22 February
F b
2006.
2006
Harry E. Crisp and Pin Chen. Coalition collaborative engineering environment. INCOSE Insight, 5(3):13–15,
October 2002.
Jose Martin Molina Espinosa and Khalil Drira. A multi-modal coordination service for cooperative distributed
systems engineering. In International Conference on Systems, Man and Cybernetics, volume 6,
Tunisia 2002
Tunisia,
2002. IEEE
IEEE.
Jose Martin Molina Espinosa, Khalil Drira, and Thierry Villemur. The responsibility management system for
collaborative meetings scheduling in the distributed system engineering project. In Workshop on
Knowledge Media Networking. IEEE, 2002.
Bjorn Fagerstrom and Lars-Erik Olsson. Knowledge management in collaborative product development. Systems
Engineering 5(4):274–285
Engineering,
5(4):274–285, May 4 2002.
2002
Michael Hammer and Steven Stanton. How process enterprises really work. Harvard Business Review,
77(6):108–112, November/December 1999.
Janeen Hammond, Richard J. Koubek, and Craig M. Harvey. Distributed collaboration for engineering design: A
review and reappraisal. Human Factors and Ergonomics in Manufacturing, 11(1):35–52, 2001.
David Harris
Harris. Supporting human communication in network-based systems engineering.
engineering Systems Engineering
Engineering,
4(3):213 – 221, August 9 2001.
Darlene A. Utter
31
References (2)
David Harris. Standardised model data exchange for dispersed systems engineering design teams. In
Proceedings of the IFIP TC5 WG5.3/5.7/5.12 Fourth International Conference on the Design of
Information Infrastructure Systems for Manufacturing, pages 340–351. Kluwer, B.V., 2001.
Craig M
M. Harvey and Richard JJ. Koubek
Koubek. Cognitive
Cognitive, social
social, and environmental attributes of distributed engineering
collaboration: A review and proposed model of collaboration. Human Factors and Ergonomics in
Manufacturing, 10(4):369–393, September 14 2000.
INCOSE. What is systems engineering? 2004. Available from:
http://www.incose.org/practice/whatissystemseng.aspx.
International Council on Systems Engineering Tools Database Working Group (TDWG)
(TDWG). Tools database
database. 2006
2006.
Available from: http://www. incose.org/ProductsPubs/products/toolsdatabase.aspx [Accessed October
11, 2006].
Sieja Komi-Sirvio and Maarit Tihinen. Lessons learned by participants of distributed software development.
Knowledge and Process Management, 12(2):108–122, 2005.
p
and Jeffrey
y Stamps.
p Virtual
Jessica Lipnack
Teams: People Working Across Boundaries with Technology. John Wiley and Sons, New York, NY,
2nd edition,2000.
Azad Madni, Weiwen Lin, and Carla C. Madni. Ideon: An extensible ontology for designing, integrating, and
managing collaborative distributed enerprises. Systems Engineering, 4(1):35–48, September 28 2004.
Aurora Vizcano,, Mario Piattini,, Manuel Martnez,, and Gabriela Aranda. Evaluating
g collaborative applications
pp
from a
knowledge management approach. In 14th IEEE International Workshops on Enabling Technologies:
Infrastructure for Collaborative Enterprise (WETICE’05), pages 221–225. IEEE Computer Society,
2005.
Randy C. Zittel. Simulation based (systems) engineering or collaborative based acquisition. In Ninth Annual
International Symposium of the International Council on Systems Engineering, Systems Engineering:
Sharing the Future,
Future Brighton
Brighton, England
England, 1999.
1999
Darlene A. Utter
32
Back Up Slides
Darlene A. Utter
33
Limitations of Research Findings
• Small sample size, relative to the size of the
programs.
programs
• Limited access to and no inclusion of company
proprietary data in findings.
• No access to or inclusion of classified data in
findings.
• Data collected limited to single-company
single company inputs
inputs,
“one-sided.”
• Interview sample skewed to SE managers and subt
team
leads.
l d
• Examined limited breadth, depth and lifecycle
phase of SE activities.
p
Darlene A. Utter
34
Recommendations for Future Work
• Address existing CDSE issues identified in research.
• Expand current research with additional case studies over longer time
frame.
• Is CDSE only for some systems engineers/personalities?
• Is there a preferred system architecture to support CDSE?
• Can all systems be developed using CDSE?
• How does the organizational structure affect CDSE and performance?
• What is the “best” relationship between system architecture and team
organization in CDSE?
• Determine methods to improve “engineer-to-engineer” communications.
• Are there better ways to build/expedite collaborative relationship
building?
g
• Propose creation of a SE collaboration maturity factor (like CMMI).
• What types of metrics can be developed and monitored to measure SE
collaboration success?
Darlene A. Utter
35
Literature Review, Topics
((See Thesis,, Chapter
p 2))
•
•
•
•
•
•
CDSE Background Information:
– How has distributed collaboration emerged?
– CDSE Research Motivations and Proposed Benefits
SE vs. software engineering, hardware engineering, and virtual enterprises
Proposed Models and Frameworks for Distributed Collaboration:
– IDEON, an enterprise ontology (Madni et al., 2004)
– SEDRES,
SEDRES standardized
t d di d engineering
i
i communication
i ti (C
(Candy
d and
dH
Harris,
i 2001)
– Virtual Laboratory (Achalakul et al., 2004)
– TPDS (Cleveland, 2006)
Collaboration Tools and Information Technology:
gy
– Existing collaboration tools and SE tools database (Becker et al, 1999) (INCOSE,
2006)
– Proposed new collaboration tools (Espinosa and Drira, 2002) (Espinosa et al.,
2002))
– Collaboration Based Acquisition tools (Zittel, 1999)
Proposed CDSE Success Factors
– Summary of all collected literature findings
Potential CDSE Barriers
– Summary of all collected literature findings
Darlene A. Utter
36
Literature Review, Topics
((See Thesis,, Chapter
p 2))
•
CDSE Factors to Consider:
– Technical: Collaborative, distributed manufacturing (Harvey and Koubek,
2000) (Hammond
(H
d ett al.,
l 2001)
– Technical: Collaborative, distributed software development (Komi-Sirvio et
al, 2005)
– Knowledge
g Management:
g
General ((Nonaka,, 1994)) ((Polyani,
y , 1966))
– Knowledge Management: KM in collaborative distributed environments
(Fagerstrom and Olsson, 2002)
– Knowledge Management: KM Performance Index (Vizcaino et al. 2005)
– Collaborative Decision Making: Decision rights matrix (Hammer and
Stanton, 1999)
– Collaborative, Distributed Negotiations: (Harris, 2001)
– Social and Cultural: 5 team dysfunctions (Vaughn and Fleming, 2006)
– Social and Cultural: Social model for collaborative, distributed work
(Harvey and Koubek, 2000)
– Communication: Human-tool integration (Harris, 2001a) (Harris, 2001b)
Darlene A. Utter
37
Research Approach: Exploratory
PRODUCT
SUCCESSFUL
CDSE
CDSE SUCCESS FACTORS
GUIDELINES
COMM
MUNICATION
SO
OCIAL &
CU
ULTURAL
Darlene A. Utter
SE PRACTICE
ES
SUPPORT
KNO
OWLEDG
GE
MAN
NAGEMEN
NT
FACTORS
INFORMATION TECHNOLOGY & TOOLS
38
Research Question Categories
•
There were 10 major categories of questions.
– Note, 2 categories are applicable to leadership interviews
only,
l marked
k d with
ith “ ** ”
1. Basic Information about Interviewee
2. Program-Specific
g
p
Information**
3. Current Collaboration Situation
4. Programmatic Issues**
5 Collaboration Tools and Information Technology
5.
6. Knowledge and Data Management
7. Technical Product Issues
8. Social and Cultural Effects
9. CDSE Benefits
10. CDSE Motivation,, Success,, and Future Work
Darlene A. Utter
39
Data Analysis Overview
• Data for each case study was analyzed separately using manual
coding:
– Heading: For each case study, the coded data from each
interviewee was grouped together according to interview heading
topic.
– Experience: Once all data was organized into interview heading
topics, the transcripts were manually coded by interviewee
experience, specifically: description, issue or barrier,
recommendation, lesson learned, success factor, or irrelevant.
– Subtopic:
S
Once
O
all data was organized by heading and experience,
the data was further coded and grouped into common subtopics, for
example “trust,” “email” or “tool training.” There was not a defined
list of subtopics,
p , as theyy varied from topic
p to topic
p and were
sometimes unique to the case study.
• A consolidated case study analysis was also performed by combining
findings at the “experience” level for each case heading.
Darlene A. Utter
40
Case Study Interview Sample
• Research Sample Breakdown
Category
6
4
% of Total
Sample
47 6
47.6
2
1
6
2
38.1
14.3
Company A Company B
Practicing
P
ti i
Systems
S t
Engineers
E i
Management (Program Managers or SE-related
Technical Directors)
SE Support Personnel
• Histogram of SE Experience (where data was available)
H is to g r a m o f In te r v ie w e e S y s te m s E n g in e e r in g (S E ) E x p e r ie n c e in Y e a r s
Number of Sy
ystems Engineers (eng
gineers)
3 .5
3
2 .5
2
1 .5
1
0 .5
0
0 -2
3 -5
5 -1 0
1 0 -2 0
20+
30+
40+
N u m b e r o f Y e a r s o f S E E x p e r ie n c e (Y e a r s )
* The SE Support Personnel are not included in this histogram. Data was unavailable from six managers at Company B.
Darlene A. Utter
41