DESIGN PROCESS COMMUNICATION METHODOLOGY
A DISSERTATION
SUBMITTED TO THE DEPARTMENT OF
CIVIL AND ENVIRONMENTAL ENGINEERING
AND THE COMMITTEE ON GRADUATE STUDIES
OF STANFORD UNIVERSITY
IN PARTIAL FULFILLMENT OF THE REQUIREMENTS
FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY
Reid Robert Senescu
June 2011
Reid Robert Senescu
Abstract
Building project design teams struggle to (1) collaborate around processes within projects, (2)
share processes between projects, and (3) understand opportunities for investment in
improving processes across projects. Overcoming each challenge requires effective and
efficient communication of design processes. Yet, methods for communicating design
processes from the design process communication research field are too cumbersome to be
useful during design, and methods from the project information management research field
focus only on information exchange and not process communication. To address these
limitations, I aggregate findings from organizational science, human computer interaction, and
process modeling fields to develop the characteristics of the Design Process Communication
Methodology (DPCM). DPCM is Computable, Embedded, Modular, Personalized, Scalable,
Shared, Social, and Transparent. Enabling these characteristics, DPCM consists of elements
which represent and contextualize processes and methods that enable designers to capture and
retrieve processes. To test DPCM, I map the elements and method s to the Process Integration
Platform (PIP). PIP is a web tool that enables project teams to organize and share files as
nodes in an information dependency map that emerges as the team works. Results from the
use of PIP in student design charrettes and class projects provide evidence for the power of
DPCM to effectively and efficiently communicate building design processes within project
teams, between project teams, and across project teams. I claim DPCM as a contribution to the
fields of design process management and project information management. DPCM lays the
foundation for commercial software that shifts focus away from incremental and fragmented
process improvement toward a platform that nurtures emergence of (1) improved multidisciplinary collaboration, (2) process knowledge sharing, and (3) innovation-enabling
understanding of existing processes.
iv
Reid Robert Senescu
Acknowledgments
This dissertation would not exist had I not met my advisor Dr. John Riker Haymaker at a
conference in Montreal in 2006. His enthusiasm for taking a multi-disciplinary approach to
solving construction industry design problems was so contagious, it prompted me to leave my
great job and begin my PhD. Without John’s insightful questions, our research group would
not have developed the innovative multi-disciplinary research that the world needs to address
its economic, social, and environmental challenges. John is a dedicated and loyal mentor, but
also a friend that has made my PhD experience unexpectedly enjoyable.
I would also like to thank Prof. Martin Fischer for his enthusiasm for my research and
Prof. Terry Winograd who introduced me to the mind-boggling philosophy of
phenomenology. I also appreciate Prof. Larry Leifer’s helpful comments as I prepared for my
oral exam and Prof. Jeff Heer for serving as chair. Also, throughout my PhD, Dr. John Kunz
has provided constant guidance, support, and encouraging feedback. Along with Dr. Kunz,
Prof. Ray Levitt’s research on Construction industry processes and organizations provided a
stepping stone for my work.
The validation of my work would have been impossible without the dedication and
ingenuity of software architect, David Anderson. David turned my vision of the Process
Integration Platform (PIP) into a reality while also demonstrating successful collaboration is
possible completely based on phone conversations without ever meeting each other in person.
Zheren Zhang and Nam Wook Kim also contributed to PIP. Also, I would like to thank the
over 200 students who used PIP in five Stanford classes, design experiments, and on their own
research projects.
My PhD was supported by generous funding from Arup. Sir Ove Arup founded his
practice in London in 1946 based on a belief in ‘total design’ - the integration of the design
v
Reid Robert Senescu
process and the interdependence of all the professions involved, the creative nature of
engineering, the value of innovation and the social purpose of design. My research was both
inspired and, I hope, contributes to this vision. In particular, I would like to thank the support
of Ibbi Almufti, Stephen Burrows, Chris Field, Anthony Fresquez, Eric Ko, Jason Krolicki,
Chris Luebkeman, Andrew Maher, Andrew Mole, Jim Quiter, Cole Roberts, Martin Simpson,
and Jeremy Watson. Arup is an inspirational organization of individuals dedicated to the
shaping of a better world.
The Center for Integrated Facility Engineering Technical Advisory Committee
provided additional support for my research. Prof. Ron Wakefield and Prof. Guillermo “red
convertible” Aranda-Mena at RMIT University supported my research in Australia. Over 40
professionals in Australia generously spent time with me explaining their design processes.
I would like to thank my colleagues at the Center for Integrated Facility Engineering:
Caroline Clevenger, Victor Gane, Matt Garr, Robert Graebert, Timo Hartmann, Peggy Ho,
Wendy Li, Tobias Maile, Jennifer Tobias, and Ben Welle. Also, there were a few close
friends that supported me as I made the difficult transition back to academic life: Susie Cho,
Matthew Kennelly, Jordan and Tara Parker, and Stephen Wolf.
Most importantly, I would like to thank my mother and father who always encouraged
me to be curious about the universe around me and always supported my quest to investigate
it. My little sister’s incredible perseverance in attaining her Doctor of Medicine (just one year
ahead of me) has been both a motivation and inspiration.
And finally, I would like to dedicate my dissertation to my two grandfathers, Samuel
J. Rubin and Louis Senescu. These great men planted the entrepreneurial spirit in me. They
taught me to not fear failures, that success comes with diligent work, and that success is the
enjoyment of the journey towards the success itself.
vi
Reid Robert Senescu
Table of Contents
Abstract ........................................................................................................................ iv Acknowledgments ......................................................................................................... v Table of Contents ........................................................................................................ vii List of Tables ............................................................................................................... xii List of Illustrations ....................................................................................................xiii Chapter 1: Introduction ............................................................................................... 1 1 Dissertation Overview ..................................................................................................... 1 2 Perspective on Research to Improve AEC Productivity .............................................. 2 3 Narrowing the Scope of the POP Communication Problem ........................................ 6 4 Intuition for Process Communication ............................................................................ 7 5 Explanation of Dissertation ............................................................................................ 9 6 References....................................................................................................................... 11 Chapter 2: Relationships between Project Complexity and Communication ...... 14 1 Abstract .......................................................................................................................... 14 2 Introduction ................................................................................................................... 15 3 Points of Departure in Project Information, Complexity, and Communication ..... 16 3.1 Product Organization Process – an ontology for project information ...................... 16 3.2 Complexity – assessing the challenge in modeling projects .................................... 18 3.3 Communication - exchanging information to create facilities ................................. 19 3.3.1 Collaboration within projects............................................................................. 21 3.3.2 Sharing information between projects ............................................................... 22 vii
Reid Robert Senescu
3.3.3 Understanding information generated across the firm or industry..................... 23 3.3.4 Collaboration, Sharing, Understanding - one for all, all for one ....................... 23 4 Research Method - Case Studies .................................................................................. 24 4.1 Interviewee Selection and Agenda ........................................................................... 24 4.2 Developing Criteria for the Two Assessment Methods............................................ 26 4.2.1 Complexity criteria development ....................................................................... 26 4.2.2 Communication criteria development ................................................................ 27 4.3 Assessing the Case Studies with respect to the Criteria ........................................... 27 4.3.1 Measurement scale for criteria scoring .............................................................. 28 4.3.2 Defining the scope of the case studies ............................................................... 28 4.4 Research Method for Validating Complexity and Communication Assessment
Methods and the Complexity-Communication Relationship ................................... 29 4.4.1 Comparing the scores from the assessment methods with intuitive estimations 29 4.4.2 Evaluating whether the assessment method is calibrated .................................. 30 4.4.3 Evaluating the applicability of the assessment methods .................................... 30 4.4.4 Evaluating the complexity-communication relationship ................................... 30 5 Complexity and Communication Assessment Methods ............................................. 31 5.1 Product-Organization-Process Complexity Criteria ................................................. 31 5.2 Communication Effectiveness and Efficiency Criteria ............................................ 33 6 Results and Discussion .................................................................................................. 37 6.1 Validating the Two Assessment Methods ................................................................ 38 6.1.1 Alignment between intuitive estimations and the scores from assessments ...... 38 6.1.2 Calibrated assessment methods.......................................................................... 39 6.1.3 Applicability of the assessment methods ........................................................... 42 6.2 Discussion of Two Case Studies .............................................................................. 43 viii
Reid Robert Senescu
6.2.1 Stadium project in depth .................................................................................... 44 6.2.2 Low-Rise residential project in depth ................................................................ 46 6.3 Discussion of the Complexity-Communication Relationship .................................. 48 6.4 Limitations and Future Work ................................................................................... 51 7 Conclusion ...................................................................................................................... 53 8 References....................................................................................................................... 54 Chapter 3: Design Process Communication Methodology - improving the
efficiency and effectiveness of collaboration, sharing, and understanding ........... 59 1 Abstract .......................................................................................................................... 59 2 Introduction ................................................................................................................... 60 3 Examples of Collaborating, Sharing, and Understanding Challenges...................... 63 3.1 Designers Struggle to Collaborate ............................................................................ 63 3.2 Designers Struggle to Share Processes ..................................................................... 64 3.3 Designers Struggle to Understand Processes ........................................................... 65 4 Lack of Effective and Efficient Design Process Communication .............................. 65 5 Synthesizing Existing Concepts to Develop DPCM .................................................... 66 5.1 Organization Science to Enable Adoption ............................................................... 67 5.1.1 AEC requires coordination without standardization .......................................... 67 5.1.2 Form new institutions around processes ............................................................ 68 5.1.3 Use processes to structure information for the matrix ....................................... 70 5.1.4 Knowledge Management without management................................................. 70 5.2 Human Computer Interaction to Create and Access Processes ................................ 72 5.2.1 Cognitive Science calls for personalized graphical representations .................. 72 5.2.2 HCI advocates information interaction and visualization .................................. 73 5.3 Process Modeling to Represent Process ................................................................... 74 ix
Reid Robert Senescu
5.3.1 Process models aimed at improving coordination and planning........................ 75 5.3.2 Process models aimed at improving automation ............................................... 76 5.4 Gaps in PIM and DPM Research Relative to Characteristics .................................. 76 5.4.1 Improving communication between AEC professionals ................................... 76 6 Theory - Design Process Communication Methodology ............................................ 80 6.1 Elements and Methods Enabling Characteristics ..................................................... 81 7 DPCM Applied to Observed Problems ........................................................................ 86 7.1 Collaboration with PIP ............................................................................................. 86 7.2 Sharing Processes with PIP ...................................................................................... 88 7.3 Understanding Processes with PIP ........................................................................... 89 8 Validation Metrics ......................................................................................................... 90 8.1 Motivation for the Metrics ....................................................................................... 90 8.2 Effectiveness and Efficiency of Capturing Processes .............................................. 91 8.3 Effectiveness and Efficiency of Using Processes ..................................................... 92 8.3.1 Using processes for Collaboration within projects ............................................ 92 8.3.2 Using processes for Sharing between projects................................................... 93 8.3.3 Using processes for Understanding across the firm or industry ........................ 94 9 Conclusion ...................................................................................................................... 96 10 References....................................................................................................................... 97 Chapter 4: Communicating Design Processes Effectively and Efficiently .......... 107 Abstract ............................................................................................................................. 107 1 Introduction ................................................................................................................. 108 1.1 Research to Improve Design Processes .................................................................. 108 1.2 Communicating Design Processes ......................................................................... 109 1.3 Overview of this Paper’s Layout and Contributions .............................................. 111 x
Reid Robert Senescu
2 Design Process Communication Methodology .......................................................... 112 3 Validation Method ....................................................................................................... 113 3.1 Summary of Validation Methods for Design Process Improvement Research ...... 115 3.2 Test Setup ............................................................................................................... 118 3.2.1 Mock-Simulation Design Charrette setup ........................................................ 118 3.2.2 Ethnographic-Action research in design class projects.................................... 122 3.3 Techniques for Collecting Data and Measuring Results ........................................ 123 4 Results and Discussion ................................................................................................ 125 4.1 Design Charrettes for Collaboration and Sharing .................................................. 126 4.1.1 Capturing processes effectively and efficiently ............................................... 126 4.1.2 Using processes effectively and efficiently to Collaborate within teams ........ 127 4.1.3 Using processes effectively and efficiently to Share between teams............... 128 4.1.4 Using processes effectively and efficiently across projects ............................. 131 5 Discussion of Power and Generality .......................................................................... 136 5.1 Internal Validity ..................................................................................................... 136 5.2 External Validity .................................................................................................... 138 5.3 Future Work ........................................................................................................... 139 6 Conclusion .................................................................................................................... 140 7 References..................................................................................................................... 142 Appendix A: Mock-Simulation Design Charrette Instructions ........................... 148 xi
Reid Robert Senescu
List of Tables
Table 2-1. Interview summary................................................................................................. 26 Table 2-2. Criteria for measuring a project's product, organization, and process complexity. 31 Table 2-3. Criteria for assessing a project’s communication................................................... 34 Table 3-1. Metrics to assess process communication. ............................................................. 95 Table 4-1. Summary of results from the Mock-Simulation Design Charrette. ...................... 130 xii
Reid Robert Senescu
List of Illustrations
Figure 2-1. Diagram of the three types of information (product, organization, and process)
and three types of communication (collaborating, sharing, and understanding). ............ 17 Figure 2-2. Assessment of the intuitiveness of results. For each project, the plot shows a
comparison of complexity and communication assessment methods. In general the
analytical assessments (large data points) are relatively close to the intuitive estimations
(small data points). The largest differences occur for the high-rise commercial and
residential, high-rise commercial, and stadium projects (estimates are 1.0 to 1.1 distance
away from analytical assessment). On the other hand, the university building and lowrise residential were close (0.3). ...................................................................................... 39 Figure 2-3. Assessment of the distribution of complexity and communication criteria. The
number of statements relevant to each criterion varied from 2 to 24. The average score
across all criteria is 2.9, which is expectedly close to three since the one to five scale was
defined based on the interviews. The average of the standard deviations across all
projects is 1.2. This deviation across projects suggests that the assessment criteria are
sufficiently granular to differentiate trends between projects.......................................... 41 Figure 2-4. Assessment of the comprehensiveness of interviews. The figure shows that there
were in general more interview examples related to project communication criteria than
complexity criteria. In fact, the High School project had so few examples of complexity
criteria that it is not considered in assessing complexity-communication trends. ........... 43 Figure 2-5. The relationship between project complexity and communication. Based on the
six AEC projects assessed, communication problems increase with increased product,
organization, and process complexity. This current trend is unsustainable, because
delivering projects of increased environmental, financial, and social sustainability
xiii
Reid Robert Senescu
requires increased complexity. AEC requires IT solutions enabling horizontal trends so
the industry can increase complexity without increased communication problems. ....... 48 Figure 2-6. Disaggregated POP complexity and communication trends. The strongest trends
exist between product and process complexity and collaborating. Little evidence exists
that sharing across projects is impacted by complexity. A less sensitive trend exists
between product and process complexity and a firm’s ability to understand information
across the firm or industry. .............................................................................................. 50 Figure 3-1. Comparison of existing research in PIM and DPM with the DPCM characteristics.
The matrix is not intended to cover all research in these two fields, but to show a few
indicative examples of current gaps in the research. While individual research may
address some of these characteristics, the authors have not found a theory that addresses
all of them. ....................................................................................................................... 79 Figure 3-2. The Design Process Communication Methodology. Elements represent and
contextualize a process and methods enable designers to capture and use the process
model. These elements and methods enable the Characteristics. .................................... 85 Figure 3-3. Collaboration in the Process Integration Platform. Users navigate to the
appropriate process level via the hierarchy view (left) or by double clicking folder icons
(right). Users create nodes, upload files to those nodes, and draw arrows to show
relationships between the nodes. Green highlights indicate the node is up-to-date, and red
indicates an upstream file has changed since the node was uploaded. ............................ 88 Figure 3-4. Sharing processes in the Process Integration Platform. Users search information
dependency paths to find processes with the input available and the output desired. Users
can then copy processes to new projects. ........................................................................ 89 Figure 3-5. Understanding processes in the Process Integration Platform. PIP tracks some
process metrics automatically, so users can evaluate the most popular and timexiv
Reid Robert Senescu
consuming processes. Discussion threads are associated with each node, so project teams
can discuss individual files or entire processes. .............................................................. 90 Figure 3-6. Use of the Process Integration Platform (PIP) by students at Stanford University.
PIP is a process-based file sharing web tool that acts as a model for DPCM. Its use
demonstrates that DPCM can be practically applied and tested. ..................................... 97 Figure 4-1. Summary of the Design Process Communication Methodology. DPCM consists of
seven elements that enable design teams to represent and contextualize processes.
Elements contain methods and attributes (not shown) described in more detail in Senescu
et al. (2011b). ................................................................................................................. 113 Figure 4-2. The Process Integration Platform. PIP is a web tool enabling project teams to
organize and share files as nodes in an information dependency map that emerges as they
work. Users navigate to the appropriate process level via the Hierarchy view or by
double clicking frame icons in a Network view. Users create nodes, upload files to those
nodes, and draw arrows to show relationships between the nodes. Green highlights
indicate the node is up-to-date, and red indicates an upstream file has changed since the
node was uploaded. Users can also search dependency paths to find relevant historic
processes from other projects. Each node contains an information ribbon providing
additional process information and the opportunity to rate process productivity or
comment. ....................................................................................................................... 114 Figure 4-3. Mock-Simulation Design Charrette test setup. All teams open up PIP to see the
historic project frames (a). Control teams can open any of the six historic project frames
to view the list of tools used to complete that project (b). Experimental teams see the
same list of tools used to complete the project and their dependencies (c). The design
teams work in the “Team C Classroom” frame, which is initially blank, but then becomes
populated with the tools used to design the classroom. The work of the control teams
xv
Reid Robert Senescu
eventually resembles a list similar to (b) and the work of the experimental teams
resembles the network in (c). ......................................................................................... 120 Figure 4-4. Example of a mock-simulation tool. All 20 mock-simulation tools resemble this
Energy Analysis tool. In this case, the Mechanical Engineer finds dependent input values
from the output values of other tools. He then chooses a design by selecting input
independent variables. Clicking the “Analyze” button produces the output values, which
become input to a subsequent tool. ................................................................................ 122 Figure 4-5. Information distribution across projects in a multi-disciplinary design and analysis
class. The shade of each square represents the number of arrows (i.e., dependencies)
from information created by a designer on the top (input) to information created by a
designer on the left (output). Designer names are hidden. A strong single-band diagonal
exists from top left to bottom right because designers depend mostly on information
created by themselves. A wide-band diagonal exists because most designers only depend
on information from within their own team. One exception is the three teams working on
different train stations along the same metro line. The projects are ordered from highest
to lowest project value, suggesting that teams with information distributed across the
team delivered more value (top left) than teams with fragmented and concentrated
information (bottom right). ............................................................................................ 133 Figure 4-6. Frequency and upload latency of information flow between different tools across
all projects in a multi-disciplinary design class. The shade of each square represents the
number of arrows (i.e., dependencies) from information created by a tool on the top
(input) to information created by a tool on the left (output). The size of the square
represents the average latency between when an input file is uploaded and an output file
is uploaded. The visualization enables a manager to understand potentially inefficient
information flows, so that he can invest in improved processes. For example, the large
xvi
Reid Robert Senescu
dark squares represent information flows that are both frequent and potentially time
consuming. ..................................................................................................................... 135 xvii
Chapter 1
Reid Robert Senescu
Chapter 1: Introduction
1
Dissertation Overview
My research asks: How can design processes be communicated effectively and efficiently
within a project team, between project teams, and across multiple project teams? From my
experience as a structural engineer on building projects and from studying the fields of design
process management and project information management I realized that existing approaches
to communicate design processes are either too cumbersome to be useful during design, and
project information management tools are too simple to capture the essence of a design
process. Building on concepts from organizational science, human computer interaction and
process modeling research, I synthesized the Design Process Communication Methodology
(DPCM) as a simple yet powerful methodology that enables design teams to capture their
processes as they work, so they can then use these process models to collaborate within
project teams, share processes between project teams, and understand processes across the
firm or industry to invest in improvement.
This dissertation follows the three journal paper format whereby chapters 2, 3 and 4
motivate, develop, and validate DPCM. Chapter 2 establishes design process communication
as an important challenge in the Architecture, Engineering, and Construction (AEC) industry.
Chapter 3 explains how existing research does not adequately address this challenge. The
chapter then builds on existing literature to develop the DPCM and proposes how the
methodology can be validated. Chapter 4 provides evidence that DPCM effectively and
efficiently communicates design processes. I then conclude claiming that DPCM, validated
1
Chapter 1
Reid Robert Senescu
using the Design Charrette and Ethnographic-Action Research Methods, contributes to the
fields of design process management and project information management.
2
Perspective on Research to Improve AEC Productivity
While the majority of my dissertation focuses on demonstrating how DPCM answers my
research question, this introduction explains my personal story leading up to my research
question, as the question itself reflects a predisposition about how researchers can contribute
to the delivery of increased value per man-hour expended in the AEC industry. Delivering
increased value requires consideration of how any particular research effort fits into not only
existing research fields, but also how it impacts industry by fitting into existing products,
organizations, and processes. Describing the AEC industry using an ontology consisting of
product, organization, and process (POP) component, both inter- and intra-related, provided
me with a lens for assessing and addressing my observations of industry (Garcia et al. 2004).
A research effort aimed at increasing value by addressing just one POP component without
considering the context of the others will be limited in its ability to dramatically impact the
industry. That is, the relationships between product, organization, and process are tightly
intertwined.
Product represents “the physical and abstract concepts that describe the artifact itself,
such as the columns and electrical system of a building.” Organization “is the agency and
agents responsible for design and construction of the artifact.” Process is a series of design and
construction tasks that the organization carries out to design and build the product (Garcia et
al. 2004). Professionals in an organization connect each task to form a process by exchanging
information between professionals. The “process of exchange of information between sender
and receiver to equalize information on both sides” is called communication (Otter and Prins
2002). Professionals exchange information about product (e.g., a reinforced concrete beam),
2
Chapter 1
Reid Robert Senescu
organization (e.g., Acme Structural Engineering firm), and/or process (e.g., the tasks outlined
in the reinforced concrete building code for designing a beam). Exchanging information about
a specific process (i.e., communicating process) is itself a process. For at least two decades,
researchers and/or industry have modeled each component of the POP ontology virtually in a
computer (e.g., AutoCAD drawings, Jin and Levitt’s Virtual Design Team (1996), and digital
Gantt charts).
The task of modeling a building (i.e., product modeling) began changing for early
technology adopters with the use of object-based modeling software applications in the late
1990’s. I was exposed to these new object-based software applications with the first version of
Revit Structure in 2005. The industry attached the term Building Information Modeling (BIM)
to any object-based software application and the industry began viewing BIM as a process
itself or as a technology that changes the design process. However, despite the association of
BIM with process change, little commercial development of BIM was process-centric, and
none of the major BIM tools provided explicit support for making processes transparent.1 3d
object-based modeling is still mostly a set of static tools, none of which aid the design team in
explicitly defining, improving, and sharing design processes. Despite the rhetoric that BIM is
a fundamental shift in the design process, the industry generally still plugs the new tools into a
design process that predates computers. When I began this research in 2007, the real impact of
BIM on industry-wide productivity appeared no more disruptive than the introduction of
Computer-Aided Design (CAD). Still, BIM has had two impacts. First, BIM drastically
improved the ability to easily communicate product information to a wider part of an
organization. For example, disciplines could easily understand clashes between different
building parts. Second, the productivity of certain tasks improved drastically mostly on
1
One recently commercialized solution, Scenario Virtual Project Delivery is an exception to
this generalization as it does explicitly link process and product models.
3
Chapter 1
Reid Robert Senescu
account of defined relationships between objects. For example, the defined connectivity of
beams to columns meant that columns could be moved and beams would follow
automatically. Thus, BIM enabled both improved product communication and improved task
productivity. However, while product communication and some task productivity improved
with the capabilities of simulation and modeling tools during the first decade of the 21st
Century, order of magnitude increases in AEC industry productivity remain elusive.
Methods for communicating organization information (e.g., business colleague
networks, decision-making authority, professionals’ skills, etc.) also changed and became
more important as the organizational structure of AEC companies flattened and projects relied
increasingly on the knowledge of specialized professionals. In 2004, I could search my firm’s
intranet people pages to find professionals with skills to help me perform a task, even if I did
not know anyone that knew that person. With tools like Linked-In, I am not only able to see
the skills of people, but also their relationships to people in my social or professional network.
Though many AEC companies have not fully leveraged this technology to enable their
organizations to communicate the relationships between professionals, the technology exists
and will become increasingly important as the organizations needed to deliver AEC projects
become increasingly complex.
While AEC has new technology for both defining the inter-relationships between
product components and organizational components, the industry still primarily communicates
processes via Gantt charts - a method invented one hundred years ago. And in industry, even
Gantt charts are primarily used solely for communicating the inter-related physical tasks of
construction. Professionals in the AEC industry rarely communicate their own tasks at all or at
least not efficiently. Actually, I frequently spoke informally with project managers to tell them
what I was working on and when acting as a manager, I frequently asked my colleagues what
they were working on. Frequently, I would have to ask the project managers in other
4
Chapter 1
Reid Robert Senescu
companies what their colleagues were working on and they would not know, so I would have
to wait for them to ask. Design is a highly interdependent process and designers waste much
time asking what others are working on to avoid working on something that will soon change.
This informal method of communicating process is inefficient.
Despite this inefficiency, the majority of AEC process planning, management, and
visualization research focuses on the construction phase of the building life cycle. It is my
intuition that inefficient and ad hoc design process communication inhibits efforts to improve
the value per man-hour expended in AEC especially since design processes disproportionately
influence the life cycle value of the resulting buildings (Paulson 1976). Two inefficiencies in
particular caused enough personal concern and frustration to prompt my focus on research to
improve design process communication. First, information flow on AEC projects are
inefficient and intermittent (Gallaher et al. 2004). Extrapolating from the report by Gallaher et
al., the global AEC industry wastes $138 billion annually due to inefficient information flow
(Young et al. 2007). This inefficient information flow cannot be improved without first
communicating the processes responsible for the inefficiency.
Second, the financial, environmental, and social impacts of completing an AEC
project are large and increasing. Buildings consume 70% of the U.S.’s electricity and 40% of
raw materials (United States Green Building Council 2007). Exploding population growth
requires more building, requiring more resources, and increasing their relative financial
impacts. More buildings potentially increase environmental impacts. As population density
increases, a building project has increased social impacts. Managing the multi-disciplinary
tradeoffs needed to maximize financial, environmental, and social value requires design
process communication between disciplines.
5
Chapter 1
Reid Robert Senescu
From the above paragraphs, I conclude that new technology has enabled the industry
to communicate product and organization, but not process. Also, project teams define product
component relationships and organizational component relationships, but not the relationships
between the tasks of design professionals. Third, while task productivity may have improved,
overall processes have not improved. Finally, the financial, social, and environmental impacts
of unproductive design processes are especially large and increasing. Improving the value of
the buildings delivered per man-hour expended requires improved design processes which
require process communication (Ford and Ford 1995). If I could have spent 95% of my time as
a structural engineer designing structures, I probably would have remained a structural
engineer. But, I spent much of my time inefficiently or ineffectively communicating process,
not an especially interesting task, and so, I began researching methods for improving the
effectiveness and efficiency of communicating processes. Project productivity cannot increase
until the industry has a method for communicating product, organization, and process. This
revelation led to the focus of my research question on process communication.
My intuition that the industry needed more research on process communication was
based on literature and experience, but I had little claims for generality of the problem. Also,
early in my PhD research my colleagues emphasized the importance of scale. Did the
problem I described only apply to large projects? Is this a new problem or at least an
increasingly important problem? What is the impact of product, organization, and process
complexity on communication?
3
Narrowing the Scope of the POP Communication Problem
My question about POP communication and its relationship with complexity led me to
interview 36 professionals working on eight different AEC projects in Australia. As described
in Chapter 2, these interviews confirmed that communication problems increase with
6
Chapter 1
Reid Robert Senescu
increased project complexity. Though not validated in this dissertation, the interviews led me
to believe that new building information modeling technology enabled improved
communication of product, even on complex buildings. While I did not add BIM maturity to
my assessments of communication and complexity, I am confident that I could have used an
assessment method such as the BIM Maturity Matrix (Succar 2010) to demonstrate that the
effectiveness of product communication increased with increased usage of BIM. Also, I found
no need to explore methods for communicating organization - interviewees did not complain
about their inability to communicate organizational information. Process communication, on
the other hand, seemed to inhibit productivity both for the project teams I interviewed and for
the projects I worked on as a structural engineer. Thus, while Chapter 2 considers
communication of all POP information both in design and construction, Chapters 3 and 4
scopes the research to communication of design processes.
4
Intuition for Process Communication
As described in Chapter 3, the design process improvement literature has spent much time
developing increasingly effective methodologies for communicating design process. In
particular, I saw firsthand the educational value of Narratives in project-oriented design
classes. Narratives are a method for communicating projects by representing product,
organization, and process as a network of nodes linked to digital files (Haymaker 2006). Yet,
during all of my interviews and professional experience, I never saw Narratives or other
methodologies for communicating design processes applied in industry. This absence is not
due to the lack of tools capable of effectively communicating design processes. Microsoft
Visio and Microsoft Project are widely available and most of the methodologies described in
the literature could be applied using these existing tools. Despite a few attempts to pilot the
use of Narratives on professional design projects, use of the Narrative-building tool did not
7
Chapter 1
Reid Robert Senescu
grow in the industry. In general, designers are not willing to spend time communicating their
processes formally. One of the primary reasons for this lack of formal process communication
is that communicating processes does not benefit designers at the instant they are designing.
Thus, it is not sufficient to merely have the methodology and tools to effectively communicate
processes. The act of communication must also be efficient. This necessity for methodologies
and tools to enable both effective (i.e., able and/or accurate) and efficient (i.e., quick and/or
with little effort) process communication is considered throughout the dissertation and is a
fundamental part of my research question.
However, from the interviews and my own experience, there is a benefit to
communicating process both for the individual, for the team, for other teams, and for the entire
firm and industry. Throughout my dissertation, I call the type of process communication that
occurs within teams, collaboration; between teams, sharing; and across the entire firm or
industry, understanding. Most process communication research focuses on just one type of
communication, but all areas agree that communication requires (1) Capturing, (2)
Structuring, (3) Retrieving, and (4) Using processes. Benkler (2002) describes these steps as
part of the information-production chain needed for collaboration in the peer-production
model. Knowledge management research describes these steps as needed for sharing of
processes across projects (Carrillo and Chinowsky 2006; Javernick-Will and Levitt 2010;
Kreiner 2002). Finally, innovation literature cites these steps as required for companies to
understand their processes to make strategic investments in process improvement (Hargadon
and Sutton 2000). The importance of leveraging process communication to benefit
collaboration within the team, sharing of processes between teams, and understanding of
processes across the firm or industry is the final and perhaps most important premise
embedded in my research question. Considering these three types of process communication
8
Chapter 1
Reid Robert Senescu
simultaneously differentiates my research from previous efforts in the project information
management and design process management research fields.
I observed that digital files are the core deliverables of AEC professionals, and
consequently, I hypothesized that the dependencies between these files are indicative of design
processes. I hypothesized that communicating the dependencies between files enables
sufficient interpretation of design processes to improve collaboration, process sharing, and
understanding of processes.
This observation led me to the intuition that designers could communicate their
processes as they managed their information in digital files. Thus, I developed DPCM – a
methodology that enables design teams to save and exchange their digital files as a node in a
file dependency map. By defining the dependencies between files as they exchange
information, designers can communicate their processes within teams, between teams, and
across the firm or industry.
5
Explanation of Dissertation
In Chapter 2, I apply complexity and virtual design and construction research to develop a
method for assessing product, organization, and process complexity. Then, through project
team interviews, I develop a communication assessment method. The method assesses (1)
collaboration within projects, (2) sharing between projects, and (3) understanding of processes
across the firm or industry. I applied these two assessment methods to seven case study design
and construction projects in Australia. Through case study interviews, I validate the usefulness
of the methods and provide evidence of the existence of a trend between increased POP
complexity and increased communication challenges. The two assessment methods provide
the opportunity for teams to learn from and improve upon their communication strategies. By
9
Chapter 1
Reid Robert Senescu
increasing the awareness of the relationship between complexity and communication, Chapter
2 describes the industry problem to motivate and provide foundation for the development of
more efficient and effective communication tools. Chapters 3 and 4 contribute to this
development.
Whereas Chapter 2 discusses product, organization, and process, Chapter 3 scopes the
communication problems to designers (1) struggling to collaborate around processes within
projects; (2) share better processes across projects; and (3) understand the best opportunities
for investment in improving processes across projects. Overcoming each challenge requires
communication of design processes. Chapter 3 aggregates findings from organizational
science, human computer interaction, and process modeling fields to develop the Design
Process Communication Methodology (DPCM). DPCM enables efficient and effective design
process communication through a Computable, Embedded, Modular, Personalized, Scalable,
Shared, Social, and Transparent environment for capture, structure, retrieval, and use of
processes. To test DPCM, the research maps the methodology to software features in the
Process Integration Platform (PIP). PIP is a web tool that enables project teams to organize
and share files as nodes in an information dependency map that emerges as they work.
Chapter 3 closes with evidence of the testability of DPCM and proposes metrics for evaluating
DPCM’s efficiency and effectiveness in communicating design process.
Chapter 4 validates DPCM using the Mock-Simulation Design Charrette Method and
Ethnographic-Action Method applied to a project-based design class. Results demonstrate that
DPCM accurately captures design processes with little effort. When collaborating within
project teams, process clarity and information consistency result in little rework, and positive
iteration enables consideration of multi-disciplinary design trends. With DPCM, designers
share processes between project teams and use the shared processes without committing
process mistakes. DPCM enables the understanding of processes across projects, which
10
Chapter 1
Reid Robert Senescu
provide insights into: how integrating information between project team members impacts
product value; and opportunities for investment in improved information flows.
These findings provide evidence for the power of DPCM to effectively and efficiently
communicate building design processes through collaboration within project teams, sharing of
processes between project teams, and understanding processes across projects to strategically
invest in improvement. I claim that DPCM, validated using the Design Charrette and
Ethnographic-Action Research Methods, contributes to the fields of design process
management and project information management. In terms of broader impact, DPCM lays
the foundation for commercial software that shifts focus away from incremental and
fragmented process improvement toward a platform that nurtures emergence of (1) improved
multi-disciplinary collaboration, (2) process knowledge sharing, and (3) innovation-enabling
understanding of existing processes.
6
References
Benkler, Y. (2002). "Coase's penguin, or, Linux and the nature of the firm." Yale Law Journal,
112(3), 367-445.
Carrillo, P., and Chinowsky, P. (2006). "Exploiting knowledge management: The engineering
and construction perspective." Journal of Management Engineering, 22(1), 2-10.
Ford, J.D., and Ford, L.W. (1995). "The role of conversations in producing intentional change
in organizations." The Academy of Management Review, 20(3), 541-570.
Gallaher, M. P., O'Connor, A. C., Dettbarn, J. L., Jr., and Gilday, L. T. (2004). "Cost analysis
of inadequate interoperability in the U.S. Capital facilities industry." National Institute
of Standards and Technology, GCR 04-967.
11
Chapter 1
Reid Robert Senescu
Garcia, A. C. B., Kunz, J., Ekstrom, M., and Kiviniemi, A. (2004). "Building a project
ontology with extreme collaboration and virtual design and construction." Advanced
Engineering Informatics, 18(2), 71-83.
Hargadon, A., and Sutton, R. I. (2000). "Building an innovation factory." Harvard Business
Review, 78(3), 157-166.
Haymaker, J. (2006). "Communicating, integrating and improving multidisciplinary design
narratives." Second International Conference on Design Computing and Cognition, J.
S. Gero, ed., Springer, Netherlands, 635-653.
Javernick-Will, A., and Levitt, R. E. (2010). "Mobilizing institutional knowledge for
international projects." Journal of Construction Engineering and Management,
136(4), 430-441.
Kreiner, K. (2002). "Tacit knowledge management: The role of artifacts." Journal of
Knowledge Management, 6(2), 112-123.
Otter, A. F. d., and Prins, M. (2002). "Architectural design management within the digital
design team." Engineering, Construction and Architectural Management, 9(3), 162173.
Paulson, B. C. J. (1976). "Designing to reduce construction costs." Journal of the Construction
Division, 102(4), 587-592.
Jin, Y. and Levitt, R.E. (1996). "The virtual design team: A computational model of project
organizations." Computational & Mathematical Organization Theory, 2(3), 171-195.
Succar, B. (2010) "Building information modelling maturity matrix." Handbook of research
on building information modelling and construction informatics: Concepts and
technologies, J. Underwood and U. Isikdag, eds., IGI Publishing, 65-103.
12
Chapter 1
Reid Robert Senescu
United States Green Building Council. Last accessed June 2007, http://www.usgbc.org.
Weber, M. (1947). The theory of social and economic organization, The Free Press, Glencoe,
IL.
Young, N., Jr., Jones, S.A. and Bernstein, H.M. (2007). "Interoperability in the construction
industry." SmartMarket Report: Design & Construction Intelligence, McGraw Hill
Construction.
13
Chapter 2
Reid Robert Senescu
Chapter 2: Relationships between Project Complexity
and Communication
Reid Robert Senescu, Guillermo Aranda-Mena, and John Riker Haymaker
1
Abstract
The Architecture Engineering Construction (AEC) industry delivers increasingly complex
projects but struggles to leverage information technology to facilitate communication on these
projects. To begin to address this challenge, the project information management research
field needs methods to assess communication, complexity, and their relationships. First, the
authors apply complexity and virtual design and construction research to contribute a method
for assessing product, organization, and process (POP) complexity. Second, through project
team interviews, the authors contribute a communication assessment method to assess
collaboration within projects, sharing of information between projects, and understanding of
information generated across the firm or industry. Applying the two assessment methods to
case studies, the authors validate the applicability of the methods to AEC industry and
contribute the establishment of a trend between increased POP complexity and increased
communication challenges. The two assessment methods provide the opportunity for teams to
learn from and improve upon their communication strategies based on project complexity. By
increasing the awareness of the relationship between complexity and communication, the
paper aims to motivate and provide foundation for the development of more effective and
efficient communication tools.
14
Chapter 2
2
Reid Robert Senescu
Introduction
The Architecture Engineering Construction (AEC) industry delivers increasingly complex
projects to meet the financial, social, and environmental goals of stakeholders. Ideally, project
teams would continue to effectively and efficiently communicate despite this complexity. Yet,
even with the increased availability and pervasiveness of Information Technology (IT), project
teams still struggle to communicate (Eckert and Clarkson 2004; Haymaker and Chachere
2011; Luiten and Tolman 1997; Senescu and Haymaker 2008; Senescu et al. 2010). These
struggles limit the ability of project teams to manage complexity to achieve stakeholder goals.
The authors adopt the information processing view of a firm to elucidate the AEC
project as a network of information exchanges (Weber 1947; Galbraith 1974). AEC Projects
consist of an organization implementing a process to deliver a product, such as a building
(Garcia et al. 2004). This product-organization-process (POP) ontology provides a useful lens
for looking at the network of information exchanges on a project. Complexity theory helps
quantify the challenge of representing these projects (Homer-Dixon 2000). Communication
research provides guidance as to how people must exchange these representations by (1)
collaborating within projects, (2) sharing of information between projects, and (3)
understanding of information generated across the entire firm or industry (Senescu and
Haymaker 2009). Despite the importance of effective and efficient communication in
managing POP complexity, the project information management research field lacks a method
to assess the relationship between complexity and communication.
This paper reviews this POP, communication, and complexity research and
synthesizes findings from literature and case study interviews to develop a method for
assessing communication and a method for assessing project complexity (Figure 2-1). To
develop the communication assessment method, the authors collected empirical data through
15
Chapter 2
Reid Robert Senescu
interviews with AEC professionals. In contrast, the authors base the POP complexity
assessment method on complexity literature, and in this case, the interviews confirm the
relevance of this literature to AEC projects.
This paper contributes a method for measuring project complexity and a method for
measuring the effectiveness and efficiency of communication. The two assessment methods
provide the opportunity for teams to learn from and improve upon their communication
strategies. Application of the two assessment methods to the interviews results in the second
contribution: evidence of a trend between increased complexity and increased communication
challenges. By establishing this trend, the authors aim to motivate and provide foundation for
the development of more effective and efficient communication tools.
3
Points of Departure in Project Information, Complexity, and
Communication
3.1
Product Organization Process – an ontology for project information
Using a “POP ontology” to define the project comes from research on virtual design and
construction. The authors choose this ontology because professionals in the industry can
control product, organization, and process to achieve their firm’s goals. Product represents
“the physical and abstract concepts that describe the artifact itself, such as the columns and
electrical system of a building.” Organization “is the agency and agents responsible for design
and construction of the artifact.” Process is the design and construction tasks that the
organization carries out to build the artifact (Garcia et al. 2004). In application, the POP model
generally only lists POP objects without explicitly defining the relationships between the
objects. For example, the entire project team rarely uses parametric product models because of
the opacity of the relationships between product objects (Gane and Haymaker 2010). Also, the
16
Chapter 2
Reid Robert Senescu
relationships (i.e., knowledge sharing networks, decision-making authority, complementary
skills, etc.) between professionals in AEC are rarely transparent (Haymaker et al. 2010). And
finally, professionals are rarely aware of the relationships between their own tasks, let alone
how their tasks interrelate with the entire project team (Eckert and Clarkson 2004; Senescu
and Haymaker 2009). Simply listing objects is conveniently simple, but this lack of
communicating POP relationships is limiting, because the true value of a model comes from
the relationships between the objects (Rechtin 1991).
Figure 2-1. Diagram of the three types of information (product, organization, and process)
and three types of communication (collaborating, sharing, and understanding).
17
Chapter 2
Reid Robert Senescu
The original intent of POP models was to develop increasingly detailed relational
models to “elucidate and eventually mitigate potential risks to project success” (Garcia et al.
2004). As project complexity increases, developing relational POP models becomes
increasingly difficult. Developing a method for assessing the complexity of the product,
organization, and process (i.e., POP complexity) provides a first step toward implementing
this relational POP model.
3.2
Complexity – assessing the challenge in modeling projects
Managing project complexity is a critical factor impacting project success and the application
of conventional management systems to complex projects is inappropriate (Allen 2008;
Baccarini 1996). This paper builds on the following complexity definition (Homer-Dixon
2000) to develop a measurable method for assessment:
1. Multiplicity – number of components.
2. Causal Connections – number of links between components (to the extreme, there is
causal feedback where a change in one component loops back to affect the original).
3. Interdependence – the larger the module that can be removed from the complex
system without affecting the overall system’s behavior, the more resilient and less
complex the system.
4. Openness – to outside environments, not self-contained, difficult to locate boundary.
5. Synergy – the degree to which the entire system is more than the sum of the parts.
6. Nonlinear behavior – the effect on the system is not proportional to the size of the
change on a component.
This fundamental point of departure for the more detailed criteria described in Section 5.1
defines complexity more specifically than other AEC research (Allen 2008; Allen et al. 1985;
Baccarini 1996). Specifically, this paper addresses Baccarini’s call for work that not only
18
Chapter 2
Reid Robert Senescu
explicitly defines complexity but also that determines the relationship between project
complexity and the degree of integration, which can also be assessed by assessing
communication. Outside of AEC, Maier et al. (2008) consider complex product development,
but while many of their communication factors such as “autonomy of task execution” are
indicative of the project’s complexity, complexity itself is not explicitly defined and the
relationship between communication and complexity is not investigated. Otter and Emmitt
(2008) also discuss communication in the context of complexity, but do not explicitly define
the latter.
Before addressing this need to relate communication to complexity in the AEC
context, this paper must first develop the two methods for assessment. As discussed in Section
3.1, firms control product, organization, and process to achieve their goals. Managing POP
requires the communication of POP information. By breaking down complexity using the
same lens used to assess information exchanges, the authors can compare how POP
complexity impacts the communication of POP information. The previous methods of
assessing a project’s complexity do not consider POP explicitly, so these previous methods do
not permit a granular explanation of the mechanism causing complexity to impact
communication.
3.3
Communication - exchanging information to create facilities
Professionals can exchange information about product (e.g., a reinforced concrete beam),
organization (e.g., Acme Structural Engineering firm), and/or process (e.g., the steps outlined
in the reinforced concrete building code for designing a beam). Exchanging information about
a specific process is itself a process. The “process of exchange of information between sender
and receiver to equalize information on both sides” is called communication (Otter and Prins
2002). This definition is consistent with “sharing of meaning to reach a mutual understanding”
19
Chapter 2
Reid Robert Senescu
(Otter and Emmitt 2008) and as a “cognitive and social process by which messages are
transmitted and meaning is generated” (Maier et al. 2008).
AEC literature distinguishes knowledge from information and data (Otter and Prins
2002). This distinction is also common throughout the knowledge management field of which
Foss et al. (2010) provide an overview. The assessment of communication does not require
this distinction, because people must exchange all three types. Furthermore, the distinction is
not useful, because the definitions of each are relative both personally and temporally. For
example, a person may possess information (aware of the data’s relevance and purpose), but
the rest of the project team, ignorant of the relevance and purpose, may consider the
information merely data. Also, people forget the relevance and purpose of data quickly,
therefore relegating current information to future data. Consequently, this paper calls data,
information, and knowledge simply information. Exchange of any information is called
communication (Luiten and Tolman 1997).
While not distinguishing between the exchange of data, information, and knowledge,
this paper does propose to distinguish between three types of information exchange: 1) within
project, (2) between projects, and (3) across firm or industry. Considering all three types of
information exchange explicitly is important, because otherwise, there are “cost-benefit
mismatches” in communication. That is, many efforts to improve communication do not
consider that “the person responsible for recording information is typically not the person who
would benefit from the information once it is recorded” (Eckert et al. 2001). Also, team
members frequently have conflicting obligations to the project and to the firm (Dossick and
Neff 2010). That is, it is possible that within-project communication is excellent, but that
communication between projects or the gathering of information from across the entire firm is
poor. Thus, distinguishing between three types of communication is useful: (1) Professionals
exchange information within project teams in order to collaborate; (2) professionals exchange
20
Chapter 2
Reid Robert Senescu
information between project teams with the intent to share; and (3) professionals gain
understanding of information generated on projects across the firm or industry. Including
these three types in the communication assessment method encourages a holistic approach to
improving communication. The following paragraphs explain the challenges and opportunities
for improvement of each type of communication and relate each type to POP complexity.
3.3.1
Collaboration within projects
Research on the exchange of information within a project is found in collaboration literature
(Kvan 2000) and project information management literature (Froese and Han 2009). Laufer et
al. (2008) and Otter and Emmitt (2008) provide an overview of literature on within-project
communication in AEC, and Laufer claims, but does not provide evidence for, a link between
communication challenges and complexity on construction projects.
Recently, product communication has improved as firms more deeply adopt Building
Information Modeling (BIM) and increasingly exchange BIM between companies on a
project. However, firms have difficulty using BIM for anything more than geometric
coordination (Taylor and Bernstein 2009). While BIM may precipitate more tightly coupled
technology within the project, decision making is often divided between different companies
on the project (Dossick and Neff 2010). Collaboration research does not examine whether
complexity has a negative impact on the type of information exchange that would enable
BIM’s use beyond geometric coordination (e.g., design decision making informed by BIMsupported performance analysis). Taylor and Bernstein call for more research on the impact of
project complexity on the evolution of BIM. As BIM is fundamentally a product
communication tool, an explanation of the impact of complexity on communication would
contextualize Taylor and Bernstein’s work and provide a foundation for further development
of BIM software applications that can accommodate increasingly complex projects.
21
Chapter 2
3.3.2
Reid Robert Senescu
Sharing information between projects
In addition to the challenges of exchanging information within projects, knowledge
management literature focuses on the exchange of information between projects (Carrillo and
Chinowsky 2006; Javernick-Will and Levitt 2010; Kivrak et al. 2008). For example, Conklin
(1996) describes a “project memory system” to define this knowledge and make it available to
other projects. According to Conklin, the project memory system is necessary for knowledge
sharing between projects, because organizations lack ability “to represent critical aspects of
what they know.” Once this system enables knowledge acquisition, Conklin claims the
knowledge must be structured. Hansen et al. (1999) describe two aspects of knowledge
structuring: codification and personalization. Codification relies on IT tools to connect people
to reusable explicit knowledge (Javernick-Will et al. 2008). Personalization relies on
socialization techniques to link people so they can share tacit knowledge. IT can provide the
general context of information and point to individuals or communities that can provide more
in depth knowledge. Knowledge management is not just acquisition and structuring (Kreiner
2002). Will and Levitt (2008) address the additional importance of the future ability of others
to retrieve the collected knowledge to use it. Aiming to improve this retrieval and reuse,
Fruchter and Demian (2002) developed a technical solution called CoMem. With CoMem,
professionals share information between projects by seeing an overview of product
information on other projects, zooming and filtering that information, and then retrieving
detailed product information on demand.2
Conklin, and Fruchter and Demian validate their technical implementations of
information sharing research on simple problems. To assess the state of current information
2
As discussed in Section Chapter 2:3.3, while the knowledge management field describes the
capturing, structuring, retrieving, and use of knowledge, this knowledge is frequently
indistinguishable from information that is shared between projects and then interpreted by
professionals to become knowledge.
22
Chapter 2
Reid Robert Senescu
sharing and its relationship with POP complexity, industry needs methods for assessing the
impact of project complexity on the ability of organizations to share information between
projects.
3.3.3
Understanding information generated across the firm or industry
The third type of communication examined in this paper is the exchange of POP information
from the project level to the firm or industry level. The authors call this gathering of
information from projects across the firm or industry, understanding. In particular, the paper
focuses on the gathering of POP information that enables corporate-level professionals to
sufficiently understand project information such that they can make strategic decisions about
investment in product, organization or process communication technology. Innovation
literature commonly discusses understanding to describe how companies make strategic
investments to innovate. For example, innovation requires a “knowledge-brokering cycle”
consisting of (1) capturing good information, (2) keeping information alive, (3) imagining new
uses for old information, and (4) testing these new uses (Hargadon and Sutton 2000). Industry
lacks a method for assessing its ability to understand POP information across the firm or
industry. Consequently, industry struggles to strategically invest in technology enabling
improved communication of product, organization, and/or process.
3.3.4
Collaboration, Sharing, Understanding - one for all, all for one
Research and industry address these different types of communication independently. For
example, companies have different systems (both technically and organizationally) for project
team collaboration, knowledge management, and research and development. However, at the
same time one is exchanging information to collaborate within a project, that person can also
contribute to sharing information across projects and to understanding of information from
across the firm or industry. The authors intuit that an opportunity exists to simultaneously
23
Chapter 2
Reid Robert Senescu
communicate information within teams, between teams, and across the firm or industry.
However, with current IT, POP complexity stifles the achievement of this communication
convergence. Project information management research requires a communication assessment
method, a complexity assessment method, and a comprehension of the complexitycommunication relationship to motivate and provide foundation for the development of more
effective and efficient communication tools.
4
Research Method - Case Studies
The previous section established a need for a communication assessment method, a
complexity assessment method, and comparison of the relationship between communication
and complexity. This section explains how the authors performed case study interviews. Then
the section explains how the authors developed the two assessment methods. After
establishing a research method to develop the assessment methods, the section discusses how
the authors scored the case studies to validate the assessment methods and evaluate the
communication-complexity relationship.
4.1
Interviewee Selection and Agenda
Using the criteria described by Yin (2003), the authors selected case study projects and
professionals within the selected projects, and prepared an interview agenda. The validity and
reliability of the collected empirical data lies within a constructivist, not a positivist, paradigm
(Crotty 1998). Thus, results are not deterministic but still insightful due to careful selection of
projects and interviewees. Interviews occurred at seven offices in three Australian cities
(Table 2-1). Two of the offices are part of the same firm. At each office, questions focused on
one project, though the authors also interviewed professionals unassociated with the project,
but possessing relevant information about how the office exchanges information within
24
Chapter 2
Reid Robert Senescu
projects, between projects, and across the firm or industry. Also, some of the interviewees
worked on the project at the office but represented, for example, the owner. Interviewees’
roles included architect, structural engineer, drafter, project manager, BIM coordinator,
quantity surveyor, and design technology director.
The authors formulated open-ended questions related to technology, business
processes, knowledge management, information management, communication, and decision
making. Interviews were semi-structured and focused on topics of interest to interviewers and
interviewees rather than on prepared questions. Generally, the authors focused on a particular
process employed by the interviewee on the project, because this focus led interviewees to
discuss specific information that they exchanged with other professionals within the project
and between projects. The authors also asked how professionals exchanged project
information to executives overseeing multiple projects across the firm and how executives
gathered information from projects across the firm or industry.
The information-centric discussion enabled the authors to use the same interviews for
validation of the complexity assessment method and both development and validation of the
communication assessment method. The open-endedness of the interviews enabled the authors
to assess complexity through specific discussions of information rather than the interviewees
own assessment of the complexity of their project. Whereas project complexity is independent
of interviewee perception, the communication assessment is dependent on the interviewees’
perception. In the communication assessments, the open-endedness of the interviews enabled
the authors to determine the communication problems of greatest concern to the interviewees
without being biased by the authors’ preconceived notions of communication problems in the
industry.
25
Chapter 2
Reid Robert Senescu
Table 2-1. Interview summary.
Company Type
Geographic
Distribution
Project Type
Multi-Disciplinary Engineering Firm
Global
University Building
6
Multi-Disciplinary Engineering Firm
Global
Commercial and
Residential High-Rise
7
Architecture and Multi-Disciplinary
Engineering Government Office
Multiple offices
in state
Call Center and Offices
8
Architecture Firm
A few offices in
Oceana
Commercial High-Rise
2
Construction General Contractor
Global
Stadium
9
Architecture Firm
One office
High School
1
Architecture Firm
Global
Low-Rise Residential
3
4.2
Interviewees
Developing Criteria for the Two Assessment Methods
For each assessment method, the authors developed criteria that the authors then used to score
the communication and complexity of the case studies. This section discusses the research
method for the development of the criteria.
4.2.1
Complexity criteria development
The authors developed the complexity criteria by applying Homer-Dixon’s (2000) synthesis of
complexity literature (Section 3.2) to product, organization, and process. Developing the
criteria was straightforward in that complexity literature views any system as objects
connected by relationships. In assessing product complexity, the objects are building objects.
For organizations, the objects are professionals. For process, the objects are tasks. For
example, Homer-Dixon’s concept of multiplicity (see Section 3.2) is applied to develop a
criterion for product complexity: quantity of building components and level of detail at which
26
Chapter 2
Reid Robert Senescu
building components are considered by the project team. Each complexity criterion is a
modification of Homer-Dixon’s criteria such that it fits the POP ontology in the AEC context.
4.2.2
Communication criteria development
To develop criteria for the communication assessment method, the authors repeatedly and
randomly chose two interviews and brainstormed ways the two interviews differed. This
cross-case searching tactic prompted the development of new unanticipated criteria
(Eisenhardt 1989). For example, one collaboration criterion emerged when authors compared
one project team that repeatedly asked for information from other team members (pulled) with
another project team whose infrequent requests for information resulted in responsive, not
proactive, collaboration. By repeatedly comparing random pairs of interviews in this way, the
authors assembled a list of communication criteria. Aggregating the list, each communication
criterion fell into one of three types of communication, which the authors later found to be
consistent with the literature discussed in 3.3. That is, professionals exchange product,
organization, or process information through three types of communication: (1) collaboration
within project teams, (2) sharing information between project teams, and (3) understanding
information across the firm or industry.
4.3
Assessing the Case Studies with respect to the Criteria
The previous section described how the authors developed the criteria that make up the two
assessment methods. This section explains the research method for scoring each case study
with respect to each criterion. As the scoring is relative to the case studies considered by the
authors, the section ends with a discussion of how the authors determined the case study
scope.
27
Chapter 2
4.3.1
Reid Robert Senescu
Measurement scale for criteria scoring
Requiring a method for assessing case studies with respect to complexity and communication,
the authors developed a one to five measurement scale. A “five” represented the most complex
instance encountered in the interviews and a “one” the simplest. Similarly, a “five”
represented the worst communication encountered and a “one” represented the best
communication. The scale was relative to the interviews conducted. That is, if interviews
included a villager building his own grass hut, a “one” would have represented a much simpler
P, O, or P than the simplest project in this research effort. The intent of quantifying this
measurement scale was to identify trends, not absolute levels of complexity and
communication, nor mathematically defined correlations.
4.3.2
Defining the scope of the case studies
The authors assessed complexity at the project level within the scope of the interviewee’s
responsibility. For example, when talking to two professionals about one project, the authors
averaged findings between the two professionals. If the first professional’s scope involved a
small product at a low level of detail (e.g., a “one” score for the multiplicity criterion) and the
second professional’s scope involved a large product at a high level of detail (e.g., a “five”
score for the multiplicity criterion) the two interviews would be averaged and the project
would be assigned a medium (e.g., “three”) for the product multiplicity criterion.
The method for defining scope varied for the three types of communication. The
authors assessed the interviewee’s collaboration on the project, but ignored statements made
about collaboration between others. Thus, an engineer may have small scope and interact
simply and collaborate effectively with one other person, even though the overall project may
have high complexity and horrible communication. The authors assessed sharing with respect
to how the team learned from other project teams or vice versa. The authors assessed the level
28
Chapter 2
Reid Robert Senescu
of understanding of information gathered across the entire firm by assuming that the
interviewees on each case study were representative of the entire firm. In some cases,
interviewees not specifically involved in a project contributed to the assessment of sharing and
understanding.
4.4
Research Method for Validating Complexity and Communication
Assessment Methods and the Complexity-Communication Relationship
The previous sections established research methods to develop the criteria that make up the
assessment methods and a scoring method to evaluate the case studies with respect to each
criterion. To score each case study, the first author coded each interview using the methods
applied by Erdogan (2008). For all the interviewee statements that the first author considered
relevant to one complexity or communication criterion, the author scored the statement on the
one to five scale. Some statements applied to multiple criteria. This section explains how the
author applied this analytical scoring method to validate that the two assessment methods are
consistent with intuitive estimation, calibrated, and capable of comprehensively acquiring
data. The section closes with an explanation of the method for evaluating the relationship
between complexity and communication.
4.4.1
Comparing the scores from the assessment methods with intuitive estimations
Before analytically scoring each interviewee statement, the first author intuitively estimated a
score for each project with respect to each criterion on a one to five scale. This method is
subjective but provides a check for the detailed analysis method to ensure significant issues
are not missed or skewed by the detailed analysis.
29
Chapter 2
4.4.2
Reid Robert Senescu
Evaluating whether the assessment method is calibrated
A useful assessment method accurately reveals insightful differences between projects (Yin
2003). If all criteria score the same or are highly skewed to one end of the assessment scale,
the method cannot reveal insights into the differences between projects. Thus, the authors
validated the assessment methods by calculating the mean score and standard deviation of the
scores for each criterion. A calibrated assessment method will have an average of all the
criterion means close to three (halfway on the one to five scale) and an average standard
deviation of greater than one, which demonstrates that different statements had different
scores. Both calculations validate that the assessment method is calibrated.
4.4.3
Evaluating the applicability of the assessment methods
An assessment method for which it is too difficult to comprehensively acquire data is not
useful for assessing complexity and communication. Thus, the authors calculated the number
of statements collected for each criterion to demonstrate that the assessment methods can be
comprehensively applied to case study projects
4.4.4
Evaluating the complexity-communication relationship
The authors assessed trends between complexity and communication by plotting the mean
scores for each case study. In evaluating trends, the authors only considered case studies with
an average of at least one relevant statement for every two criteria for both complexity and
communication. The authors made this decision, because the high school case study lacked
sufficient complexity data for a meaningful mean score. Thus, the authors did not use the high
school project for evaluating a trend, but did use this case study’s data for validation of the
assessment method.
30
Chapter 2
Reid Robert Senescu
5
Complexity and Communication Assessment Methods
5.1
Product-Organization-Process Complexity Criteria
As discussed in more detail in Section 3.2, the criteria for assessing complexity come from the
aggregation performed by Homer-Dixon (2000). These criteria provide a useful method for
addressing the project information management needs of AEC (Froese and Han 2009).
Extending Froese’s application, this paper disaggregates complexity assessment further by
applying it individually to a project’s product, organization, and process. Table 2-2 provides a
definition for each criterion and an example from interviewee statements.
Table 2-2. Criteria for measuring a project's product, organization, and process complexity.
Very Simple =1.
Example Interview Statementi
not many components, low level of
detail. We model a diffuse sky, one
floor, one façade, varying only
height.
Causal Connections one component has no impact on
others. We focus collaboration on a
few crunch points, e.g., floor-to-floor
height.
Product
Complexity
Criteria Label
Multiplicity
Interdependencies
Openness
Synergy
Very Complex = 5.
Example Interview Statementi
many components, high level of
detail. We performed energy analysis
within a millimeter of detail.
one component impacts many others.
Choosing controls and sensors is not
trivial: on/off vs. dimming, sensor
locations, etc. [all have many impacts
on performance].
functions independently, resilient,
does not function independently, not
separable. We use packaged systems resilient, integrated. As so many
for HVAC and electrical.
things are highly sensitive to it, they
know they need to look at solar load
more carefully.
not sensitive to environments beyond sensitive to environment beyond
product scope or control. To design
product scope or control. Our design
the elevator, all I need is building
goal is to come in 2nd or 3rd in
height and how much space I have.
tendering.
building is simply the sum of its
Need to assess everything together.
parts. We have an assembly code
The bits between disciplines are
parameter and/or data set for each
where the real innovation occurs.
element in the model.
31
Chapter 2
Process
Organization
Nonlinear
Reid Robert Senescu
linear, easy to predict whether
stakeholder goals will be met, even
with precedence-based design. We
use back-of-hand calcs to
approximate ducts and shaft sizes.
emergent/non-linear, requires intense
performance-based design and even
then, still difficult to predict. It was
hard to predict how long it would
take to do roof construction, [because
there are so many factors and it's
unprecedented].
Multiplicity
single discipline, few people. For
multiple firms, hundreds of people.
Scheme Design, there were no
We are doing design of all services to
models, design was generic, no
Detailed Design [and then subparticipation of engineers.
contractors take over].
Causal Connections hierarchical chain of command, one highly networked, one person impacts
person affects only the person above. multiple other people. MEP, facades,
[He didn't know about the
ESD, acoustics, and building physics
construction sequence or why his
[all participated in design decisions].
supervisor prioritized certain areas
over others; he just responded to
requests].
Interdependencies people can work independently. The people need each other to work. In
Senior Engineer decides [by himself] Design Development, we ditched our
own model of slabs and instead, used
and considers [options] based on
the architect's model [because we
experience.
couldn't work independently].
project group is ill-defined, because
Openness
independent self-contained group.
The developer gave us a spreadsheet of heavy government and/or citizen
showing their value function for the
involvement. [Contractor] gives 80%
return on the apartments [there was of what the [government owner
representatives] are expecting and
just one stakeholder with a clear
we need to make sure it's ok with [the
delineation of the organizational
government's facility] operator.
extent of the project].
Synergy
team with distinct and separated
a team working together is more than
responsibilities. It's my job to figure the sum of its parts. There is a fine
out how to arrange and stack panels line between deciding what the
discipline should take responsibility
on site [he then went into details of
exactly his job versus other people's for versus what I should handle.
responsibilities].
Nonlinear
one person does not have a huge
everyone is on the critical path, no
impact on everyone else. The QA
one is easily replaceable. [Specific
sheet gives a sense of ownership to
other engineer] could probably find it
the crew so they take the quality of
quickly, but she is out.
the panels seriously [a large check
and balance system existed, so a
failure by one person would not be
catastrophic].
Multiplicity
few tasks. We decided ahead of time many tasks. Designed MEP, facades,
what options to consider [so, the
ESD, acoustics, and building physics
process was straightforward]
[all participated in design decisions].
Causal Connections non-iterative. We [architects] work
iterative (necessarily, positive). Cost
the services into the building and
planning is iterative with the
then hold [the engineers] to it. We
architect.
don't let [the engineers] change their
mind.
32
Chapter 2
Interdependencies
Reid Robert Senescu
tasks need to be performed all at once
and cannot be broken up. Models
were needed for constructability,
because we're building from top
down (not typical), so we need to
work out geometry and construction
together.
Openness
easy to scope the tasks and predict the unknown factors make it difficult to
tasks in advanced. [It seemed to them know in advance what tasks will be
unconscionable that they would miss required. 75% of the rooms randomly
vanished in Revit, requiring 5 days
these deadlines; that nothing would
prevent them from meeting the
work to replace.
deadlines, and I saw the date on the
drawing sets had the same day as the
scheduled delivery].
Synergy
two tasks sum as two (e.g., interface two tasks sum as greater than the
between the two tasks are seamless). whole (e.g., due to transactions
between the two). Then, the fire
We knew thermal performance was
ok for 100% glazing so then, we just consultants said they needed huge
fans. Working out with the architect
find glazing that works for
daylighting [as opposed to a process how to accommodate [geometrically]
that considered interactions between the new fans took 7-10 man days and
put us over budget.
thermal and daylighting].
unforeseen large repercussions, a
Nonlinear
each task is more or less isolated,
small error in a small task can have
errors are not cumulative. The
huge repercussions on future tasks.
process is that everyone fills out a
field for each object they create in the It's intense to make changes. You
Revit model.
move wall and have to check all
drawings.
i
brackets [ ] represent explanation or description by interviewer. Otherwise, statements are paraphrased
words of the interviewee, unless quoted.
5.2
tasks can be performed independently
and broken up into smaller and
smaller tasks. We established zones in
underfloor space for each discipline
"so we could work independently and
speed up iterations."
Communication Effectiveness and Efficiency Criteria
The authors developed the communication criteria by categorizing differences between
interviews. Table 2-3 provides a label and definition for each criterion and examples from
interview statements. The contrasting example statements in the table demonstrate how each
criterion was developed to reflect differences in the case studies using the cross-case searching
tactic (Eisenhardt 1989). Contrasting statements led to the development of criteria that differed
in how directly or indirectly the criteria assessed communication and in how the criteria
assessed communication effectiveness or efficiency. For example, the criterion labeled speed
directly assesses relative efficiency of collaboration. On the other hand, the criterion labeled
info pull assesses how often the organization asks for information they need. The info pull
33
Chapter 2
Reid Robert Senescu
criterion is indirectly indicative of collaboration effectiveness, because not asking for
information (i.e., remaining ignorant of others information) takes zero time, but is indicative
of ineffective exchange of information (unless the other people know to give you the
information they require). Another example of an indirect measurement is the options
criterion. For example, the interior designer on one project team presented just one option for
the interior design that would then be approved or rejected by the owner representative. In
contrast, the cost estimator on another project provided feedback to the other disciplines on
thirty design options. This criterion indicates indirectly the extent to which the teams
collaborated to consider multi-disciplinary tradeoffs between different design options versus a
case of collaboration where a single option progressed until it was determined to be
insufficient by a particular discipline. Aggregated, the criteria provide a direct and indirect
assessment of the effectiveness and efficiency of how the professionals collaborated within the
team, shared information between teams, and understood information across the entire firm or
industry.
Table 2-3. Criteria for assessing a project’s communication.
Communication
Criteria Label
Speed
Collaboration
Clarity
Consistency
Great Communication = 1.
Example Interview Statementi
fast information exchanges. [With
these three semi-automated steps]
overall scheme design takes 3-5 man
days.
explain product, organization, or
process clearly. Sketchup allows
clients to connect with what we
design.
consistent information between team
members. There was consistency
between discipline assumptions.
Poor Communication = 5.
Example Interview Statementi
time consuming information
exchange. Takes about 100 days to get
coordinates of every piece of steel out
of the model.
unclear product, organization or
process explanation. The
documentation is "appalling." The
industry relies more and more on shop
drawings. The designer just creates
schematics and relies on detailing and
coordination after design.
inconsistent information between team
members. Sometimes the .dwg is
revised or people e-mail files [instead
of saving the file correctly] and that
screws things up.
34
Chapter 2
Integrated
Decisions
Reid Robert Senescu
considers multiple disciplines in
decision making. Multi-disciplinary
workshops really help to integrate
ideas.
Common Process the entire team knows the processes
Knowledge
within each discipline. The architect is
using ArchiCAD to IFC, but we
haven't really used it [i.e., Engineer
knows the architect's processes even if
disconnected from their own process].
Standardized
well defined standard for digital
information exchange. We created a
BIM process manual.
Planned
Incentivized
Options
Connected
Info Pull
Sharing
Transparency
decision making silos. The architect
was not concerned about anything but
aesthetics.
no knowledge of people's processes
outside discipline. We purposefully
delete [permanently] annotations
[showing design rationale] after
submitting the drawings.
no standard, all different formats. The
modelers are not consistent, so it's
difficult to extract information for
cost.
planned coordination. At the beginning reactive coordination. Fire consultants
of the project, we ask what are the key weren't on board early even though we
things that should be coordinated.
knew the building would need
performance-based smoke ventilation,
[this caused problems].
incentivized to optimize within
incentivize to consider multidiscipline silos. Structural engineer
disciplinary tradeoffs and strive
toward a global optimum. Acoustician had no incentive to make all the
panels similar, so constructability was
says [engineer] wants to expose
ceiling for thermal mass, even though difficult.
acoustics is not good [but acoustician
recognized other goal was more
important]
considers many options. We costed
considers only one option. They
about 30 different options for [the
produce 3D renderings of the selected
building].
option …for verification…[but the
owner representative wasn't happy
with results].
team is well-connected. We hold
few connections between the team.
workshops regularly to integrate
[The manager complained], the
ideas.
consultants give us Navisworks files,
but they are "useless to us." [In the
same office, an engineer said she is
using Navisworks on the same
project.]
proactively explain what information rarely requests information from
is needed from other people. The
others. We changed shop detailers and
mechanical engineer knows what he
they didn't give us the ProSteel model,
needs and lets others know via
only the Navisworks file…in the future
conversations or e-mail.
we need to get the detailers to give us
the information we needed [in the
format we need].
all information created is organized
all information created is temporary,
transparently in a single location and the process is never documented. We
the information is linked to the process highlighted the potential problem
and organizations that created it. We
verbally….and then, later, we got
have a directory of [interoperability] blamed for it (we didn't have a record
scripts that are reusable.
of us warning about it).
35
Chapter 2
Awareness
Dissemination
Externally
Connected
Standardized
Processes
Understanding
Measurable
Success
Measured Time
Process
Documentation
Reid Robert Senescu
exchange processes with global
community. [The engineer] presented
at [the research institute] about
Grasshopper, so the company shares
processes via community talks.
little awareness of processes outside
the project, company, and city. I am
predicting actual performance within
3%. "No way they could use more
energy than predicted." [either the
engineer is unaware of the huge
differences between prediction and
actual performance common in
industry, or the industry is unaware of
the extremely advanced technique
employed by this engineer].
disseminate processes so other teams don't disseminate processes. The truth
can adopt them. Wiki helps explain
is we have no idea how we transfer
how to use the server.
knowledge to the organization as it
went from 4 to 18 architects
well connected with communities
little personal contact with
outside project. We are working with a communities outside project. "BIM is
a software product" so everything
[professor in Brisbane] and with the
needs to be compatible between all
IAI in Scandinavia
disciplines [contrasting with general
view of BIM as a method and general
acceptance that BIM is useful without
complete compatibility, suggesting he
is not well connected with the global
BIM community.]
when possible, processes are
processes that could be standardized
standardized. Now, we're running our are not. Setting full procedures [for
own scripts on servers, solidifying our standard parametric modeling
basic processes.
techniques] is very expensive….but it's
scary because students [or newly
graduated] create geometric messes
that would kill a project.
easy to measure process success.
difficult to measure process success.
Profit measures success.
[The person managing investment in
new design process technology, when
asked about how they will know if
their latest investment is successful]
"Yes, it would be a good idea to have
a measure of success" [they didn't
have any].
measured time spent on tasks. Before did not measure time spent on tasks.
coordinating penetrations took 2 days We can't quantify the cost of running
[in Navisworks], now we have a
BIM, nor the benefit of BIM…nor the
customized process that takes half a
cost of "appalling" documentation
day.
documented some aspect of the
no documentation and no knowledge
process. The government is investing of the process. We start with an image
in reducing energy, so our mechanical of how the building should be, and
spec will be used in many schools, so then the process should just make that
that is why we could spend the time to a reality. All the creation happens in
the mind [the process is just producing
document our mechanical design
process.
what's in the mind at the beginning].
36
Chapter 2
Categorization
Incentivized To
Innovate
Reid Robert Senescu
categorized projects for comparison of
process. We looked at the ten best and
ten worst projects [in terms of profit
and then evaluated which ones used
BIM]
incentivized to innovate. We have a
new revenue stream where our
modelers go into sub's offices and
model for them.
no categorization, so impossible to
compare. [Some companies simply
had no formal way of categorizing
projects]
no incentives to innovate. It's our
traditional policy to keep paper, so we
just do that. Besides, it's [her] job
[implying she would not have anything
to do without paper].
informal, non-computable method for
Process Language formal computable language for
describing the process. I set up facade describing process. no language or
configurations. Then, Radcalc is a
grammar. process description is likely
perl code that reads xml. The code
interpreted differently by different
parses the xml and feeds it into
people. [When asked about a
Radiance, and it analyzes all the
particular energy analysis process, the
options in the servers in Melbourne.
engineer tried to find the answer in the
project folder, but frustrated, said]
This folder is "such a disaster.
Someone deleted the presentation
folder….This is such a mess…I can't
remember what we did."
Vertical
vertical understanding of process
process is not understood at different
levels of the organizational hierarchy.
Understanding
throughout the organizational
hierarchy and buy-in to the importance PM viewed BIM as only a software
product and didn't want to upgrade
of process improvement. 64 bit
until it was compatible with all
computers is not a big deal. We just
upgrade.
disciplines.
Investing
methodical about investment
investing in new technology is
decisions. We analyze when we need
random. We make decisions about
to upgrade and investigate options for investing in software or script
computers in detail.
development to go between them
“somewhat randomly.”
i
brackets [ ] represent explanation or description by interviewer. Otherwise, statements are paraphrased
words of the interviewee, unless quoted.
6
Results and Discussion
This section presents the results from applying the assessment methods on seven case studies.
The authors calculated the results by applying the research method described in Section 4. The
section uses two example case studies to illustrate how using the two assessment methods
together provides insight into the mechanisms responsible for causing complexity to impact
communication.
37
Chapter 2
6.1
Reid Robert Senescu
Validating the Two Assessment Methods
This section provides evidence that the two assessment methods provide results that align with
intuitive estimates; that the two methods are calibrated to the case studies; and that the two
methods can be comprehensively applied to case studies.
6.1.1
Alignment between intuitive estimations and the scores from assessments
The first validation of the two assessment methods came from comparing the intuitive
estimates with the assessment-based scores. The assessment methods should produce more
precise results than the intuitive estimations, so large deviations between what is expected
intuitively would reveal potential problems with the assessment methods or the intuitive
estimations. Plotting each project’s complexity on the x-axis and communication on the y-axis
(Figure 2-2), the difference between the analytical assessment (large data points) and the
intuitive estimate (small data points) for each project is the distance between the two points on
the graph. The average difference was only 0.72, demonstrating reasonable alignment between
the intuitive estimations and the scores from the assessments.
38
Chapter 2
Reid Robert Senescu
Figure 2-2. Assessment of the intuitiveness of results. For each project, the plot shows a
comparison of complexity and communication assessment methods. In general the analytical
assessments (large data points) are relatively close to the intuitive estimations (small data
points). The largest differences occur for the high-rise commercial and residential, high-rise
commercial, and stadium projects (estimates are 1.0 to 1.1 distance away from analytical
assessment). On the other hand, the university building and low-rise residential were
close (0.3).
6.1.2
Calibrated assessment methods
Across all seven projects, the authors applied the 42 criteria (18 complexity criteria in Table
2-2 and 24 communication criteria in Table 2-3) 398 times to interview statements. A valid
assessment method should not have highly skewed scores across these criteria or the same
scores for all the criteria. Balanced and varied scores enable insights to be drawn from the
39
Chapter 2
Reid Robert Senescu
assessment method – an important feature of a useful assessment method (Yin 2003). Figure
2-3 (a) shows the number of interview statements that the authors considered relevant for
assessing the project with respect to each criterion. For each criterion, the authors measured
the mean and standard deviation of scores across the seven projects (Figure 2-3 (b)). The mean
score of all the criteria was 2.9 with a standard deviation of 0.6. This calculation revealed that
each criterion was well balanced with about as many statements scoring higher and lower than
three. The average standard deviation within each criterion was 1.2, revealing that in general
each criterion was sufficiently granular that variations existed between projects. However, two
exceptions exist. There are only two statements relevant to Nonlinear Organization and four
statements relevant to Categorization. For both criteria, the examples come from a single
project, so the standard deviation is zero. This exception probably arose from insufficiently
thorough interviews that failed to explore these criteria, rather than a problem with the criteria
themselves. Since in general Figure 2-3 reveals that the assessment method is balanced around
three and varied away from three, the assessment method is well calibrated and potentially
useful in discovering trends.
40
Chapter 2
Reid Robert Senescu
Figure 2-3. Assessment of the distribution of complexity and communication criteria. The
number of statements relevant to each criterion varied from 2 to 24. The average score across
all criteria is 2.9, which is expectedly close to three since the one to five scale was defined
based on the interviews. The average of the standard deviations across all projects is 1.2. This
deviation across projects suggests that the assessment criteria are sufficiently granular to
differentiate trends between projects.
41
Chapter 2
6.1.3
Reid Robert Senescu
Applicability of the assessment methods
An assessment method for which it is too difficult to comprehensively acquire data is not
useful for assessing complexity and communication. Thus, the authors calculated to what
extent the case study interviews covered the project complexity and communication criteria
comprehensively. A higher number of statements per criterion signified a more comprehensive
interview. Averaged across all projects, there were 0.9 statements per project complexity
criterion and 1.6 statements per project communication criterion. That is, on average each
project had just less than one statement for each project complexity criterion. As there were 18
complexity criteria, each project had on average 16 statements related to complexity.
For the High School project, there were only 0.3 statements per project complexity
criterion (Figure 2-4). As this value was below 0.5, interviews on this project did not offer a
comprehensive view of the project complexity, increasing the potential inaccuracy of the
project’s complexity assessment. Further supporting this potential inaccuracy, the intuitive
estimate for complexity was 0.8 points higher than the analytical assessment (the third largest
difference). Also, this was the only project where only one person was interviewed.
Consequently, the authors disregarded this project’s complexity and communication
assessment, and it is not used for evaluating trends. However, because of the comprehensive
set of statement examples for the other six case studies, the authors conclude that in general
the two assessment methods can be applied comprehensively.
42
Chapter 2
Reid Robert Senescu
Figure 2-4. Assessment of the comprehensiveness of interviews. The figure shows that there
were in general more interview examples related to project communication criteria than
complexity criteria. In fact, the High School project had so few examples of complexity
criteria that it is not considered in assessing complexity-communication trends.
6.2
Discussion of Two Case Studies
As the previous section demonstrated that the two assessment methods produce results that are
aligned with intuitive estimates, calibrated, and able to be comprehensively applied, this
section discusses the results from applying the assessment methods to two representative case
studies (the stadium and low-rise residential). The discussion explains the reasoning behind
the criteria scores assigned to each example (criteria are indicated in parenthesis) and the
mechanisms by which complexity impacted communication. The authors chose these two case
studies because they are at the two extremes of all the case studies examined: the stadium is
complex with many communication challenges and the low-rise residential is simple with
good communication.
As the criteria scores for the stadium and low-rise residential case studies are relative
to the other cases discussed in the paper, the section describes these examples in the context of
43
Chapter 2
Reid Robert Senescu
two graphs. Figure 2-5 shows the mean criteria scores for complexity and communication for
each case study. Figure 2-6 shows a more nuanced relationship between complexity and
communication by disaggregating project complexity by product, organization, and process;
and communication by collaboration, sharing, and understanding. Section 6.3 discusses in
more detail the overall trends revealed by these two graphs.
6.2.1
Stadium project in depth
The stadium project was a complex product because of its size and geometrical nonuniformity (product multiplicity). Interviewees stated repeatedly that every panel in the
stadium was unique. Organizationally, the project was complex in some respects and simple in
others (see top middle plot in Figure 2-6). The organization was complex in that the project
involved government representatives, many sub-contractors and fabricators, and many
different designers (organizational multiplicity). Also, it was vulnerable to a wide variety of
public organizations outside the immediate project team (organizational openness). However,
the organization was simple in that team members had delineated roles (organizational
synergy) with clear lines of hierarchy (organizational causal connections). The process was
complex, because design was influenced by constructability, making it difficult to separate
contractor and designer tasks (process interdependence). Because decision makers were
unclear, frequent iteration occurred (process causal connections). This complexity
corresponded with many communication problems. For example, the aspect of the
organization that was complex inhibited team members from collaborating across discipline
silos to make decisions. In one case, the contractor submitted an interior design option to an
owner representative for approval. Neither the owner representative nor other stakeholders
participated in the decision making, and the contractor lacked incentive to consider multidisciplinary tradeoffs (collaboration – incentivized). At the same time, the team considered
just one design option (collaboration – options) in one month (collaboration – speed). The
44
Chapter 2
Reid Robert Senescu
project complexity also stifled efforts to maintain consistency (collaboration – consistency) in
information management systems (the first author observed a request to find an AutoCAD file
that took 30 minutes to fulfill).
Sharing process information across projects was difficult, because the process was not
documented. A Design Manager for the Contractor stated, ‘he would like to link memos to
build a story of what happened’ (sharing – transparent). In one case, the Drafter explained how
he needed to come up with a system for managing information from scratch. This example
demonstrated that the firm did not disseminate processes across projects (sharing –
dissemination), perhaps because of organizational and process complexity. On the other hand,
for information more easily standardized, such as material costs and productivity rates, the
firm had a database shared across projects (sharing – standardized processes).
The ability of the contractor to understand the project to invest in improvement lagged
only slightly behind the other projects. During a discussion about the cumbersome process of
finding paper documents, the interviewee explained paper was both tradition and policy and
many jobs were solely dedicated to managing the paper (understanding – incentivized to
innovate). It is more difficult for a large complex organization to quickly migrate to digital
systems. Also, according to the project manager, ‘we can’t quantify the cost of running BIM
nor the benefit of BIM so it’s difficult to make a value proposition for introducing new
technologies’ (understanding – measured time). Measuring complex and unique processes is
more difficult than measuring simple frequently repeated processes. On the other hand, a
quality assurance professional clearly strived toward ensuring no roof leakage and to provide
traceability if leakage occurred (understanding – measurable success). This definition of
success and documentation of its achievement allowed the firm to compare different project
processes and outcomes. Of course, while this professional had a well-scoped and measurable
goal, the complexity of the project made many other goals more difficult to measure.
45
Chapter 2
6.2.2
Reid Robert Senescu
Low-Rise residential project in depth
The low-rise residential project was relatively simple and communication excellent. The lowrise residential project contained many pre-packaged HVAC and electrical systems common
across all the apartments, but also functioning independently (product interdependence). Also,
the client was a real estate developer, contrasting with the many stakeholders on the stadium
project. The client measured project success based on the number of apartments built on the
site. This product goal was relatively unaffected by the outside environment (product
openness) and clear to the entire team making collaboration easier (collaboration – clarity).
Also, the site was highly constrained. Few feasible layout options existed and so, iteration was
unnecessary (process causal connections). Also, the organization was hierarchical. For
example, the geotechnical consultant gave a report to the architect and then, the architect gave
the same report to the structural engineer (organization causal connections). The report did not
have implications for the entire project, nor did the structural engineer have to work closely
with the geotechnical engineer to develop a sophisticated foundation strategy. The process was
also relatively simple since it seemed unimaginable to the team that they would miss drawing
submission deadlines; all previous deadlines were met. This confidence and consistency
suggests that it was easy to scope and predict tasks (process openness).
The team used Google Sketchup to communicate and reported that the tool effectively
allowed the owner to connect with their designs (collaboration – clarity). It is easier for
architects to use tools such as Sketchup to communicate designs of simpler buildings than
complex buildings (Figure 2-6 top left). They reported taking a week to create apartment
layouts, because they worked closely with the structural engineers and developed an integrated
design together (collaboration – integrated decisions).
The interview focused less on sharing, but some examples existed where the firm
created standardized processes for going between for example, parametric modeling tools and
46
Chapter 2
Reid Robert Senescu
Microsoft Excel (sharing – standardized processes). Also, a senior associate in the office
provided ample evidence that he was aware of global technology trends in the industry
(sharing – awareness).
Relative to other projects, the architecture firm understood their current processes and
invested in improvement. For example, the firm categorized BIM projects and non-BIM
projects (understanding – categorization). They considered tangible BIM investment costs
such as hardware, software, human resource costs (understanding – measured time), and even
when including these costs, measured increased profit on BIM projects. The firm also invested
in BIM technology at the corporate level (understanding – vertical understanding), though the
investments themselves were not especially methodical (understanding – investing).
47
Chapter 2
Reid Robert Senescu
Figure 2-5. The relationship between project complexity and communication. Based on the
six AEC projects assessed, communication problems increase with increased product,
organization, and process complexity. This current trend is unsustainable, because delivering
projects of increased environmental, financial, and social sustainability requires increased
complexity. AEC requires IT solutions enabling horizontal trends so the industry can increase
complexity without increased communication problems.
6.3
Discussion of the Complexity-Communication Relationship
The previous section uses two case study examples to demonstrate how the authors applied the
assessment methods to the case studies. The discussion of the case studies also illustrates the
difference between a complex project with poor communication and a simple project with
48
Chapter 2
Reid Robert Senescu
good communication. Assessing the six case studies reveal a general trend between increased
complexity and increased communication problems (Figure 2-5) – the second contribution of
this paper. Because the complexity and communication assessment methods consist of three
types of information and three types of communication, respectively, the assessment methods
enable comprehension of how each complexity type impacts each communication type. This
disaggregation of the assessment methods reveals a more nuanced set of smaller trends (Figure
2-6). For example, a trend exists between product and process complexity and collaboration
problems (top left and top right of Figure 2-6). However, the middle column of Figure 2-6
suggests little trend between the impact of organization complexity on communication. Also,
the middle row reveals little trend between POP complexity and sharing, suggesting that the
complexity of one project is not indicative of how companies share information across
projects. One explanation for this observation is that the architecture firm that designed the
simple low-rise residential is a large global firm that also works on many complex projects.
Discovering a relationship between project complexity and sharing may require interviewing
multiple project teams within the firm.
Unlike sharing between projects, a trend exists between how product and process
complexity impacts a firm’s ability to understand information across projects, but this trend is
less steep than the collaboration trend. The stadium project supports this observation because
the project was so complex that it was both difficult and there was little incentive to
understand the project. But for the low-rise residential building, the project’s simplicity and
the ability of the firm to understand information on the project may have been circumstantial.
Again, more thorough investigation of complex projects by the same firm would be needed to
confirm that understanding increases with project complexity.
49
Chapter 2
Reid Robert Senescu
Figure 2-6. Disaggregated POP complexity and communication trends. The strongest trends
exist between product and process complexity and collaborating. Little evidence exists that
sharing across projects is impacted by complexity. A less sensitive trend exists between
product and process complexity and a firm’s ability to understand information across the firm
or industry.
50
Chapter 2
6.4
Reid Robert Senescu
Limitations and Future Work
Unlike previous studies outlined by (Maier et al. 2008), the results presented above avoid
measuring communication purely on quantity of information exchanged. Also, the authors do
not attempt to assess complexity by counting product, organization, and process objects.
Instead, the authors assess complexity qualitatively, and similar to Maier et al., assess
communication effectiveness and efficiency. The case study interview approach leads to
qualitative analysis potentially more subject to authors’ bias than methods that count
information on projects or survey project participants. Instead, the paper applies inductive
logic, not seeking to be hypothetico-deductive. That is, the interview approach limits the
authors’ ability to claim a particular linear (or exponential) correlation between complexity
and communication or perform causality analysis, because there are just six case studies with
scores based on qualitative information.
As opposed to relying on the survey method employed by Maier (2008) to assess
communication or an information counting approach to assess complexity, the interview
method revealed communication problems unknown to the project teams and was able to
assess complexity relative to other projects. For example, in one interview, the manager
complained, the consultants give us Navisworks files, but they are "useless to us." In the same
office, an engineer said she is using Navisworks on the same project. By interviewing the
team, as opposed to surveying, the authors could assess that the team struggled to collaborate
effectively, because they were not connected even though they were collocated. Thus, the
Connection criterion permitted assessment of their struggle to communicate their processes
and organizational skills. With respect to complexity, the architects on the low-rise residential
building discussed the process of laying out apartments as complicated because of the
geometric challenges in maximizing the number of apartments. But the case study interview
method enabled the authors to compare this relatively simple process (perceived by the
51
Chapter 2
Reid Robert Senescu
interviewees as complex) with the university building that had to layout many different types
of rooms while simultaneously taking into account other goals such as minimizing energy.
Thus, the interview approach both revealed communication problems unknown to the
interviewee and avoided assessments biased by interviewees’ narrow context of complexity.
The advantages of the qualitative approach lead the authors to suggest that future work
should apply the two assessment methods to more case studies to develop more robust
conclusions about complexity-communication trends. Additionally, recording interviews and
then blindly coding statements with respect to criteria would provide stronger evidence for the
trends observed in this paper. In particular, more case studies are needed to assess the
existence of trends between the disaggregated project complexity and communication data
points in Figure 2-6. The case study interview approach could be combined with a quantitative
approach that could claim with statistical significance the correlation between communication
and complexity. Determining the steepness of the complexity-communication curve through
quantitative methods would also allow researchers to correlate complexity and communication
with other methods of assessing projects.
In particular, more research is needed to assess whether the implementation of product
communication tools such as BIM software applications or organization communication tools
such as intranet-based people pages are more likely to enable excellent communication despite
complexity. While this paper did not assess the maturity of such technology on the different
case study projects, interviews provided insight on the state of IT. Many interviewees
described how BIM enabled them to better communicate the building product. While
communicating information about the organization may be challenging, interviewees
infrequently cited this challenge as a major problem. On the other hand, many interviewees
cited how their daily work was negatively impacted by poor process communication
52
Chapter 2
Reid Robert Senescu
technology. These observations are speculative and future work is needed to assess the
additional impact of technology adoption on complexity and communication.
7
Conclusion
This paper describes two contributions to project information management literature. First, the
paper contributes a project complexity assessment method and a communication assessment
method. The paper develops the first method by extending others’ definition of complexity to
apply to products, organizations, and processes. The authors develop the communication
assessment method by revealing differences from interview statements. The breadth of project
types and the case study scores demonstrating calibration and comprehensive applicability to
the case studies provide evidence that the methods are generalizable to different types of
products (i.e., from low-rise residential to stadium) and organizations (i.e., from architecture
firm to general contractor). The paper also demonstrates through the low-rise and stadium
project examples how the methods enable assessment of the mechanisms by which product,
organization and process complexity impacts collaboration, sharing, and understanding.
Second, the paper contributes evidence of a trend between a project’s product, organization,
and process (POP) complexity and communication.
This trend between increased project complexity and poor process communication is
unsustainable as the AEC industry must resort to more complex solutions to provide more
financial, social, and environmental value to stakeholders. By identifying this trend, this paper
aims to motivate future research that discovers types of interventions (e.g., innovative
communication tools, project contract structures, professional education programs, etc.) that
decouple the relationship between increased complexity and increased communication
problems. The complexity and communication assessment methods provide a foundation for
53
Chapter 2
Reid Robert Senescu
developing and measuring the impact of these interventions on the sensitivity of
communication effectiveness and efficiency to POP complexity.
8
References
Allen, P. M. (2008). "The importance of complexity for the research agenda in the built
environment." Architectural Engineering and Design Management, 4(1), 5-14.
Allen, P. M., Sanglier, M., Engelen, G., and Boon, F. (1985). "Towards a new synthesis in the
modeling of evolving complex systems." Environment and Planning B: Planning and
Design, 12(1), 65-84.
Baccarini, D. (1996). "The concept of project complexity--a review." International Journal of
Project Management, 14(4), 201-204.
Carrillo, P., and Chinowsky, P. (2006). "Exploiting knowledge management: The engineering
and construction perspective." Journal of Management Engineering, 22(1), 2-10.
Conklin, J. (1996). Designing organizational memory: Preserving intellectual assets in a
knowledge economy, CogNexus Institute, Glebe Creek, MD.
Crotty, M. (1998). The foundations of social research: Meaning and perspective in the
research process, Sage Publications, London, UK.
Dossick, C. S., and Neff, G. (2010). "Organizational divisions in BIM-enabled commercial
construction." Journal of Construction Engineering And Management, 136(4),
459-467.
54
Chapter 2
Reid Robert Senescu
Eckert, C. M., and Clarkson, P. J. (2004). "If only i knew what you were going to do:
Communication and planning in large organizations." Methods and tools for cooperative and integrated design, S. Tichkiewitch and D. Brissaud, eds., Kluwer
Academic Publishers, Dordrecht, Netherlands, 375-384.
Eckert, C. M., Clarkson, P. J., and Stacey, M. K. (2001). "Information flow in engineering
companies - problems and their causes." International Conference on Engineering
Design, Professional Engineering Publishing, Suffolk, UK, 43-50.
Eisenhardt, K. M. (1989). "Building theories from case study research." Academy of
Management Review, 14(4), 532-550.
Erdogan, B., Anumba, C. J., Bouchlaghem, D., and Nielsen, Y. (2008). "Collaboration
environments for construction: Implementation case studies." Journal of Management
in Engineering, 24(4), 234-244.
Foss, N. J., Husted, K., and Michailova, S. (2010). "Governing knowledge sharing in
organizations: Levels of analysis, governance mechanisms, and research directions."
Journal of Management Studies, 47(3), 455-482.
Froese, T., and Han, Z. (2009). "Project information management and complexity in the
construction industry." Proceedings of the 26th International Conference on IT in
Construction, Istanbul Technical University, Istanbul, Turkey, 1-10.
Fruchter, R., and Demian, P. (2002). "CoMem: Designing an interaction experience for reuse
of rich contextual knowledge from a corporate memory." Artificial Intelligence for
Engineering Design, Analysis and Manufacturing, 16, 127-147
Galbraith, J.R. (1974). “Organization design: An information processing view.” Interfaces, 4
(3), 28-36.
55
Chapter 2
Reid Robert Senescu
Gane, V., and Haymaker, J. (2010). "Benchmarking current conceptual high-rise design
processes." Journal of Architectural Engineering, 16(3), 100-111.
Garcia, A. C. B., Kunz, J., Ekstrom, M., and Kiviniemi, A. (2004). "Building a project
ontology with extreme collaboration and virtual design and construction." Advanced
Engineering Informatics, 18(2), 71-83.
Hansen, M. T., Nohria, N., and Tierney, T. (1999). "What's your strategy for managing
knowledge?" Harvard Business Review, 77(2), 106-116.
Hargadon, A., and Sutton, R. I. (2000). "Building an innovation factory." Harvard Business
Review, 78(3), 157-166.
Haymaker, J., Riker, and Chachere, J., Marvin. (2011). "Framework for measuring the
rationale clarity of AEC design decisions." Journal of Architectural Engineering, in
press.
Haymaker, J. R., Chachere, J. M., and Senescu, R. R. (2011). "Measuring and improving
rationale clarity in a university office building design process." Journal of
Architectural Engineering, in press.
Homer-Dixon, T., F. (2000). The ingenuity gap: Facing the economic, environmental, and
other challenges of an increasingly complex and unpredictable world, Knopf, New
York, NY.
Javernick-Will, A., Levitt, R., and Scott, W. R. (2008). "Mobilizing knowledge for
international projects." Proceedings of the 2008 ASCE LEED Conference, CIB Task
Group 64 and ASCE Construction Research Council, Lake Tahoe, CA.
56
Chapter 2
Reid Robert Senescu
Javernick-Will, A., and Levitt, R. E. (2010). "Mobilizing institutional knowledge for
international projects." Journal of Construction Engineering and Management,
136(4), 430-441.
Kivrak, S., Arslan, G., Dikmen, I., and Birgonul, M. T. (2008). "Capturing knowledge in
construction projects: Knowledge platform for contractors." Journal of Management
in Engineering, 24(2), 87-95.
Kreiner, K. (2002). "Tacit knowledge management: The role of artifacts." Journal of
Knowledge Management, 6(2), 112-123.
Kvan, T. (2000). "Collaborative design: What is it?" Automation in Construction, 9(4), 409415.
Laufer, A., Shapira, A., and Telem, D. (2008). "Communicating in dynamic conditions: How
do on-site construction project managers do it?" Journal of Management in
Engineering, 24(2), 75-86.
Luiten, G. T., and Tolman, F. P. (1997). "Automating communication in civil engineering."
Journal of Construction Engineering and Management, 123(2), 113-120.
Maier, A. M., Kreimeyer, M., Hepperle, C., Eckert, C. M., Lindemann, U., and Clarkson, P. J.
(2008). "Exploration of correlations between factors influencing communication in
complex product development." Concurrent Engineering, 16(1), 37-59.
Otter, A. d., and Emmitt, S. (2008). "Design team communication and design task complexity:
The preference for dialogues." Architectural Engineering and Design Management,
4(2), 121-129.
57
Chapter 2
Reid Robert Senescu
Otter, A. F. d., and Prins, M. (2002). "Architectural design management within the digital
design team." Engineering, Construction and Architectural Management, 9(3), 162173.
Rechtin, E. (1991). Systems architecting: Creating and building complex systems, Prentice
Hall, Englewood Cliffs, NJ.
Senescu, R., and Haymaker, J. (2008). "Requirements for a process integration platform." 7th
International Workshop on Social Intelligence Design: Designing Socially Aware
Interactions, Universidad de Puerto Rico, San Juan, Puerto Rico.
Senescu, R. R., and Haymaker, J. R. (2009). "Specifications for a social and technical
environment for improving design process communication." Proceedings of the 26th
International Conference on IT in Construction, Istanbul Technical University,
Istanbul, Turkey, 227-237.
Senescu, R. R., Haymaker, J. R., and Anderson, D. J. (2010). "PIP: A process communication
web tool." Modelling and Management of Engineering Processes, Cambridge
University, Cambridge, UK.
Taylor, J. E., and Bernstein, P. G. (2009). "Paradigm trajectories of building information
modeling practice in project networks." Journal of Management in Engineering,
25(2), 69-76.
Weber, M. (1947). The theory of social and economic organization, The Free Press, Glencoe,
IL.
Yin, R. K. (2003). Case study research: Design and methods, Sage Publications, Thousand
Oaks, CA.
58
Chapter 3
Reid Robert Senescu
Chapter 3: Design Process Communication Methodology improving the efficiency and effectiveness of
collaboration, sharing, and understanding
Reid Senescu, John Haymaker, and Martin Fischer
1
Abstract
Architecture, Engineering, and Construction (AEC) designers struggle (1) to collaborate
within projects, (2) share processes across projects, and (3) understand processes across the
firm or industry. Overcoming each of these challenges requires communication of design
processes. The paper aggregates concepts from organizational science, human computer
interaction, and process modeling fields to develop the Design Process Communication
Methodology (DPCM). DPCM is a social, technical, and representational environment for
communicating design processes that is Computable, Embedded, Modular, Personalized,
Scalable, Shared, Social, and Transparent. To apply and test DPCM, the research maps the
methodology to software features in the Process Integration Platform (PIP). PIP is a process
communication web tool where individuals exchange and organize files as nodes in
information dependency maps in addition to folder directories. The paper provides evidence of
the testability of DPCM and proposes metrics for evaluating DPCM’s efficiency and
effectiveness in communicating design process. DPCM lays the foundation for commercial
software that shifts focus away from incremental and fragmented process improvement toward
a platform that nurtures emergence of (1) improved multi-disciplinary collaboration, (2)
process knowledge sharing, and (3) innovation-enabling understanding of existing processes.
59
Chapter 3
2
Reid Robert Senescu
Introduction
Design processes disproportionately influence the life cycle value of the resulting products
(Paulson 1976). While the total cost of design is relatively small, the design phase of a project
greatly influences total project value. Also, final project value generally increases with the
number of different design options considered (Akin 2001; Ïpek et al. 2006). Yet, despite
information technology advances during the last half of the 20th Century, the number of design
options explored for any one design decision is typically less than five and almost always a
small percentage of the possible design space available (Flager et al. 2009; Gane and
Haymaker 2010). Not surprisingly, the construction value per man-hour expended actually
decreased from 1964 to 2003, while the rest of the American non-farm industry more than
doubled (United States Department of Commerce, 2003). Research leading to new information
management systems can improve design processes to increase project value per man-hour
expended. Improving design processes requires not just isolated technological improvements
but also process change within companies. This change requires process communication (Ford
and Ford 1995). The Design Process Communication Methodology (DPCM) contributes to the
project information management (PIM) and design process management (DPM) research
fields by laying the foundation for the development of commercial software that
communicates design processes to increase the value per man-hour expended by the
Architecture, Engineering, and Construction (AEC) industry.
Organizations within the AEC industry create information to represent the product
through a process (Garcia et al. 2004). The process can be viewed through three lenses:
conversion, flow, and value generation (Ballard and Koskela 1998). The authors choose the
information flow lens because of the relatively large potential for capturing information paths.
The design process is then organizations exchanging information that leads to a plan for the
building Product. Jin and Levitt’s Virtual Design Team (1996) similarly apply this
60
Chapter 3
Reid Robert Senescu
information processing view of the organization to the AEC industry – first described by
Weber (1947) and later adopted by March and Simon (1958) and Galbraith (1977). This
research scopes the focus further to only include digital information exchange from scheme
design to construction documentation on projects involving complex (Homer-Dixon 2000;
Senescu et al. 2011a) products, organizations, and/or processes.
This section uses this information processing lens to describe how the AEC industry
can improve design processes through improving three types of process communication:
1. The organization can collaborate more effectively and efficiently within the project
team. In this case, the organization does not significantly change the topology of
information exchanges on a project, but executes the exchanges better through
improved comprehension of the project team’s processes. For example, the
information may be more consistent throughout the project or a particular project team
member may make a discipline-specific decision with more insight about how that
decision impacts other disciplines.
2. One project team may share a process between project teams. For example, a team
may learn about a process employing more effective software that they then
implement on their project.
3. A team may consciously develop an improved process. Developing improved design
processes requires investment, which requires a claim that the return will be an
improvement on the current state. Organizations must understand their current
processes across the firm or industry to strategically invest in process improvement.
For example, a team may understand that across the firm, they repeatedly count
objects in their building information model and then manually enter quantities into a
cost estimating spreadsheet, and so they invest in developing a script that performs the
process automatically.
61
Chapter 3
Reid Robert Senescu
The AEC industry struggles to collaborate around processes, share processes, and understand
processes effectively and efficiently as project complexity increases (Senescu et al. 2011a).
Considering all three types of information exchange (i.e., communication) explicitly
and in unison is important, because otherwise, there are “cost-benefit mismatches” in
communication. That is, many previous communication improvement efforts do not consider
that “the person responsible for recording information is typically not the person who would
benefit from the information once it is recorded” (Eckert et al. 2001). Also, team members
frequently have conflicting obligations to the project and to the firm (Dossick and Neff 2010).
Holistically considering the three communication types, this paper asks: how can design
processes be communicated efficiently and effectively within project teams, between project
teams, and across a firm or industry?
This paper first describes three observed design process communication challenges that
motivated this research (Section 3). Then, the authors look to the PIM and DPM research
fields to find a solution to the observed challenges. Not finding a solution, the authors describe
in Section 3 how these two research fields lack a methodology that both effectively and
efficiently communicates design processes. Describing the literature review, Section 6
aggregates concepts from the organizational science, human computer interaction, and process
modeling research fields to develop DPCM, which describes a representational, technical, and
social environment for process communication. From the literature review, the authors
conclude that DPCM should be Computable, Embedded, Modular, Personalized, Scalable,
Shared, Social, and Transparent. Section 5.4 explains how PIM and DPM process
communication or information management methodologies do not exhibit these
characteristics. Sections 6 through 8 explain DPCM and propose a method for validating its
impact on design process communication.
62
Chapter 3
3
Reid Robert Senescu
Examples of Collaborating, Sharing, and Understanding
Challenges
While Senescu et al. (2011a) provides evidence of the generality of communication challenges
in the AEC industry, this section uses the design of a university business school campus to
provide three examples to provide context to the problems DPCM addresses. The first author
gathered these observations directly and through interviews during his role as structural
engineer on this project.
3.1
Designers Struggle to Collaborate
When designing the campus, researchers identified six discrete stakeholder groups with 29
project goals. The design team evaluated seven mechanical heating/cooling options with
respect to these goals. They divided one building into five different zones and assigned five of
the seven mechanical options to these zones. The team created a Microsoft Excel spreadsheet
to gain consensus on the mechanical heating/cooling decision. The spreadsheet showed the
underfloor air distribution system as the best choice. In the same project folder, the design
team also created AutoCAD files with floor plans to communicate the mechanical systems in
the various zones to owner representatives. These AutoCAD files showed that the designers
frequently chose options other than the underfloor air distribution.
The problem was not that the design team chose the incorrect systems or that their
process for designing the mechanical systems was inconsistent. The problem was that the
process was opaque to everyone but the mechanical engineers. The mechanical engineers
saved the spreadsheet and the AutoCAD files in the project folder with no representation of
the dependencies between any of the supporting files responsible for causing this apparent
inconsistency. If the team knew the process, they would have known that the AutoCAD files
were the most up-to-date and not intended to be dependent on the spreadsheets. Instead, the
63
Chapter 3
Reid Robert Senescu
apparent inconsistency between the two documents inhibited acoustics, lighting, and structural
engineering consultants to maintain information consistency and to comprehend information
dependencies. This inefficient collaboration caused negative rework (Ballard 2000). It also
inhibited effective multi-disciplinary system integration, which was necessary for meeting
many of the stakeholders’ environmental sustainability goals. Designers do not work in an
environment where the project team communicates information dependencies, which makes
collaboration within project teams challenging.
3.2
Designers Struggle to Share Processes
The stakeholders explicitly communicated the importance of material responsibility when
choosing structural systems (Haymaker et al. 2008). The structural engineer created schematic
Revit Structure models of steel and concrete options. The engineering firm had recently
purchased Athena, software that uses a database to output the environmental impact of
building materials. Despite a 3D object-oriented model (containing a database of structural
materials and quantities), a database of the environmental impacts of those materials, and a
desire by the stakeholders to consider environmental impacts of materials in their design
decision, the structural engineer was unable to find a process for conducting an environmental
impact analysis comparing the concrete and steel options. Several months later, the structural
engineer met a researcher in California who had worked in Australia to develop a process for
performing model-based assessments of the environmental impact of construction materials
(Tobias and Haymaker 2007). The Australian research center had worked directly with the
Australian offices of the same engineering firm at the time of the project.
In this case, a clear demand for an improved process existed in the California office.
The engineer could not find a design process to compare options with respect to stakeholder
goals, even though researchers in California and engineers from the same firm in Australia had
64
Chapter 3
Reid Robert Senescu
already performed this process. This example illustrates that designers struggle to efficiently
and effectively share processes between projects teams.
3.3
Designers Struggle to Understand Processes
With the goal of informing the design team’s decision regarding the quantity and size of
louvers on the south façade of the campus library, daylighting consultants created video
simulations of sunlight moving across a space. The process required much manual
manipulation of geometry and materials to reformat the information, so each new software
package could interpret the data representing the building. This process was not productive as
the consultants spent 50% of their time on these non-value adding tasks and considered only
2-3 options resulting in a sub-optimal design.
The individual consultants lacked incentive to invest time in process improvement.
Their tools did not capture their process (and the resulting lack of productivity), place them in
peer communities to improve the process together, nor provide transparent access to other
processes that could form the basis for improvements. Also, managers lacked a transparent
method for understanding process productivity rates and therefore, could not develop a
monetary justification for encouraging process innovation. AEC organizations struggle to
understand their processes across the firm or industry to strategically invest in increased
process productivity.
4
Lack of Effective and Efficient Design Process Communication
Within design process management, the design rationale (Moran and Carroll 1996a) and
design process improvement (Clarkson and Eckert 2005) research field have already
developed effective design process communication methodologies to overcome the challenges
faced by the university building design team. Yet, these research methodologies have not been
65
Chapter 3
Reid Robert Senescu
adopted by industry (Conklin and Yakemovic 1991; Moran and Carroll 1996b). This lack of
adoption is not due to the lack of tools capable of effectively communicating design processes.
Rather, the lack of adoption stems from the lack of incentive for designers to communicate
processes at the instant they are designing. Thus, it is not sufficient to merely have the
methodology and tools to effectively communicate process. The act of communication must
also require little effort; it must be efficient.
This need for efficiency prompted the authors to also investigate the project
information management research field as PIM focuses on improving the efficiency of
information exchange (Froese and Han 2009). The authors’ intuition is that communicating
information exchange would have been sufficient for addressing the university building design
team’s challenges. However, PIM does not address the need to communicate information
exchanges for collaboration, leveraging information systems to benefit sharing across projects
(Malone et al. 1999), and understanding of processes by the firm and industry (Ballard and
Koskela 1998; Hartmann et al. 2009). PIM lacks methodologies for effectively (i.e., able
and/or accurate) communicating processes, whereas DPM literature describes methodologies
for communicating processes, but lacks a sufficiently efficient (i.e., quick and/or with little
effort).methodology for industry to adopt these methodologies.
5
Synthesizing Existing Concepts to Develop DPCM
To address the lack of effective and efficient methodologies for communicating design
processes in the PIM and DPM fields, the authors synthesized concepts from organizational
science, human computer interaction and process modeling research fields to develop
characteristics for the Design Process Communication Methodology. The authors chose these
three fields because of the importance of developing a methodology that would: be adopted by
organizations; facilitate the creation and accessibility of design processes in a computer; and
66
Chapter 3
Reid Robert Senescu
specify a grammar for representing the processes. The authors reviewed 92 papers in these
three research fields by utilizing a snowball approach. After explaining the origins of the
characteristics using a subset of the most influential papers, Section 5.4 evaluates some
examples of PIM and DPM research efforts with respect to the characteristics.
5.1
Organization Science to Enable Adoption
This section first explains why highly interdependent tasks inhibit process standardization and
so, process documentation should be embedded. Research on institutions suggests that
technology should be transparent, social, modular, and shared to best allocate human capital
and creativity. Institutional research on matrix organizations suggests hierarchically structured
information is not suitable in AEC, which the authors interpret to mean information should be
represented in a way that makes process transparent. Finally, Knowledge Management
research calls for embedding and socializing of design process knowledge acquisition,
structuring, and retrieval so processes can be shared.
5.1.1
AEC requires coordination without standardization
Standardization permits coordination when situations are relatively “stable, repetitive and few
enough to permit matching of situations with appropriate rules” (Thompson 1967). In AEC,
the International Alliance for Interoperability developed the Industry Foundation Class (IFC)
to standardize data schema for describing buildings. The Georgia Tech Process for Product
Modeling (GT-PPM) and Integrated Delivery Manuals (IDM) also depend on a standard
design process (Lee et al. 2007; Wix 2007). The new capabilities of simulation software, the
complex demands of stakeholders, and the global nature of design teams make design
processes increasingly complex, dynamic, and based on performance (not precedence).
Organizations with variable and unpredictable situations inhibit process standardization.
Instead, coordination must be achieved by “mutual adjustment,” which “involves the
67
Chapter 3
Reid Robert Senescu
transmission of new information during the process of action” (March and Simon 1958).
Extrapolating to design processes, coordination should occur by embedding the process
documentation in the minute-to-minute work of designers rather than by developing standard
coordination methods. This lack of embedment inhibited the project team described in Section
3.1 from collaborating to make a mechanical system decision, because they were not aware of
each others’ processes. This concept explains why process standards have been relatively
unsuccessful in practice and why convergence to a single product model has not emerged in
AEC.
5.1.2
Form new institutions around processes
Institutionalism research explains relationships between firms and information. In Coase’s
(1937) model for the firm, a firm forms when the gains from setting up the firm including
organizational costs are greater than setting up a market including transaction costs. The open
source software institution does not fit within Coase’s model, and so, Benkler (2002) proposes
the alternative peer production model. Benkler claims that this emerging third type of
institution “has certain systematic advantages over the other two in identifying and allocating
human capital/creativity.” In describing the necessary conditions for processes to be
implemented and shared in this peer production model, Benkler breaks down the “act of
communication” into three parts. First, someone must create a “humanly meaningful
statement.” Second, one must map the statement to a “knowledge map,” so its relevance and
credibility is transparent. Finally, the statement must be shared. In utilizing these advantages
and conditions, a process communication environment can mimic the success of the open
source software industry.
Berger and Luckmann’s (1967) explanation of the firm provides insight as to how to
instantiate Benkler’s peer production model. Berger and Luckmann explain that many menial
68
Chapter 3
Reid Robert Senescu
tasks take much effort to complete. They argue that “habitualization” is human nature, because
it “frees energy” for creativity and “opens up a foreground for deliberation and innovation.” In
building design, habitualization is possible, because many individual tasks are repeated.
Thompson’s “standardization” is difficult, because the same collection of tasks (i.e., a process)
rarely occurs more than once with the same actors. Berger and Luckmann argue that
habitualization of tasks is the reason why institutions form, because institutions can invest in
technology to perform standard tasks, providing an advantage over the sole practitioner. A
larger institution that collectively develops more institutional habits can then focus more on
creative endeavors. For these institutions to exist, “there must be a continuing social situation
in which the habitualized actions of two or more individuals interlock” (Berger and Luckmann
1967). But what happens when the quantity and diversity of tasks and actors is so great that
these social institutions do not occur naturally? Individuals in the organization must
continuously waste energy on tasks that from an institutional perspective seem habitual, but
from the perspective of the individual are unique (e.g., the daylighting consultants in Section
3.3 thought they were the only ones performing the tasks). Can technology facilitate “social
situations” where “individuals interlock” to create reciprocal typification? Habitualization
(i.e., recognition of one’s own repetitive tasks) combined with reciprocal typification (i.e.,
when two people recognize each other’s habits) are critical for the formation of a peerproduction institution. Technology is needed to socialize (i.e., promote collective engagement)
and share (i.e., distribute among the organization) information exchange and make
typification transparent, so institutions can form around common processes. For example, a
community focused on finding the environmental impacts of structural materials could have
made the process described in Section 3.2 habitual within the organization. To reach this
point, however, the community must first find a way to socialize and share this process.
69
Chapter 3
5.1.3
Reid Robert Senescu
Use processes to structure information for the matrix
Programmers in the open-source software movement are simultaneously part of Benkler’s peer
production model and Coase’s traditional firm. Designers also exist within this peer
production model and the traditional AEC matrix organization. The matrix organizations in
large AEC design firms generally form by project, by geography and/or by discipline, but the
firm stores information hierarchically in folder directories. Just as Davis and Lawrence (1977)
claim that new business conditions required a change to the matrix organization, analogously,
expanded uses of digital information require a deconstruction of the hierarchical information
structure. Information now serves multiple purposes. A project team uses a building object
such as a window for architectural rendering, daylighting analysis, and energy analysis.
Designers exert much effort to create this object and so, it no longer belongs to just one
project, but is utilized on multiple projects. In addition, with increased computer power and
demand to view tradeoffs, more designers exchange more information, more frequently. As
shown by the mechanical system design problem, it is difficult to maintain information
consistency. Organizing the information by dependency brings the transparency needed for
consistency.
5.1.4
Knowledge Management without management
An organization’s knowledge is a resource. In this knowledge-based theory of the firm, the
organization is a social community that transforms knowledge into economically rewarded
products and services (Grant 1996; Khanna et al. 2005). Conklin (1996) describes a “project
memory system” to define this knowledge and make it available to others. The project
memory system is necessary, because organizations lack ability “to represent critical aspects
of what they know.” Whereas Conklin generally applies this system to capturing knowledge
from meetings, the same lessons apply to capturing design process knowledge. A process
communication environment that acts as “an evolutionary stepping stone to organizational
70
Chapter 3
Reid Robert Senescu
memory” would allow designers to track information exchanges on a project (e.g., in
Australia) from which designers on another project (e.g., in California) could deduce design
process knowledge. Preserving organizational knowledge requires more than just “capturing
lots of information.” The knowledge must be made sharable by capturing and structuring the
knowledge in ways “that create and preserve coherence and ‘searchability’” (Conklin 1996).
Once Conklin’s stepping stone from project to organization enables knowledge
acquisition, the knowledge must be structured. Hansen et al. (2005) describe two aspects of
knowledge structuring: codification and personalization. Codification relies on information
technology tools to connect people to reusable explicit knowledge (Javernick-Will and Levitt
2010). Personalization relies on socialization techniques to link people so they can share tacit
knowledge. Information Technology can provide the general context of knowledge and then,
point to individuals or communities that can provide more in depth knowledge. Knowledge
management is not just acquisition and structuring (Kreiner 2002). Javernick-Will and Levitt
(2008) address the additional importance of the future ability of others to retrieve the collected
knowledge.
The lack of design process knowledge sharing inhibited the successful material
environmental impact analysis process in Section 3.2 from being utilized by other project
teams outside Australia. This sharing also exists in Benkler’s peer production model. Yet,
Benkler’s model requires minimal if any management. Combining the peer production model
with knowledge management research provides guidance for developing an environment for a
self-perpetuating acquiring, structuring, and retrieval of design process knowledge that is
completely embedded in the design process and requires minimal management.
71
Chapter 3
5.2
Reid Robert Senescu
Human Computer Interaction to Create and Access Processes
HCI specifies how to facilitate the designer’s interaction with the digital representation of the
process. Cognitive Science research develops models for predicting how humans will behave,
but these models will not predict perfectly (Winograd and Flores 1987) and so, it is also
important to research and describe how humans interact with computers (Winograd 2006). In
the former, researchers use cognitive models for Artificial Intelligence (AI) whereas the latter,
HCI, focuses on designing computer systems for humans; not to model humans. However, in
the Human-Information Interaction (HII) and Information Visualization (two branches of
HCI) fields, the distinction between AI and HCI blurs. For example, HII applies Information
Foraging Theory, which itself draws on Cognitive Science research (Pirolli 2007).
Programmers best develop systems for a user to search and comprehend information through
descriptive research on how to model human searching and comprehending; HCI benefits
from AI research. At the same time, the programmer can only model human searching and
comprehending behavior through iteratively testing how humans search and comprehend; AI
also benefits from HCI research.
This section takes this mutually beneficial perspective on HCI and AI. First, Cognitive
Science research calls for personalized process views. Next, the section discusses practical
implications of Cognitive Science with respect to HII and Information Visualization. These
branches of HCI provide insight to make the communication environment sharable, scalable,
social and transparent.
5.2.1
Cognitive Science calls for personalized graphical representations
“The power of the unaided mind is highly overrated…The real powers come from devising
external aids that enhance cognitive abilities” (Norman 1993). Can a technical environment
enhance a designer’s abilities to collaborate, share, and understand? “Solving a problem
72
Chapter 3
Reid Robert Senescu
simply means representing it so as to make the solution transparent” (Simon 1981). To
illustrate, Norman (1993) presents the ticktacktoe game. As a mathematical word problem,
finding a solution is difficult, but represented graphically in the game ticktacktoe, the solution
is obvious. Similarly, the best representation of airline schedule depends on the user’s
objective; a communication environment should represent information according to the user’s
task (Norman 1993). In terms of processes, the graphical representation of information
dependencies should be personalized to the user’s skills. For example, more analyticalminded decision makers (as measured by the Witkin GEFT) made better decisions when
presented with a graph representation of information dependency as opposed to an
“Interaction Matrix” (nearly identical to the Design Structure Matrix used in AEC (Steward
1981)). The decision-making performance of heuristic type individuals was less sensitive to
the graphical presentation of the information dependencies (Pracht 1986). More personalized
views of the mechanical system design process in Section 3.1 would have permitted process
transparency and a more collaborative design decision.
5.2.2
HCI advocates information interaction and visualization
This section seeks to find “how information environments can best be shaped for people”
(Pirolli 2007). Providing methods for achieving this goal, Information Visualization is the
“use of computer-supported, interactive, visual representations” of abstract, non-physical data
to amplify cognition (Card et al. 1999). For example, the human eye processes information in
two ways. Controlled processing, like reading, “is detailed, serial, low capacity,
slow…conscious.” Automatic processing is “superficial, parallel…has high capacity, is fast, is
independent of load, unconscious, and characterized by targets ‘popping out’ during search”
(Card et al. 1999). Therefore, visualizations to aid search and pattern detection should use
features that can be automatically processed. Designers will be able to better draw meaning
from information dependency graphs if the graphs use images, process views at appropriate
73
Chapter 3
Reid Robert Senescu
scales, and spatial layouts indicative of topology (Card et al. 1999; Nickerson et al. 2008).
These strategies will make the environment more transparent.
The capabilities of the human eye also influence information scent – the perceived
value of choosing a particular path to find information (Pirolli 2007). To promote an accurate
and intense scent for the designer to find useful shared processes, search results should show
the actual information dependency graphs. Also, the environment should track the most useful
processes and prioritize these processes in search results.
Heer et al. (2007) shows that social groups will reveal more patterns than the same
number of individuals. Combining conversation threads with visual data analysis helps people
to explore the information broadly and deeply, suggesting a promising opportunity for
supporting collaboration in design activities. The environment should allow the community to
point to specific locations in the graphs to discuss patterns socially.
5.3
Process Modeling to Represent Process
Process modeling research creates a formal grammar for communicating processes to
collaborate, share, and understand. Austin et al. (1999) provide an overview of process
modeling techniques used to communicate the building design process. AEC researchers
delineate process modeling research by different views of the process or by the objectives of
the modeling. For example, Ballard and Koskela (1998) view engineering processes through
conversion, flow, and value generation and hypothesize that transparency of these views will
result in design success from the perspective of that view. AEC researchers develop
generalized process models with the intent of supporting new working methods, identifying
gaps in product information models, and informing new information models (Wix 2007).
Process models may also aim to facilitate collaboration, share better practice, or communicate
decisions. Though process modeling research frequently overlaps multiple objectives, the next
74
Chapter 3
Reid Robert Senescu
sections are organized according to research aimed at improving coordination and planning,
and automation. This literature claims models should be embedded, scalable, shared,
transparent, social, and computable.
5.3.1
Process models aimed at improving coordination and planning
Narratives attempt to overcome the challenges of multi-disciplinary, iterative, and unique
design processes (Haymaker 2006). To facilitate coordination, Narratives create task-specific
views of information flow (consistent with the views suggested by Norman in Section 5.2.1).
Haymaker et al. also express the need to facilitate coordination by representing the status of
information. While the Narratives research calls for embedding of process modeling into the
design process, and identifies and facilitates the need to make the source, status, and nature of
the information dependencies transparent, these concepts are not validated.
As opposed to Narrative’s graph view which communicates a planned or historically
implemented process, the Design Structure Matrix (DSM) uses a matrix view to plan and
algorithms to improve the process. Originally, DSM tracked the dependencies of activities
(Steward 1981), but the Analytical Design Planning Technique (ADePT) extends DSM by
utilizing Data Flow Diagrams (Fisher 1990) and IDEF0 (Austin et al. 1999) to model not just
tasks but also information flow between tasks (Austin and Baldwin 1996; Austin et al. 2000;
Baldwin et al. 1998). An important part of both modeling techniques is their ability to take a
complex system and scale it down into sub-systems.
Embedding such process descriptions in the design process may have permitted the
owner representatives to be more confident in the mechanical system decision by quickly and
accurately comprehending the process. Similarly, the vision of Integrated Practice includes “a
world where all communication throughout the process are clear, concise, open, transparent,
and trusting: where designers have full understanding of the ramifications of their decisions”
75
Chapter 3
Reid Robert Senescu
(Strong 2006). Thus, the process, not just product models, should be shared with the entire
project team, and the information on which decisions are dependent should be transparent.
5.3.2
Process models aimed at improving automation
Comprehending how project teams coordinate aids development of automated information
flow, so recent process modeling efforts support both goals. “Interoperability exists on the
human level through transparent business exchanges” (American Institute of Architects 2007).
The importance of associating people with information exchange to develop automation is
analogous to Hansen’s claim that knowledge must be social, not just codified.
IDMs aim to provide a human-readable integrated reference identifying “best
practice” design processes and the data schemas and information flows necessary to execute
effective model-based design analyses (Wix 2007). IDMs recognize that information must be
tracked at varying scales of detail. To help identify best practice processes, the environment
must also promote sharing by using metrics so designers can evaluate processes. IDMs
contrast with Narrator’s focus on designer communication, but are similar to Geometric
Narrator, which emphasizes the use of process models to perform modular computations on
information (Haymaker et al. 2004).
5.4
Gaps in PIM and DPM Research Relative to Characteristics
This section identifies gaps in PIM and DPM Research with respect to the characteristics
described above.
5.4.1
Improving communication between AEC professionals
Several methods have advanced the design of process models for use by AEC professionals in
practice. Narrator and Geometric Narrator form two fundamental points of departure.
Geometric Narrator enables a designer to build a modular, computable process but lacks a
76
Chapter 3
Reid Robert Senescu
general infrastructure to easily share these processes (Haymaker et al. 2004). Narrator plans
processes and visually describes processes retroactively (Haymaker 2006). Narratives
incorporate Ballard’s information flow view via information dependency arrows and the
conversion view by showing the tool used to transform the information. Narrator addresses
some of the sharing deficiencies (via a searchable database) of Geometric Narrator, but at the
expense of its computable power and embedment in the design process. Neither Narrator nor
Geometric Narrator can be shared widely on the Internet nor personalized to create views to
individual users.
More powerful at process planning, DSM similarly plans the design process through
task dependencies, but also more explicitly identifies iteration and includes methods for
scheduling activities to minimize rework (Eppinger 1991; Steward 1981). Though these task
sequence optimization methods are computable algorithms, it is not within DSM’s scope to
automate information flow, nor act as an information communication tool that links to
particular information. Austin et al. (1999) demonstrated the ability to model 10,015 data
flows on a hospital project, which required 40 hours to capture, though 91% of the data flows
came from a generic process. While this Analytical Design Planning Technique focused on
process modules that could be shared and reused across projects, most DSM research instead
focuses on optimizing the ordering of design tasks without much concern for the effort and
difficulty required to map out task dependencies. “While people have a tacit understanding of
when a process plan is no longer relevant, it is difficult to describe the relationship between
the process plan and the process that actually occurs… Process models are typically generated
to plan, i.e., before the project, and hardly any company goes to the trouble of comparing the
model with the process that actually exists. Process post mortems are rarely done, because
everybody is busy moving onto the next project…” and there are rarely lessons learned about
the process itself (Clarkson and Eckert 2005).
77
Chapter 3
Reid Robert Senescu
DePlan (Choo et al. 2004) attempts to tie ideal schedules derived from ADePT with
the realistically possible execution of those plans based on constraints and resource limitations
as described by LastPlanner (Ballard 1999). DePlan makes the plan dynamic, but there is still
much additional overhead, and it is not integrated with information management systems, so it
is likely relied upon only weekly - not embedded in design.
Critical Path Method (Kelley and Morgan 1959), Lean Production (Howell 1999;
Koskela 1992; Krafcik 1988), Last Planner (Ballard 1999), and Virtual Design Team (Jin and
Levitt 1996) are all fundamentally process planning techniques - not embedded in design.
While they offer insights to DPCM as process planning and control methodologies, they do
not include concepts for communicating digital information and are not discussed here in more
detail.
The Information Value Based Mining for Sequential Patterns (VMSP) is intended for
embedment in the design process to capture design process knowledge (Ishino and Jin 2006).
Ishino and Jin wrote a customized tool that captures changes in a CAD tool, and attempts to
derive design rationale from those changes. VMSP requires intense customization of software
tools and is not scalable to the hundreds of tools used in professional practice (a problem
typical of many of the tools proposed by Moran and Carroll (1996a)). While also intended to
be embedded in design and addressing shared processes, ActivePROCESS may also suffer
from scaling issues when applied to problems more complex than the simple block design
scenario because of the detail with which engineers would need to document all their design
moves (Jin et al. 1999).
Decision Dashboard (DD) improves design rationale transparency by communicating
options, alternatives, and criteria (Kam and Fischer 2004). DD contains some abilities to
compute values associated with the process nodes from information contained in related nodes
78
Chapter 3
Reid Robert Senescu
but does not intend to automate information flows. DD does not easily support multi-user
sharing. DD models design rationale, an important aspect of the design process to
communicate. However, DD does not address research findings that demonstrate that design
rationale systems are rarely implemented in practice, because designers struggle to document
their rationale when performing design (Conklin and Yakemovic 1991; Ishino and Jin 2002;
Moran and Carroll 1996b). Finally, DD focuses on one decision at a time and does not address
how to organize the thousands of decisions made on a typical project; it is not scalable.
Figure 3-1. Comparison of existing research in PIM and DPM with the DPCM characteristics.
The matrix is not intended to cover all research in these two fields, but to show a few
indicative examples of current gaps in the research. While individual research may address
some of these characteristics, the authors have not found a theory that addresses all of them.
79
Chapter 3
Reid Robert Senescu
As opposed to efforts such as GT-PPM (Lee et al. 2007) aimed at modeling processes
to develop a product model standard, DPCM calls for a computable web of individual
interoperability solutions. Many of the current process modeling approaches to improving
interoperability are formulated at an abstract level to define general data exchanges and
processes, and have limited value as a project-specific design guidance and management tool.
That is, software developers, not designers, use these process models, and they are therefore
not intended to be transparent, and sharable from the perspective of the typical designer.
Generally, PIM research efforts focus on modular and computable methods (Figure
3-1) that enable more efficient exchange of information. The methods are either themselves
intended to be embedded in the design process or are intended to facilitate the development of
software embedded in the design process. DPM research focuses on process transparency for
planning before design or story-telling afterwards and is more convincingly scalable to real
projects. DPCM aims to satisfy all the characteristic gaps in the existing DPM and PIM
research.
6
Theory - Design Process Communication Methodology
Gaps exist between the characteristics exhibited by the methodologies described in PIM and
DPM literature and those characteristics recommended by organizational science, human
computer interaction, and process modeling research. Modeling research recommend for a
communication environment and what the existing PIM and DPM literature has contributed.
These points of departure provide important characteristics for the development of a process
communication environment. This section describes elements of DPCM that represent and
contextualize the processes and methods that describe how designers capture and use these
processes by interacting with a computer. The main contribution of this paper, the DPCM, is
the combination of the elements and methods that enable the Characteristics.
80
Chapter 3
6.1
Reid Robert Senescu
Elements and Methods Enabling Characteristics
Embedded
Users use the environment simultaneously to organize and exchange information as well as
communicate processes.
They Save or Open Information, and can Open old versions of information. Each
Information node contains a list of previous Versions of the files. The Status of the
information can be up-to-date, being worked on, or out-of-date. These information
management elements and methods encourage the users to use the environment as the primary
means of exchanging information while they work. The ability to effortlessly Draw arrows
after saving a file embeds process capturing in this information management work flow.
Scalable
The environment scales to the tens of thousands of files exchanged on the largest construction
projects, and also scales across the industry to apply to many different types of projects.
The environment enables scaling within a project by providing access to
representations of information dependencies through a Frame. A Frame is a type of Node that
itself contains views onto a collection of other Contained Nodes. Unlike the nodes which exist
in a single non-hierarchical network, the frames are organized hierarchically. Thus, the user
can choose to Open each frame via a Hierarchy or Network type of Window. This hierarchical
organization enables the representation of processes at multiple levels of detail and ensures
that users are not overwhelmed by visualizations of networks containing dozens of nodes not
relevant to their task.
81
Chapter 3
Reid Robert Senescu
The environment scales across the industry, because it uses a discretization and format
of information common across the industry: the File and the URL. As every project uses
digital files or URL’s to describe some aspect of the building, the environment can be utilized
across the industry.
Personalized
The environment personalizes communication to each user.
As the Frame is simply a view onto nodes, a single Node can exist within multiple
frames. Thus, designers can create custom views of the nodes and their relationships that are
comprehensible and relevant to them. They just Drag and drop nodes into their personal
frames without affecting how others see the nodes.
Transparent
The environment enables the comprehension of processes by the designers.
The environment achieves transparency through arrows between information and
frame nodes. Each arrow represents information Dependency. That is, the End Node is
dependent on the Start Node if information contained within the Start Node was used to create
the information in the End Node.
The environment additionally enables transparency by assigning each node an
information Ribbon. The Ribbon contains a Description of the information contained within
the node. For each Frame node, the Ribbon displays the difference in time between the most
recently uploaded file and the oldest file, indicating the latency since the initiation of the
process, the Duration. The Ribbon also shows how many times (Times viewed) users opened
82
Chapter 3
Reid Robert Senescu
the Frame – an indication of the popularity or importance of the process. Also, each
Information node has a Time stamp showing when the file was last uploaded, and what Tool
was used to create the file based on the file suffix. All nodes have a Title and an Actor that
denotes the person responsible for the node.
Social
The environment promotes social engagement with project information and dependencies.
Within each node’s Ribbon users can Post comments about the information and the
processes in the Discussion thread. They can also Rate the process in terms of its productivity.
Shared
The environment facilitates the distribution of processes.
Users can Search Dependency paths and individual Nodes. Also, users can easily
share their views of processes with others, because each Frame has a URL that can be sent to
other users.
Modular
The environment enables users to combine several parts of other processes into a new process.
It also allows geographically separated users to work on different parts of a process and then
combine their work. This modularity contrasts with strategies aimed at representing all project
information within a single type of data schema and instead encourages discrete modules of
information dependent on each other.
83
Chapter 3
Reid Robert Senescu
Users can thus, mix and match Process modules containing all of the above elements
and Duplicate the Process modules and customize them to specific projects.
Computable
The environment enables users to attach Scripts to a Dependency that would automate
information flow from the Start Node to the End Node. Defining each dependency as a
computable relationship between two pieces of information enables the gradual development
of improved interoperability between tools.
84
Chapter 3
Reid Robert Senescu
Figure 3-2. The Design Process Communication Methodology. Elements represent and
contextualize a process and methods enable designers to capture and use the process model.
These elements and methods enable the Characteristics.
85
Chapter 3
7
Reid Robert Senescu
DPCM Applied to Observed Problems
This section demonstrates how by operationalizing DPCM as the process-based file sharing
tool, the Process Integration Platform, DPCM addresses the three types of communication
challenges described in Section 3.
7.1
Collaboration with PIP
If PIP had been available to the mechanical engineering team on the university building
project, they could have used PIP to collaborate around their digital files. After logging in, the
user sees two personalized home page Windows: a hierarchy view on the left of the screen and
a network view on the right (Figure 3-3). In this case, the mechanical engineer wants to use an
Architecture Model file and a Daylighting Analysis file as input to an energy analysis. The
engineer navigates through the Frame hierarchy to a more detailed process level showing the
architecture and daylighting models. This hierarchical organization of frames enables the
process to be scaled to many files. He drags and drops the Information Nodes containing the
Architecture Model file and the Daylighting Analysis file into his Energy Analysis frame. The
Frames are thus personalized in that the same Information Node containing the Architecture
Model file exists within the context of the Daylighting Analysis frame and within the context
of the Energy Analysis frame. The mechanical engineer then double clicks on each file to
open it on his desktop. The ability to Open and Save files directly in PIP Information Nodes
enables process capturing to be embedded in the design process. He imports the Revit model
into his energy analysis tool. Looking at the daylighting analysis results, he manually enters
the energy required for artificial lighting into the energy analysis tool. After completing the
energy analysis, he double clicks in the graph view to Create an Information Node and Saves
the energy analysis file to that node. As he used the architecture model and daylighting
analysis as input to the energy analysis, he also Draws Arrows from those two nodes to the
86
Chapter 3
Reid Robert Senescu
new energy analysis node to represent this Dependency and make the Process transparent.
Now that the energy analysis is complete, he uses the results to create a decision matrix in
Microsoft Excel. He uploads the Excel file to a new node and draws an arrow to it. When the
architectural design changes, prompting the upload of a new energy analysis file, the
downstream decision matrix file Status is no longer up-to-date (indicated by red highlight),
because it was created based on an out-dated energy analysis file. If based on this new energy
analysis, the mechanical engineers decide on a displacement ventilation system and create an
AutoCAD file dependent on the new energy analysis, the rest of the project teams now know
to integrate their designs with the AutoCAD file and not the out-dated decision matrix. Using
PIP makes the mechanical design process transparent to the entire project team, so they can
comprehend information relationships, consider tradeoffs, and make related information
consistent.
87
Chapter 3
Reid Robert Senescu
Figure 3-3. Collaboration in the Process Integration Platform. Users navigate to the
appropriate process level via the hierarchy view (left) or by double clicking folder icons
(right). Users create nodes, upload files to those nodes, and draw arrows to show relationships
between the nodes. Green highlights indicate the node is up-to-date, and red indicates an
upstream file has changed since the node was uploaded.
7.2
Sharing Processes with PIP
In addition to facilitating collaboration, other teams can also share design processes with the
structural engineer on the university building project allowing calculation of the
environmental impact of materials. Since PIP is web-based, sharing is enabled by the
structural engineer searching for a Process where a project team started with input “Arch .ifc”
to denote an architecture model with an Industry Foundation Class file format and produced
“LCA,” life cycle assessment (Figure 3-4). The results display three projects and the engineer
browses to find the most relevant process. The engineer can Duplicate the relevant Process
module and paste it within the university building frame to be used as a planning template,
which can then be populated with project-specific information.
88
Chapter 3
Reid Robert Senescu
Figure 3-4. Sharing processes in the Process Integration Platform. Users search information
dependency paths to find processes with the input available and the output desired. Users can
then copy processes to new projects.
7.3
Understanding Processes with PIP
With PIP, professionals can understand the processes across the firm or industry, so they can
identify popular inefficient processes and strategically invest in improvement. Each Node has
a Ribbon containing information that describes the process within the frame or the information
contained within the node. PIP offers a process-centric discussion forum for users to Post
Comments and Rate process productivity (Figure 3-5). By socially discussing processes, a
community of designers can discuss where the firm should invest in process improvement. A
community of daylighting consultants could see that their process is Viewed often, but that the
process Duration is long. They could discuss the inefficiencies of the process and decide to
collectively program a script to extract information from a Revit file and convert it to a format
that would be interoperable with the daylighting analysis software. The consultants could then
save that Script in PIP and drive computable information flows automatically.
89
Chapter 3
Reid Robert Senescu
Figure 3-5. Understanding processes in the Process Integration Platform. PIP tracks some
process metrics automatically, so users can evaluate the most popular and time-consuming
processes. Discussion threads are associated with each node, so project teams can discuss
individual files or entire processes.
8
Validation Metrics
8.1
Motivation for the Metrics
Validating DPCM requires measuring the efficiency and effectiveness for each type of
communication. Process communication requires (1) Capturing, (2) Structuring, (3)
Retrieving, and (4) Using processes. Benkler (2002) describes these steps as part of the
information-production chain needed for collaboration in the peer-production model (Section
5.1.2). Knowledge management research describes these steps as needed for sharing of
processes across projects (Section 5.1.4) (Carrillo and Chinowsky 2006; Javernick-Will and
Levitt 2010; Kreiner 2002). Finally, innovation literature cites these steps as required for
companies to understand their processes to make strategic investments in process
improvement (Hargadon and Sutton 2000).
90
Chapter 3
Reid Robert Senescu
Historically, these different types of communication were independent. Companies
have different systems (both technical and organizational) for project management, knowledge
management, and research and development. Yet for each type, the literature suggests steps
for improving collaboration, sharing, and understanding that are similar. Because of this
similarity, at the same time one is exchanging information to collaborate within a project, that
professional can also contribute to sharing processes across projects and to strategic
understanding of processes across the firm or industry. Thus, Section 8.2 measures capturing
and structuring of processes and Section 8.3 measures the retrieving and using of processes
within projects, between projects, and across the firm or industry. The sections combine
capturing and structuring into simply capturing, and retrieving and using into simply using,
because capturing and using provides sufficient granularity for assessment.
8.2
Effectiveness and Efficiency of Capturing Processes
In typical design projects it is difficult to determine the theoretical or ideal information
dependencies. Measuring how accurately the process model matches the actual process is
nearly impossible. However, in a controlled design experiment, the theoretical information
dependencies are known, and capturing effectiveness can be measured as the:
(1.1.1) Percentage of true dependencies captured by the process model.
A communication method that captures a high number of dependencies in a controlled
environment should also capture a relatively high number of the dependencies on an industry
project.
Design projects consist of “production work that directly adds value to final products,
and coordination work that facilitates the production work” (Jin and Levitt 1996). An efficient
method for communicating process will not decrease the amount of time spent on production
work nor increase the amount of time spent on coordination work. That is, capturing processes
91
Chapter 3
Reid Robert Senescu
accurately should not cause any burden on other aspects of the project. Two measurements
indicative of burden are:
(1.1.2) Frequency of value-adding information transfer between designers;
(1.1.3) Number of design iterations.
Design iteration and exchange of information between designers are valuable parts of
production work. When design teams are burdened with managing information, they iterate
and exchange information less frequently.
In the University Building example, project managers planned the process through a
series of milestones. The milestones provided a coarse view of process resulting in the capture
of zero information dependencies. The authors hypothesize that applying DPCM would
capture a much larger percentage of dependencies without the burden caused by previous
methods which required hours of effort invested early in the project (Austin et al. 1999).
8.3
Effectiveness and Efficiency of Using Processes
Once DPCM captures processes, designers can use the processes for the three types of
communication.
8.3.1
Using processes for Collaboration within projects
The ability of a team to collaborate effectively around a process can be measured by the:
(2.1.1) Number of local iterations;
(2.1.2) Number of statements about design trends.
These two metrics both indicate multi-disciplinary collaboration effectiveness. Without
collaboration, teams will optimize locally within their discipline silos. Successful design
solutions require global consideration of multi-disciplinary tradeoffs and the resulting iteration
that enables the best solutions to be found (Akin 2001; Ïpek et al. 2006).
92
Chapter 3
Reid Robert Senescu
Inefficient project teams perform negative rework without ever completing an
internally consistent and complete design (Ballard 2000). The lack of up-front collaboration
means most problems are resolved during construction when the cost of resolution is highest.
Thus, the efficiency of collaboration around a process can be measured by:
(2.1.3) Number of complete and accurate design options produced.
Throughout the design process, a team that collaborates efficiently will produce multiple
design options as they iterate toward a final design. During the design process, efficiency can
be measured by:
(2.1.4) Internal consistency of design assumptions.
For example, in the mechanical engineering problem, the structural engineer may have
assumed no underfloor air distribution in his structural design based on the HVAC Decision
Matrix file, while the electrical engineer may have assumed he could place all his wires in the
underfloor space based on the Mechanical Zoning plans. This inconsistency would delay the
completion of an accurate design option. These types of inconsistencies cause statements of
confusion (See Section 3.1), so collaboration effectiveness can also be assessed by the:
2.1.5 Number of expressions of confusion.
Together these metrics allow researchers to assess the relative ability of different
communication methodologies to impact the efficiency and effectiveness of collaboration
within projects.
8.3.2
Using processes for Sharing between projects
Effective use of other projects’ processes requires retrieving productive processes.
Researchers need a scoring system to evaluate processes. The actual scoring system used may
vary depending on the goals of the project. Clevenger and Haymaker (2011) provide one
93
Chapter 3
Reid Robert Senescu
method for evaluating design processes, though the actual scoring method used can vary as
long as the same method is used to evaluate the comparison of the process communication
methodologies employed. The ability of a methodology to enable the effective sharing of
processes between project teams is indicated by the:
(2.2.1) Score of projects selected to imitate.
For example, many processes for leveraging building information models to perform life cycle
assessments of structural systems may exist. An effective communication methodology will
enable project teams to effectively retrieve and use the best processes. However, retrieving
and attempting to use an appropriate process is insufficient. A project team must be able to use
another project’s process efficiently. Efficient use of a shared process should minimize:
(2.2.2) Number of errors made implementing the shared process.
Errors may include redundant steps such as using more tools than required, using tools
incompatible with other tools, or missing critical analysis. For example, the structural engineer
on the university building project may retrieve the Australian LCA process in Figure 3-4, but
if the structural engineer forgets a critical part of the process, then the methodology does not
enable efficient sharing of processes.
8.3.3
Using processes for Understanding across the firm or industry
AEC companies consider IT investments to be costly and risky, yet investments proceed based
on “gut feel” without understanding current processes and how the specific investment will
improve them (Marsh and Flanagan 2000). An effective process communication method
enables the firm or industry to effectively use their understanding of current processes to
strategically invest in process improvement. Unlike the above communication types, the
authors evaluate effective understanding qualitatively by investigating the ability of a
communication methodology to answer the following questions:
94
Chapter 3
Reid Robert Senescu
1. What are the most important types of information on projects?
2. Who are the most critical individuals on projects?
3. What information flows between tools are most common?
4. What are the latencies between tools or between people?
5. How well is information distributed within the team?
6. What is the relationship between information distribution and project performance?
Of course, some insights require case studies or ethnographic research, and other insights can
be derived more efficiently through IT-based communication methods embedded in projects.
The authors measure understanding efficiency as the time required to achieve the insights. The
time is trivial for DPCM as data visualization tools provide nearly instantaneous access to the
process information.
Table 3-1. Metrics to assess process communication.
Process Communication
Steps
Effectiveness
Efficiency
Capturing
(1.1)
(1.2)
Frequency of valueadding information
transfer between
designers
(1.3)
Number of local iterations
Using
within projects
Percentage of
dependencies captured
(2.1.1) Number of local iterations (2.1.3) Number of complete and
accurate design options
(2.1.2) Number of statements
about design trends
(2.1.4) Internal consistency
(2.1.5) Number of expressions of
confusion
between projects
(2.2.1) Score of projects selected
to imitate
(2.2.2) Number of errors made
implementing a shared
process
across firm or
industry
Time required to gain insight
Insights provided by process
information
95
Chapter 3
9
Reid Robert Senescu
Conclusion
The DPCM contributes to PIM and DPM research fields by laying the foundation for the
development of commercial software that communicates design processes to increase the
value generated per man-hour expended by the AEC industry. The paper makes a case for the
need for such a methodology both based on three examples of communication struggles in
practice and by a review of the DPM and PIM research fields. The authors develop DPCM by
synthesizing concepts from organizational science, human computer interaction, and process
modeling research to conclude that a communication environment should be Computable,
Embedded, Modular, Personalized, Scalable, Shared, Social, and Transparent. However,
current research efforts do not exhibit these characteristics and thus, industry lacks a method
for effectively and efficiently communicating process. In particular, prior research focuses
insufficiently on embedding process communication in minute-to-minute work, fostering a
social community around processes, personalizing process views, and sharing processes.
Elements that represent and contextualize process and methods for capture and using of
process enable these eight characteristics.
The paper validates the legitimacy of the DPCM theory by proposing metrics for
comparing it with other communication methods. Also, PIP shows that developers can
implement the theory, and that such an implementation addresses the three types of
communication struggles observed in practice. Providing additional evidence of the testability
of DPCM, over 200 students used PIP in class projects, design charrettes, and on graduate
student research projects (
Figure 3-6). This adoption of the tool demonstrates both the perceived usefulness of DPCM
and the ability of future research to measure the impact of DPCM on communication
effectiveness and efficiency. This future research will provide further evidence that DPCM
can lay the foundation for commercial software that shifts focus away from incremental and
96
Chapter 3
Reid Robert Senescu
fragmented process improvement toward a platform that nurtures emergence of (1) improved
multi-disciplinary collaboration, (2) process knowledge sharing, and (3) innovation-enabling
understanding of existing processes.
Figure 3-6. Use of the Process Integration Platform (PIP) by students at Stanford University.
PIP is a process-based file sharing web tool that acts as a model for DPCM. Its use
demonstrates that DPCM can be practically applied and tested.
10
References
Akin, O. (2001). "Variants in design cognition." Design knowing and learning: Cognition in
design education, C. Eastman, M. McCracken, and W. Newstetter, eds., Elsevier
Science, Amsterdam, Netherlands, 105-124.
97
Chapter 3
Reid Robert Senescu
American Institute of Architects. (2007). "Integrated project delivery - a working definition."
McGraw Hill Construction, May 15, 2007.
Austin, S., and Baldwin, A. (1996). "A data flow model to plan and manage the building
design process." Journal of Engineering Design, 7(1), 3.
Austin, S., Baldwin, A., Li, B., and Waskett, P. (1999). "Analytical design planning technique:
A model of the detailed building design process." Design Studies, 20(3), 279-296.
Austin, S., Baldwin, A., Li, B., and Waskett, P. (2000). "Analytical design planning technique
(ADePT): A dependency structure matrix tool to schedule the building design
process." Construction Management and Economics, 18(2), 173-182.
Baldwin, A. N., Austin, S. A., Hassan, T. M., and Thorpe, A. (1998). "Planning building
design by simulating information flow." Automation in Construction, 8(2), 149-163.
Ballard, G. (1999). "Can pull techniques be used in design management?" Proceedings of the
Conference on Concurrent Engineering in Construction, Helsinki, Finland.
Ballard, G. (2000). "Positive vs negative iteration in design." 8th Annual Conference of the
International Group for Lean Construction, Brighton, UK.
Ballard, G., and Koskela, L. (1998). "On the agenda of design management research." 6th
Annual Conference of the International Group for Lean Construction, Guaruj, Brazil.
Benkler, Y. (2002). "Coase's penguin, or, Linux and the nature of the firm." Yale Law Journal,
112(3), 367-445.
Berger, P. L., and Luckmann, T. (1967). The social construction of reality, Doubleday, New
York, NY.
Card, S. K., MacKinlay, J. D., and Shneiderman, B. (1999). Readings in information
visualization: Using vision to think, Morgan Kaufmann, San Francisco, CA.
98
Chapter 3
Reid Robert Senescu
Carrillo, P., and Chinowsky, P. (2006). "Exploiting knowledge management: The engineering
and construction perspective." Journal of Management in Engineering, 22(1), 2-10.
Choo, H. J., Hammond, J., Tommelein, I. D., Austin, S. A., and Ballard, G. (2004). "DePlan:
A tool for integrated design management." Automation in Construction, 13(3), 313326.
Clarkson, J., and Eckert, C. (2005). Design process improvement: A review of current
practice, Springer-Verlag, London, UK.
Clevenger, C., and Haymaker, J. (2011). "Metrics to assess design guidance." Stanford
University, Center For Integrated Facility Engineering, Technical Report 191.
Coase, R. H. (1937). "The nature of the firm." Economica, 4(16), 386-405.
Conklin, E. J., and Yakemovic, K. C. B. (1991). "A process-oriented approach to design
rationale." Human Computer Interaction, 6, 357-391.
Conklin, J. (1996). Designing organizational memory: Preserving intellectual assets in a
knowledge economy. CogNexus Institute, Glebe Creek, MD.
Davis, S. M., and Lawrence, P. R. (1977). Matrix, Addison-Wesley Publishing Co., Reading,
MA.
Dossick, C. S., and Neff, G. (2010). "Organizational divisions in BIM-enabled commercial
construction." Journal of Construction Engineering Management, 136(4), 459-467.
Eckert, C. M., Clarkson, P. J., and Stacey, M. K. (2001). "Information flow in engineering
companies - problems and their causes." International Conference on Engineering
Design, Professional Engineering Publishing, Suffolk, UK, 43-50.
Eppinger, S. D. (1991). "Model-based approaches to managing concurrent engineering."
Journal of Engineering Design, 2(4), 283-290.
99
Chapter 3
Reid Robert Senescu
Fisher, N. (1990). "The use of structured data analysis as a construction management research
tool: 1. The technique." Construction Management and Economics, 8(4), 341-363.
Flager, F., Welle, B., Bansal, P., Sorekmekun, G., and Haymaker, J. (2009).
"Multidisciplinary process integration and design optimization of a classroom
building." Journal of Information Technology in Construction, 14, 595-612.
Ford, J. D., and Ford, L. W. (1995). "The role of conversations in producing intentional
change in organizations." The Academy of Management Review, 20(3), 541-570.
Froese, T., and Han, Z. (2009). "Project information management and complexity in the
construction industry." 26th International Conference Managing IT in Construction,
CIB W078, Istanbul, Turkey, 1-10.
Galbraith, J. R. (1977). Organization design, Addison-Wesley Publishing Co., Reading, MA.
Gane, V., and Haymaker, J. (2010). "Benchmarking current conceptual high-rise design
processes." Journal of Architectural Engineering, 16(3), 100-111.
Garcia, A. C. B., Kunz, J., Ekstrom, M., and Kiviniemi, A. (2004). "Building a project
ontology with extreme collaboration and virtual design and construction." Advanced
Engineering Informatics, 18(2), 71-83.
Grant, R. M. (1996). "Toward a knowledge-based theory of the firm." Strategic Management
Journal, 17, 109-122.
Hansen, M. T., Nohria, N., and Tierney, T. (2005). "What's your strategy for managing
knowledge?" Knowledge Management, 77(2), 106-16.
Hargadon, A., and Sutton, R. I. (2000). "Building an innovation factory." Harvard Business
Review, 78(3), 157-166.
100
Chapter 3
Reid Robert Senescu
Hartmann, T., Fischer, M., and Haymaker, J. (2009). "Implementing information systems with
project teams using ethnographic-action research." Advanced Engineering
Informatics, 23(1), 57-67.
Haymaker, J. (2006). "Communicating, integrating and improving multidisciplinary design
narratives." J. S. Gero, ed., Second International Conference on Design Computing
and Cognition, Springer, Netherlands, 635-653.
Haymaker, J. R., Chachere, J. R., and Senescu, R. R. (2011). "Measuring and improving
rationale clarity in a university office building design process." Journal of
Architectural Engineering, in press.
Haymaker, J., Kunz, J., Suter, B., and Fischer, M. (2004). "Perspectors: Composable, reusable
reasoning modules to construct an engineering view from other engineering views."
Advanced Engineering Informatics, 18(1), 49-67.
Heer, J., Viegas, F. B., and Wattenberg, M. (2007). "Voyagers and voyeurs: Supporting
asynchronous collaborative information visualization." Computer Human Interaction,
Association for Computing Machinery, San Jose, CA, 1029-1038.
Homer-Dixon, T. (2000). The ingenuity gap: Facing the economic, environmental, and other
challenges of an increasingly complex and unpredictable world, Knopf, New York,
NY.
Howell, G. A. (1999). "What is lean construction." 7th Annual Conference of the International
Group For Lean Construction, University of California, Berkeley, CA, 1-10.
101
Chapter 3
Reid Robert Senescu
Ïpek, E., McKee, S., Caruana, R., de Supinski, B., and Schulz, M. (2006). "Efficiently
exploring architectural design spaces via predictive modeling." Proceedings of the
12th international conference on architectural support for programming languages
and operating systems, Association for Computing Machinery, New York, NY, 195 –
206.
Ishino, Y., and Jin, Y. (2002). "Estimate design intent: A multiple genetic programming and
multivariate analysis based approach." Advanced Engineering Informatics, 16(2), 107125.
Ishino, Y., and Jin, Y. (2006). "An information value based approach to design procedure
capture." Advanced Engineering Informatics, 20(1), 89-107.
Kelley, J. E. Jr., and Morgan, R. W. (1959). "Critical-path planning and scheduling." Papers
Presented at the December 1-3, 1959, Eastern Joint IRE-AIEE-ACM Computer
Conference, Association for Computing Machinery, Boston, MA, 160-173.
Javernick-Will, A., Levitt, R., and Scott, W. R. (2008). "Mobilizing knowledge for
international projects." Proceedings of the 2008 ASCE LEED Conference, CIB Task
Group 64 and ASCE Construction Research Council, Lake Tahoe, CA.
Javernick-Will, A., and Levitt, R. E. (2010). "Mobilizing institutional knowledge for
international projects." Journal of Construction Engineering Management, 136(4),
430-441.
Jin, Y., and Levitt, R. E. (1996). "The virtual design team: A computational model of project
organizations." Computational & Mathematical Organization Theory, 2(3), 171-195.
102
Chapter 3
Reid Robert Senescu
Jin, Y., Zhao, L., and Raghunath, A. (1999). "ActivePROCESS: A process-driven and agentbased approach to supporting collaborative engineering." Proceedings of the Design
Engineering Technical Conferences in ASME, Las Vegas, NV.
Kam, C., and Fischer, M. (2004). "Capitalizing on early project decision-making opportunities
to improve facility design, construction, and life-cycle performance--POP, PM4D, and
decision dashboard approaches." Automation in Construction, 13(1), 53-65.
Khanna, T., Palepu, K. G., and Sinha, J. (2005). "Strategies that fit emerging markets."
Harvard Business Review, 83(6), 63-76.
Koskela, L. (1992). "Application of the new production philosophy to construction." Stanford
University, Center for Integrated Facility Engineering, Technical Report 72.
Krafcik, J. F. (1988). "Triumph of the lean production system." Sloan Management Review,
30(1), 41-52.
Kreiner, K. (2002). "Tacit knowledge management: The role of artifacts." Journal of
Knowledge Management, 6(2), 112-123.
Lee, G., Sacks, R., and Eastman, C. (2007). "Product data modeling using GTPPM -- a case
study." Automation in Construction, 16(3), 392-407.
Malone, T. W., Crowston, K., Lee, J., Pentland, B., Dellarocas, C., Wyner, G., Quimby, J.,
Osborn, C. S., Bernstein, A., Herman, G., Klein, M., and O'Donnell, E. (1999). "Tools
for inventing organizations: Toward a handbook of organizational processes."
Management Science, 45(3), 425-443.
March, J. G., and Simon, H. A. (1958). Organizations, John Wiley & Sons Inc., New York,
NY.
103
Chapter 3
Reid Robert Senescu
Marsh, L., and Flanagan, R. (2000). "Measuring the costs and benefits of information
technology in construction." Engineering, Construction and Architectural
Management, 7(4), 423-435.
Moran, T. P., and Carroll, J. M. (1996a). Design rationale: Concepts, techniques, and use,
Lawrence Erlbaum Associates, Mahwah, NJ.
Moran, T. P., and Carroll, J. M. (1996b). "Overview of design rationale." Design rationale:
Concepts, techniques, and use, T. P. Moran and J. M. Carroll, eds., Lawrence Erlbaum
Associates, Mahwah, NJ.
Nickerson, J. V., Corter, J. E., Tversky, B., Zahner, D., and Rho, Y. J. (2008). "Diagrams as
tools in the design of information systems." Design Computing and Cognition, J. S.
Gero and A. K. Goel eds., Springer, Atlanta, GA, 103-122.
Norman, D. (1993). "The power of representation." Things that make us smart, Basic Books,
43-76.
O'Donovan, B., Eckert, C., Clarkson, J., and Browning, T. R. (2005). "Design planning and
modelling." Design process improvement: A review of current practice, J. Clarkson
and C. Eckert, eds., Springer, London, UK, 60-87.
Paulson, B. C. J. (1976). "Designing to reduce construction costs." Journal of the Construction
Division, 102(4), 587-592.
Pirolli, P. (2007). Information foraging theory: Adaptive interaction with information, Oxford
University Press, New York, NY.
Pracht, W. E. (1986). "GISMO: A visual problem-structuring and knowledge-organization
tool." IEEE Transactions on Systems, Man and Cybernetics, 16(2), 265-270.
104
Chapter 3
Reid Robert Senescu
Senescu, R., Aranda-Mena, G., and Haymaker, J. (2011). "Relationships between project
complexity and communication." Stanford University, Center For Integrated Facility
Engineering, Stanford, CA, Technical Report 196.
Simon, H. A. (1981). "Cognitive science: The newest science of the artificial." Cognitive
Science, 4(1), 33-46.
Steward, D. (1981). "The design structure matrix: A method for managing the design of
complex systems." IEEE Transactions on Engineering Management, 28(3), 74–87.
Strong, N. (2006). “Report on Integrated Practice.” American Institute of Architects,
Washington D.C.
Thompson, J. D. (1967). Organizations in action; social science bases of administrative
theory, McGraw-Hill, New York, NY.
Tobias, J., and Haymaker, J. (2007). "A model-based LCA process on Stanford University’s
green dorm." International Life Cycle Assessment and Management Conference,
Portland, Oregon, October 2 to 4.
United States Department of Commerce: Bureau of Labor Statistics. (2003). "Construction &
non-farm labor productivity index (1964-2003)."
Weber, M. (1947). The theory of social and economic organization, The Free Press, Glencoe,
IL.
Winograd, T. (2006). "Shifting viewpoints: Artificial intelligence and human–computer
interaction." Artificial Intelligence, 170(18), 1256-1258.
Winograd, T., and Flores, F. (1987). Understanding computers and cognition: A new
foundation for design, Addison-Wesley, Reading, MA.
105
Chapter 3
Reid Robert Senescu
Wix, J. (2007). "Information delivery manual: Guide to components and development
methods." buildingSMART, Norway.
106
Chapter 4
Reid Robert Senescu
Chapter 4: Communicating Design Processes Effectively and
Efficiently
Reid Robert Senescu and John Riker Haymaker
Abstract
Existing design process management methodologies for communicating design processes
cannot be efficiently implemented. Project information management methodologies developed
to efficiently exchange information do not effectively communicate processes. Consequently,
the Architecture, Engineering, and Construction industry struggles to: (1) collaborate within
projects, (2) share processes between projects, and (3) understand processes across projects
to strategically invest in improvement. This paper contributes a set of metrics and an
accompanying test method that validate a methodology’s ability to communicate processes
effectively and efficiently to overcome all three of these communication challenges. Second,
the paper uses the set of metrics and the test method to validate the Design Process
Communication Methodology (DPCM). Results demonstrate that designers employing DPCM
accurately capture processes with little effort. When collaborating within project teams,
process clarity and information consistency result in little rework, and positive iteration
enables consideration of multi-disciplinary design trends. Designers share processes between
project teams and use the shared processes without committing process mistakes. DPCM
enables the understanding of processes across projects to provide insights into the
relationship between design integration and project performance as well as opportunities for
strategic investment in improved processes. Thus, this paper both validates DPCM’s ability to
107
Chapter 4
Reid Robert Senescu
improve design processes through effective and efficient process communication and enables
future researchers to validate new process improvement methodologies.
1
Introduction
The Design Process Communication Methodology (DPCM) aims to increase the effectiveness
and efficiency with which Architecture, Engineering, and Construction (AEC) industry design
professionals communicate processes. Senescu et al. (2011b) explain the operationalization of
DPCM through the development of the Process Integration Platform (PIP) - a web tool that
enables project teams to organize and exchange files as nodes in an information dependency
map that emerges as they work. Senescu et al. established both theoretical and practical
justification for DPCM as a contribution to the design process management (DPM) and
project information management (PIM) research fields. This paper first contributes a set of
metrics and an accompanying test method for comparing DPM or PIM methodologies. Then,
this paper validates that DPCM enables the effective and efficient communication of processes
within project teams, between project teams, and across project teams.
This introduction first explains how the DPM and PIM research fields attempt to
improve design processes. Next, this section explains three types of process communication
that can improve design processes: collaboration within project teams, sharing of processes
between teams, and understanding of processes across projects. Then, the introduction closes
with an overview of the rest of the paper.
1.1
Research to Improve Design Processes
Design processes are often unproductive (Flager and Haymaker 2007; Gallaher et al. 2004;
Navarro 2009; Scofield 2002; Young et al. 2007). The design process management research
field addresses this lack of productivity through the design rationale (Moran and Carroll 1996)
108
Chapter 4
Reid Robert Senescu
and design process improvement research (Clarkson and Eckert 2005). The research
frequently attempts to improve design processes by first validating descriptive and predictive
process modeling methods using industry observation and case studies (Cross and Roozenburg
1992). DPM research then lays the foundation for normative research that proposes new
methods aimed at directly improving industry design processes. Researchers attempt to
validate these new methods by demonstrating increased value per man-hour expended. For
this validation, DPM researchers either use a research method such as a design experiment
(Clayton et al. 1998), or they directly apply the method in industry using a research method
such as Ethnographic-Action (Hartmann et al. 2009).
Both the design rationale and design process improvement literature frequently
attempt to improve design processes through more effective and efficient methods of process
communication (Ford and Ford 1995). As digital files are the primary deliverable of AEC
design professionals, the authors intuit that maps of the dependencies between these files
would be indicative of design processes. This intuition that design processes can be
communicated through visualizing information dependency maps led the authors to
investigate the project information management research field. PIM attempts to improve
design processes by improving the efficiency with which project teams exchange information
by aiding the management of a project’s information systems (Froese and Han 2009).
PIM focuses on methods for efficiently exchanging information within a project and thus
improving process productivity. On the other hand, DPM attempts to reach similar
productivity gains by focusing on improving the effectiveness of process communication.
1.2
Communicating Design Processes
The AEC design process consists of organizations exchanging information that lead to
drawings of a building product (Garcia et al. 2004). Communication is the “process of
109
Chapter 4
Reid Robert Senescu
exchange of information between sender and receiver to equalize information on both sides”
(Otter and Prins 2002). Combining the two definitions, professionals communicate processes
by exchanging information that describes how professionals exchange information. Design
processes can be improved through three types of effective and efficient process
communication:
1. The organization can collaborate more effectively and efficiently within the project
team. In this case, the organization does not significantly change the topology of
information exchanges on a project, but simply executes the exchanges better through
improved comprehension of the project team’s processes. For example, the
information may be more consistent throughout the project or a particular project team
member may make a discipline-specific decision with more insight about how that
decision impacts other disciplines.
2. One project team may share a process between project teams. For example, a team
may learn about better software that they then implement on their project.
3. A team may consciously develop an improved process. Developing improved design
processes requires strategic investment, which requires a claim that the return will be
an improvement on the current state. Organizations must understand their current
processes to invest in improvement across projects. For example, a team may
understand that they repeatedly count objects in their building information model and
then manually enter quantities into a cost estimating spreadsheet, and so they invest in
developing a script that performs the process automatically.
The AEC industry struggles to collaborate, share, and understand design processes effectively
and efficiently (Senescu et al. 2011a; Senescu et al. 2011b).
110
Chapter 4
Reid Robert Senescu
Each types of process communication challenge requires consideration of how
designers: (1) Capture, (2) Structure, (3) Retrieve, and (4) Use processes. Benkler (2002)
describes these steps as part of the information-production chain needed for collaboration
when team members are not aligned by the goals and hierarchy set by a single firm.
Knowledge management research describes these steps as needed for sharing of processes
across projects (Carrillo and Chinowsky 2006; Javernick-Will and Levitt 2010; Kreiner 2002).
Finally, innovation literature cites these steps as required for companies to understand their
processes to make strategic investments in process improvement (Hargadon and Sutton 2000).
However, the literature lacks a unifying methodology enabling the development of technology
to support design teams in employing these four steps necessary for the three types of
communication.
1.3
Overview of this Paper’s Layout and Contributions
DPCM provides this unifying collaboration-sharing-understanding methodology by enabling
the communication of processes within project teams, between project teams, and across
multiple project teams. Section 2 summarizes DPCM. To model DPCM and validate it, the
authors developed the Process Integration Platform. PIP is a web tool that enables project
teams to manager and exchange files as nodes in an information dependency map that emerges
as they work. This paper validates the impact of DPCM with the Mock-Simulation Design
Charrette (MSDC) and a modified Ethnographic-Action method (Hartmann et al. 2009) as
described in Section 3. After presenting validation results (Section 4), the paper concludes
with an explanation of how the set of metrics, accompanying test method, and DPCM
contribute to DPM and PIM through validation of DPCM’s impact on design process
communication effectiveness and efficiency (Section 5 and 6).
111
Chapter 4
2
Reid Robert Senescu
Design Process Communication Methodology
DPCM lays the foundation for commercial software that fosters effective and efficient process
communication. DPCM consists of elements (Figure 4-1) that represent and contextualize the
process and methods that describe how designers capture and use these processes by
interacting with a computer. One example of an element is the Dependency, which represents
information from one file being used to create information in another file. One example of a
method is Draw Arrow, which is how the designer defines the Dependency. The elements and
methods enable the characteristics derived from the points of departure as described in detail
in Senescu et al. (2011b). For example, the element, Dependency, enables process
Transparency. The method, Draw Arrow, enables the capturing of process to be Embedded in
minute-to-minute work as designers organize and exchange files.
112
Chapter 4
Reid Robert Senescu
Figure 4-1. Summary of the Design Process Communication Methodology. DPCM consists of
seven elements that enable design teams to represent and contextualize processes. Elements
contain methods and attributes (not shown) described in more detail in Senescu et al. (2011b).
3
Validation Method
Before it was possible to validate DPCM, the authors operationalized DPCM using the Agile
Development Method (Cohn 2004) as a framework for mapping DPCM to the Process
Integration Platform’s internet application features. PIP is a web tool that enables project
teams to manage and exchange information files as nodes in an information dependency map
(Figure 4-2). After PIP became usable, the authors shifted toward the Ethnographic-Action
Method (Hartmann et al. 2009) to further develop PIP, which follows the close study of user
behavior advocated by Kohavi et al. (2007).
113
Chapter 4
Reid Robert Senescu
Figure 4-2. The Process Integration Platform. PIP is a web tool enabling project teams to
organize and share files as nodes in an information dependency map that emerges as they
work. Users navigate to the appropriate process level via the Hierarchy view or by double
clicking frame icons in a Network view. Users create nodes, upload files to those nodes, and
draw arrows to show relationships between the nodes. Green highlights indicate the node is
up-to-date, and red indicates an upstream file has changed since the node was uploaded. Users
can also search dependency paths to find relevant historic processes from other projects. Each
node contains an information ribbon providing additional process information and the
opportunity to rate process productivity or comment.
This section summarizes previous research methods used to validate design process
improvement research. Building on these previous efforts, the section describes the test setup,
metrics, and data collection system used to validate DPCM.
114
Chapter 4
3.1
Reid Robert Senescu
Summary of Validation Methods for Design Process Improvement
Research
Senescu et al. (2011b) discusses and compares DPM and PIM, concluding that previous
research lacks a methodology enabling both effective and efficient design process
communication. In response, that paper proposed DPCM. However, the extent to which
DPCM is effective and efficient remained an open question, so this section builds on previous
design process improvement work to synthesize two research methods appropriate for
evaluation of DPCM: Mock-Simulation Charrette method (MSDC) and Ethnographic-Action
Method.
The first method for validating DPCM comes primarily from the Charrette Test
Method, which combines the architectural notion of charrette (a short, intense design exercise)
with the software usability testing common in the human computer interaction research field.
Clayton et al. (1998) developed the charrette method “to provide empirical evidence for
effectiveness of a design process to complement evidence derived from theory.” The method
permits:

multiple trials which increases reliability;

repeatable experimental protocol;

objective measurements;

comparison of two processes to provide evidence for an innovative process.
Researchers can widely apply this method to design computing research questions, but they
must customize the method to their particular question. For example, Clevenger and
Haymaker (2009) create a customized charrette called the Design Exploration Assessment
Methodology, which enables designers to use Energy Explorer (a Microsoft Excel
115
Chapter 4
Reid Robert Senescu
spreadsheet) to quickly generate and record design alternatives to provide quantitative
measurements of different design strategies.
The authors similarly customized the charrette method with additional inspiration
from research outside of AEC. For example, Heiser and Tversky (2006) performed A-B
experiments with students and concluded that showing students diagrams with arrows caused
the students to describe equipment with functional verbs as opposed to nouns and adjectives
describing the equipment structure. Also, students with text descriptions containing functional
verbs were more likely to draw arrows. Heiser and Tversky did not make normative claims
about whether, for example, teams should use more arrows when collaborating with each
other, but, unlike Clayton’s implementation of the Charrette Test Method, they described a
cognitive phenomenon by recording user language. Using instant messaging, the MSDC also
records language to evaluate the impact of DPCM on, for example, designer confusion.
Another inspiration for MSDC was GISMO – a method that aims to improve decision
making by graphically displaying information dependencies. Pracht (1986) demonstrated that
business students made decisions leading to higher net income for their mock companies when
presented dependencies in graphical form. Though not applied to a design problem, the
validation method for GISMO presented quick, quantitative results demonstrating that a new
computer-aided process resulted in students making more effective decisions to achieve a
clearly defined goal.
Inspired by these validation methods, the authors developed the MSDC method which
customizes Clayton’s method while still leaving it sufficiently general such that other
researchers could use the method to compare their DPM research with DPCM. The authors
also validated the DPCM using student class projects, because compared to the charrettes,
student class projects (1) work on a time scale more on par with professional projects; (2) have
more freedom to choose processes and tools like in professional projects; (3) work at a level of
116
Chapter 4
Reid Robert Senescu
product detail similar to scheme design in professional projects; (4) tackle more of the
“wicked” and complex problems with more ambiguous design goals faced by professionals
(Bachman 2008; Rittel and Webber 1973; Senescu et al. 2011a). These attributes of student
design projects translates to more intertwined production and coordination work. This
intertwining makes it difficult to isolate and measure impacts on coordination work, which
inhibits the ability to use the method for validation of collaboration and sharing. However,
these drawbacks do not inhibit the authors’ ability to draw insights about their design
processes to validate the impact on process understanding.
Thus, the authors selected the MSDC to validate collaboration and sharing and the
Ethnographic-Action method in class projects to validate understanding. The authors utilized
the Ethnographic-Action method, because of its success in validating the development and
implementation of information systems (Hartmann et al. 2009). In particular, Beylier et al.
(2009) demonstrated how a research method nearly identical to Ethnographic-Action
successfully validated the KALIS methodology, which, similar to DPCM, embedded process
sharing capabilities into an information management tool. Like the KALIS research method,
both the MSDC and Ethnographic-Action research can be categorized as part of a
comprehensive study in the Design Research Methodology’s “Descriptive Study II stage to
investigate the impact of the support and its ability to realize the desired situation” (Blessing
and Chakrabarti 2009). MSDC and Ethnographic-Action use the information processing view
of design to validate research that aims to support design (Blessing and Chakrabarti 2009).
Alternatively, adopting the CIFE Horseshoe research framework (Fischer 2006), MSDC and
Ethnographic-Action are specific research methods used in the testing task to validate the
DPCM theory presented in Senescu et al. (2011b).
117
Chapter 4
3.2
Test Setup
3.2.1
Mock-Simulation Design Charrette setup
Reid Robert Senescu
The authors chose students as opposed to professionals for the charrettes, because it was easier
to recruit student volunteers and previous charrettes found no correlation between professional
experience and performance in the charrette (Clevenger 2010). The author incentivized
students to participate with a pizza party and a prize to the team that completed an accurate
design of the classroom with the highest Net Present Value (NPV).
Students signed up for different sessions during a three month period without knowing
whether the sessions were going to be control or experiment cases - a between-subject
experiment design. Each session consisted of one team of five students. The first author
randomly assigned each student to a design role: Architect, Environmental Consultant,
Mechanical Engineer, Structural Engineer, and Cost Estimator. The first author presented a
summary of the information in the remainder of this section to each session via a PowerPoint
presentation (see Appendix) and a tutorial.
The teams were chosen to design the next generation of green classrooms and their
goal was to collaborate to maximize the NPV. The teams calculated NPV using a subset of the
20 Microsoft Excel mock-simulation tools available.
The teams had access to the project frames of six historic projects (Figure 4-3a). Prior
to the charrette, the first author completed the six different designs in each project frame using
six different subsets of the 20 tools. Thus, the teams could choose what tools to use for their
new classroom project by looking at the tools from the historic project frames, or they could
choose the tools from a project frame containing all 20 tools. The teams did not need to use all
the tools and certain tools are not interoperable with other tools.
118
Chapter 4
Reid Robert Senescu
For example, the structural engineer could choose to use GreenStructuralAnalysis.xls,
StructuralAnalysis.xls, GravityAnalysis.xls, SeismicAnalysis.xls, FoundationDesign.xls,
and/or SuperStructuralAnalysis.xls. Many of these tools were redundant.
SuperStructuralAnalysis.xls conducted both a gravity and seismic analysis, so using the
Gravity, Seismic and SuperStructural tools would have been inefficient. On the other hand,
only GreenStrucutralAnalysis.xls outputted the Environmental Cost of Structural Materials, so
if the Cost Estimator depended on this value for the GreenNetPresentValueCalc.xls, the two
designers would have had to collaborate to ensure they use these two interoperable tools. The
various tools received different inputs and gave different outputs. Some also used different
units, allowed simulation of different types of designs, and took a different amount of time to
analyze. The variety of tools simulated the real choices of professional designers and the
interoperability problems of complex software tools.
119
Chapter 4
Reid Robert Senescu
Figure 4-3. Mock-Simulation Design Charrette test setup. All teams open up PIP to see the
historic project frames (a). Control teams can open any of the six historic project frames to
view the list of tools used to complete that project (b). Experimental teams see the same list of
tools used to complete the project and their dependencies (c). The design teams work in the
“Team C Classroom” frame, which is initially blank, but then becomes populated with the
tools used to design the classroom. The work of the control teams eventually resembles a list
similar to (b) and the work of the experimental teams resembles the network in (c).
Each designer inputted independent variables into one or more of the mock-simulation
tools. The mock-simulation tools then analyzed the input values to output performance values
(Figure 4-4). The conversion of inputs to outputs did not correlate with first principle
predictions of building performance but did follow general trends based on actual analysis.
This lack of correlation was acceptable because the intent of MSDC was to model the
coordination of design work; the work performed between simulations. The actual input and
output values had little absolute significance, which was preferable to using real simulation
120
Chapter 4
Reid Robert Senescu
tools because MSDC nullified the domain specific skills and experience of the test participants
and focused instead on the coordination design work impacted by the design process
management research.
Two differences existed between control and experiment groups. The control group
could not draw arrows. They simply exchanged files with each other in a list similar to most
teams in professional practice save files in Windows Explorer. The first author instructed
experimental groups to draw arrows to show the dependency between tool files. This
difference measured the impact of DPCM on collaboration
In addition, the control group could search and view information on historic projects,
but not information dependencies (Figure 4-3b). The experimental group could search and
view historic projects’ information dependencies (Figure 4-3c). This difference enabled the
measurement of the impact of DPCM on sharing.
The teams began the charrette with a ten minute meeting to plan their design process.
Simulating the typical non-collocated, asynchronous project team, the participants dispersed
and sat at different computers and communicated only via instant messaging and PIP.
121
Chapter 4
Reid Robert Senescu
Figure 4-4. Example of a mock-simulation tool. All 20 mock-simulation tools resemble this
Energy Analysis tool. In this case, the Mechanical Engineer finds dependent input values from
the output values of other tools. He then chooses a design by selecting input independent
variables. Clicking the “Analyze” button produces the output values, which become input to a
subsequent tool.
3.2.2
Ethnographic-Action research in design class projects
The authors selected student projects as opposed to a professional project, because student
projects: (1) do not require the usability, reliability, security, and legacy integration required
122
Chapter 4
Reid Robert Senescu
by professionals; and (2) enable simultaneous comparison of multiple projects of similar scope
within a three month period. Four professors chose to use PIP as the primary process capturing
and file sharing web tool in five classes, but the results presented in this paper came from use
in one Multi-disciplinary Design and Analysis course taught in the winter of 2010. 32 student
designers in this class worked on eight teams of three to five students over three months. The
designers adopted specific roles similar to the charrettes and professional practice. Each team
presented a design solution for a real project for which they received a grade.
3.3
Techniques for Collecting Data and Measuring Results
The authors instrumented PIP to measure the effectiveness and efficiency of (1) Capturing, (2)
Structuring, (3) Retrieving, and (4) Using processes. This section describes how the authors
measured and/or calculated each result in Table 4-1.
PIP stores the DPCM attributes and logged most user actions in a MySQL database.
Each Excel tool records design values every time a designer calculated output values. The
designers used logged instant messaging to communicate with each other. The first author told
students they could only exchange design values using PIP and the first author did not observe
cases of work not uploaded into PIP, so it is unlikely that designers performed much work that
was not recorded.
The authors calculated the percentage of dependencies captured with a VB algorithm
that matched the input values of uploaded files with the output values of previously uploaded
files. If there was a match and an arrow was drawn, then the dependency was captured. If a
match existed and an arrow was not drawn, then the dependency was not captured. If no
matches were found, the authors manually investigated the dependency. As the authors did not
know what information influenced decision making, false-positives (i.e., cases where a user
draws an arrow without using information from the start node) could not be calculated.
123
Chapter 4
Reid Robert Senescu
Each tool contained a Visual Basic (VB) macro recording design options calculated in
each tool. Another VB macro aggregated all the tools uploaded to PIP and calculated the total
number of local iterations per person.
The first author compiled all logs of instant messages and blind coded the chats for the
number of statements about trends and of confusion. An example of a trend statement was
“cement weight has a lot more environmental cost than steel.” A statement of confusion
example was “sorry I was confused, hold on. Let me redo the structure calcs.”
Inconsistency occurred when an input value in one tool did not match the output value
in a previous tool. A VB algorithm checked the consistency between dependent files.
Occasionally, the algorithm incorrectly identified inconsistent variables, and so the first author
manually checked the variables the tool identified as inconsistent. A high percentage of
inconsistent variables could occur because the designers simply copied a number incorrectly
or because they used the wrong precedent file to retrieve an output value
Whereas inconsistency looks at information transfer between any tools, the number of
complete and accurate designs reflected a global iteration that includes an NPV calculation.
The first author emphasized to students that they cannot simply fabricate values. For example,
if a designer inputs Total Energy as 145 MJ/year into the NPV tool, a corresponding energy
analysis tool must have an output of 145 MJ/year. Otherwise the NPV calculation was invalid
and the global iteration was incomplete or inaccurate.
The effectiveness of sharing required an assessment of whether teams chose better
historic projects to mimic. Before looking at what historic projects the teams mimicked, the
authors developed an equation that calculates a score for the process employed in the previous
projects. The score considers number of tools required, number of arrows, rating (1 to 5 stars),
discussion posts, process duration, and times viewed. Looking at the tools used, teams
124
Chapter 4
Reid Robert Senescu
frequently combined process modules from more than one previous project, so the authors
calculated the mean process score of projects mimicked.
Once evaluating the effectiveness of the teams’ retrieval of historic projects to mimic,
the authors looked at the efficiency with which the teams applied these historic processes. The
number of missing tools is calculated with respect to the tools required to complete the NPV
calculation. Frequently, designers were confused and wasted time trying tools not necessary
for their NPV calculation, which prompted the calculation of the number of redundant tools
used. The number of non-interoperable tools used reflects pairs of tools that the designer
attempted to use together, but that were incompatible, because the output format of one
variable did not match the input format of the variable in the dependent tool.
Applying these metrics to the charrettes resulted in non-normally distributed data.
Consequently, p-values of counts are calculated using the non-parametric permutation method
(Hothorn et al. 2010). For binary calculations such as whether a global iteration was
completed accurately, the authors used a binary two sample test called the Fisher Test (R
Development Core Team and contributors worldwide). Neither method depends on an
assumed distribution. The authors considered a p-value less than 0.10 as statistically
significant; i.e., the authors are at least 90% confident that DPCM caused the observed
differences between control and experimental teams.
4
Results and Discussion
This section presents results (Table 4-1) according to the four steps required for
communication: (1) Capturing, (2) Structuring, (3) Retrieving, and (4) Using processes.
Capturing and structuring is combined into one activity validated by the charrettes to enable
effective and efficient Collaboration, Sharing, and Understanding. Retrieving and Using is
125
Chapter 4
Reid Robert Senescu
also combined and is evaluated within projects, between projects, and across multiple projects.
Senescu et al. (2011b) explains the rationale behind the metrics.
4.1
Design Charrettes for Collaboration and Sharing
The time designers worked on their classroom designs varied between 57 and 67 minutes, with
just a one minute average difference between the four control and four experiment teams. PIP
crashed for experiment team D, so their results are not reported. From California, the authors
also led a charrette with a team of professionals located in Singapore and Australia. Technical
difficulties and a temporarily absent team member limited this session to 30 minutes, so this
session is not included in the results.
4.1.1
Capturing processes effectively and efficiently
DPCM’s ability to communicate relies on accurate process capturing. Individuals employing
DPCM captured 93% of the 132 actual information dependencies between the tools.
Moreover, the same designer was responsible for seven of the nine missing arrows. The
authors interpret this result as evidence for the power of DPCM to capture process effectively.
In conventional practice and in the design charrette teams without DPCM, designers simply
list files with zero capturing of the dependencies between the files.
As discussed in the Introduction and extensively in Senescu et al. (2011b), many
previous DPM research efforts demonstrate the ability to capture process effectively. One of
DPCM’s unique contributions is the efficiency with which process capturing occurs. This
efficiency is important since designers use the information exchange method requiring the
least effort (Ostergaard and Summers 2009). After uploading a file to share with teammates,
the designer effortlessly draws an arrow from the files containing information that the designer
used for the newly shared file. By recording the time between file upload and arrow drawing,
the authors confirm that capturing is trivially efficient. Furthermore, no evidence exists that
126
Chapter 4
Reid Robert Senescu
suggests a burden on the teams drawing arrows. Designers employing DPCM exchanged
about the same number of files with others. Designers also iterate more frequently in their
tools and discussed trends more frequently. These results suggest that capturing processes did
not reduce the time spent on value-added production work and thus, DPCM enables not just
effective but also efficient process capture.
4.1.2
Using processes effectively and efficiently to Collaborate within teams
Capturing process effectively and efficiently does not necessarily equate to usefulness. After
all, other DPM research captures the rationale behind decision-making or dependency
relationships at the variable level. DPCM only captures the dependencies between aggregated
groups of variables (i.e., files) and does not require specification of the rationale behind the
process nor the extent or type of dependency. Does DPCM’s limited definition of process
capture still enable effective and efficient use of process for collaboration within projects?
Effective collaboration entails consideration of multi-disciplinary design tradeoffs.
Isolated in discipline-specific silos, designers will iterate to optimize designs only within their
discipline as opposed to working collaboratively to optimize globally. The design charrette
results provide evidence that DPCM positively impacts collaboration, because teams
employing DPCM discussed trends 37% more frequently (p=0.06) and designers accordingly
iterated 60% more (p=0.02). The authors interpret these statistically significant results to mean
that DPCM enables the designers to collaborate around their processes more effectively within
the project team.
The teams with DPCM also collaborated more efficiently. The three teams employing
DPCM completed four accurate classroom designs. The four teams without DPCM completed
zero accurate designs. Both sets of teams worked for about one hour uploading and
downloading files, discussing designs, etc., but the teams without DPCM never completed an
127
Chapter 4
Reid Robert Senescu
accurate calculation of their classroom’s NPV based on analysis performed in other tools. This
result is analogous to a professional project team working for three months performing
analysis and discussing designs, but never actually producing drawings that a construction
team could use to build the design. The collaboration was inefficient, because it did not
produce a design.
Two other results support the conclusion that teams with DPCM collaborated more
efficiently. First, teams without DPCM inconsistently transferred information from one tool to
another five times more frequently (p~0). The authors conclude DPCM enabled design teams
to work more consistently with each other. Consistent with this result, designers without
DPCM instant messaged each other expressions of confusion with three times more frequency
(p=0.12). Frequently, these expressions of confusion were followed by rework. Both this
confusion and the related information inconsistency provide additional evidence that teams
collaborate more efficiently when employing DPCM.
4.1.3
Using processes effectively and efficiently to Share between teams
Teams looked at previous projects’ processes to choose tools to use or they chose tools from a
separate frame containing all 20 tools. Effective use of shared processes should result in the
selection of better processes to mimic with DPCM than without. Every time a team attempted
a new NPV calculation (i.e., an attempted global iteration); the authors calculated the score of
the processes they mimicked based on the historic processes. The authors normalized the
scores, so exclusive use of the best historic process resulted in a score of one. The
experimental teams on average chose precedent projects that scored 17% higher than the
control groups. The variation of the experimental teams’ was 0.17 versus 0.06 for the control.
Thus, the teams with DPCM more consistently retrieved processes with higher scores
(p=0.03).
128
Chapter 4
Reid Robert Senescu
Once the teams chose precedent projects to mimic, they mixed and matched them and
used them for their classroom design process more efficiently. For example, 20% of the time a
team without DPCM used a tool, the tool performed a redundant calculation to a tool that the
team had already uploaded. Teams with DPCM used redundant tools just 6% of the time,
suggesting that DPCM made the processes from previous projects more transparent, so they
did not waste time on tools they did not need. Also, teams with DPCM never missed a tool nor
used a tool that was not interoperable with another tool. Teams without DPCM missed eight
tools that were necessary for completing a design. For example, one team without DPCM
never performed an energy analysis, suggesting that they had trouble learning from previous
projects, which always included some type of energy analysis tool. 5% of the time, teams
without DPCM chose tools not interoperable with another tool they had chosen. Choosing an
inappropriate tool or missing a tool altogether is a process mistake that detrimentally impacted
the efficiency with which the team used historic processes. Teams with DPCM made six times
fewer process mistakes per tool used (p~0), which the authors interpret as evidence that
DPCM enables teams to efficiently use processes shared between teams.
129
Chapter 4
Reid Robert Senescu
Table 4-1. Summary of results from the Mock-Simulation Design Charrette.
Process Communication Metrics
No DPCM
p
DPCM
Capturing Processes
Effectiveness
Efficiency
n = # of true dependencies
151
132
% of dependencies captured
0%
93%
n = # of people
20
15
# of files uploaded per person
4.6
4.8
-
# of local iterations per person
26
43
0.02
# of statements about trends per person
3.3
4.5
0.17
4
3
16.3
22.7
n = number of people
20
15
# of local iterations per person
27
43
n = completed global iterations
8
8
# of complete & accurate designs
0
4
367
277
21%
4%
20
15
0.45
0.13
n = attempted global iterations
11
9
mean process score of projects
mimicked
0.63
0.74
n = # of files uploaded
92
72
# of redundant tools used
18
4
-
# of missing tools
8
0
-
# of non-interoperable tools used
5
0
-
31
4
~0
-
Using Processes
Collaboration within projects
n = number of teams
Effectiveness
Efficiency
# of statements about trends per team
n = total number of variables
transferred
% of inconsistent variables
n = number of people
# of statements of confusion per person
0.06
0.02
0.08
~0
0.12
Sharing processes between projects
Effectiveness
Efficiency
# of total process decision mistakes
0.03
Understanding processes across multiple projects
See Figure 4-5 and Figure 4-6 and related discussion in Section 4.1.4.
130
Chapter 4
4.1.4
Reid Robert Senescu
Using processes effectively and efficiently across projects
During the ten-week Multi-Disciplinary Design and Analysis class, 32 students on eight
project teams used PIP to upload 1222 files, and they downloaded files 1939 times (an average
of 60 downloads per designer). They drew 2057 arrows to capture the dependencies between
the files they created. This usage data provides evidence that students used PIP as a primary
file sharing tool on their projects.
DPCM enables effective and efficient understanding of design processes across the
eight projects. Unlike the quantitative evidence provided for collaboration and sharing, this
section provides qualitative evidence by demonstrating that employing DPCM provides
insights into design processes. DPCM enables teams to visualize the degree of information
distribution across a team (Figure 4-5). The darkness of a square in Figure 4-5 (see legend)
indicates the number of arrows drawn between information created by e.g., Jones and
information created by e.g., Smith. The relatively dark diagonal suggests that designers most
frequently depend on information created by themselves. However, some teams depend on
information distributed evenly across the team (top left) whereas others have information that
is more fragmented or concentrated (bottom right). The former suggests that the teams in the
top left have integrated their designs, because generally the designers created information
dependent on other designers. The latter (bottom right) suggest the potential for fragmented
designs, because designers created much information independent of information created by
others on the team. It is important to interpret this graph as indicating only the potential of
integration problems, because it is possible that the difference between the top left and bottom
right reflects the teams on the bottom right prefer communicating verbally, using paper
documents or that the tools they used or projects they work on did not require as much
integration.
131
Chapter 4
Reid Robert Senescu
DPCM also enables understanding of how information flows across multiple projects.
For example, three project teams worked on different train stations along the same rail line.
Figure 4-5 shows that one primary liaison exchanged information between these teams.
Though not shown in the figure, DPCM enables the overlay of average information latency
between people where information latency is the difference in time between uploads of
dependent files. Graphing information latency revealed that Metro Station 3 used information
from Metro Station 1, but that the Metro Station 1 team then updated that information. The
Metro Station 3 team never used that updated information. Again, this situation just reveals a
potential problem. Still professional designers frequently create spreadsheets that are then
used on subsequent projects. Years later, construction of the building may reveal an error in a
design calculation that has since propagated to other projects. DPCM enables understanding of
the process by which this information propagates between projects.
As the teams are sorted by project value (normalized final presentation grades
provided a proxy for project value), a trend exists between the degree of information
distribution and the project value delivered. The authors make no claims about the statistical
correlation between higher value and more distributed information in PIP. Rather, Figure 4-5
provides evidence for the power of DPCM to enable managers to answer questions such as:
Does the amount of information distribution across projects correlate with client satisfaction,
project profit, or change orders during the construction phase? Project managers can use
Figure 4-5 as a live project dashboard enabling interpretation of large quantities of blank
squares as potential integration problems. Employing DPCM thus provides effective
understanding of design integration across projects.
132
Chapter 4
Reid Robert Senescu
Figure 4-5. Information distribution across projects in a multi-disciplinary design and analysis
class. The shade of each square represents the number of arrows (i.e., dependencies) from
information created by a designer on the top (input) to information created by a designer on
the left (output). Designer names are hidden. A strong single-band diagonal exists from top
left to bottom right because designers depend mostly on information created by themselves. A
wide-band diagonal exists because most designers only depend on information from within
their own team. One exception is the three teams working on different train stations along the
same metro line. The projects are ordered from highest to lowest project value, suggesting that
teams with information distributed across the team delivered more value (top left) than teams
with fragmented and concentrated information (bottom right).
AEC companies consider IT investment to be costly and risky, yet investments
proceed based on “gut feel” without understanding current processes and how the specific
investment will improve them (Marsh and Flanagan 2000). The AEC industry wastes $138
133
Chapter 4
Reid Robert Senescu
billion annually due to poor interoperability, but few companies have the tools to understand
their detailed inefficiency problems (Young et al. 2007). Just as Figure 4-5 shows the
frequency of information flow between people, the shade of the squares in Figure 4-6 shows
the frequency of information flow between tools. A global building design firm uses over 200
tools to assess AEC project impact on the environment, so understanding the best tools to use
together is not trivial (Ayaz 2009). From Figure 4-6, a manager can see that across the eight
projects, designers frequently used an AutoCAD file to create a 3D Revit model and on
average about 20 days passed between the uploading of the AutoCAD file and the uploading
of the Revit file. This square represents the process of students frequently acquiring 2D plans
from their professional project mentors, and then, building up 3D models in Revit that they
could feed to other analysis. As this process was frequent and time consuming, a manager
would want to invest in improving this process, perhaps by creating a program that
automatically created 3D models from 2D plans. In practice, a manager could immediately
decide against process investments involving information flows that are not time consuming
or infrequent (small, light squares) and immediately focus attention on improving the
information flows of common, time consuming processes (large, dark squares). Thus, DPCM
enables effective understanding of the latency and frequency of information flows between
tools across multiple projects.
134
Chapter 4
Reid Robert Senescu
Figure 4-6. Frequency and upload latency of information flow between different tools across
all projects in a multi-disciplinary design class. The shade of each square represents the
number of arrows (i.e., dependencies) from information created by a tool on the top (input) to
information created by a tool on the left (output). The size of the square represents the average
latency between when an input file is uploaded and an output file is uploaded. The
visualization enables a manager to understand potentially inefficient information flows, so that
he can invest in improved processes. For example, the large dark squares represent
information flows that are both frequent and potentially time consuming.
135
Chapter 4
5
Discussion of Power and Generality
5.1
Internal Validity
Reid Robert Senescu
Internal validity is the “extent to which the structure of a research design enables us to draw
unambiguous conclusions from our results” (Vaus 2001). The ability to conclude that the
research design validates DPCM is contingent on an acceptance that PIP accurately models the
abstract DPCM. The features in PIP map directly to DPCM, but it is possible other researchers
could develop different technical instantiations of DPCM. As discussed more thoroughly in
Senescu et al. (2011b), the authors mapped DPCM to PIP using the Agile Software
Development (Cohn 2004). After PIP became usable, the authors shifted away from the Agile
Development Method toward the Ethnographic-Action Method (Hartmann et al. 2009). For
example, at first the Node element in DPCM included an attribute containing an image of the
information referenced by the node, but this feature was never requested by students, so the
authors iterated and removed this attribute from DPCM. Also, students did not insist on
automation between nodes, and so the Computable characteristic was deprioritized. Students
did require the notion of a “home frame” where students could personalize their views onto
different project frames. The authors iterated to find that this notion was consistent with HCI
literature emphasizing the importance of personalizing visualizations of information and so,
the attribute is included in DPCM. This iterative research process bound the abstract
methodology (DPCM) with the usable technological model (PIP) and ensured that test results
from PIP apply to DPCM.
This PIP-DPCM coupling enables overall conclusions to be made about DPCM, but
not granular conclusions about the relative importance of individual Characteristics, elements,
and methods. This paper does not provide evidence that PIP is transparent, social, scalable,
embedded, shared, computable, and modular, nor that the elements and methods enable these
136
Chapter 4
Reid Robert Senescu
Characteristics (see Senescu et al. (2011b)). Rather, this paper provides evidence that
collectively, elements and methods aimed at enabling these Characteristics results in effective
and efficient communication. A more exhaustive literature review and application of the
Ethnographic-Action method to practice would likely reveal more Characteristics, elements,
and methods that may result in even greater effectiveness and efficiency. And similarly, it is
possible DPCM includes some components that contribute only marginally to effectiveness
and efficiency. For example, few students utilized the method for searching dependencies, so
the validation provides no evidence that this particular method is necessary for process
communication. However, Senescu et al. (2011b) provides theoretical justification for DPCM
in its entirety, and this paper validates that DPCM does enable effective and efficient
communication – a theoretical contribution to PIM and DPM and a practical contribution to
industry.
Another potential limitation to the power of the results lie in the internal validity of
the charrette method due to the limitations in eliminating “demand characteristics” –
indications to participants about the researchers’ hypothesis. It is possible that Control and
Experimental teams knew of each other and consequently tried to act like “good subjects”
(Goodwin 2009). Previous attempts to change the charrette to a within-subject design failed
both because participants were unwilling to design the classroom twice and it was difficult to
nullify the impacts of learning. Though qualitative and subjective, observations of chats and
the general morale of the teams suggested that the challenge of the design task and incentive
to win the prize were sufficient to ameliorate the desire on the part of the students to appease
the authors’ expected outcome. This conclusion is consistent with a finding that found that the
desire to perform well when evaluated by peers (it was clear to students that results would be
made public) overpowers the desire to confirm the hypothesis of the researcher (Rosnow et al.
1973).
137
Chapter 4
Reid Robert Senescu
Experimenter bias may also have impacted results. The authors tried to minimize their
impacts on the designers’ performance by using a scripted presentation for each group.
Furthermore, many of the results had p-values below 0.05 combined with differences between
control and experiment of greater than 50%. It is unlikely that individuals desire to appease
the researcher or experimenter bias could account for such dramatic differences. A claim for
internal validity is further enhanced by the repeatability of the charrettes.1
5.2
External Validity
External validity is the “extent to which results from a study can be generalized beyond the
particular study” (Vaus 2001). Two main features of the validation method limit the extent to
which conclusions can be generalized. First, the charrettes and the class projects are merely
models for the complexity of real project environments. Second, the participants are students
as opposed to professionals. In both cases, the emphasis on coordination work reduces the
limitations of the validation methods. While the production tasks performed individually differ
greatly from practice, the patterns of information exchange in the class and in the charrette do
not appear to differ from industry projects. Furthermore, a similar application of the charrette
method by Clevenger (2010) showed that the experience and skills of professionals did not
influence their ability to assess the relative impact of different variables on energy
performance. In fact, the professionals in that study could not identify important variables for
energy performance with any more accuracy than random guessing. Thus, it is not surprising
that the authors found that improved collaboration and sharing did not lead to a building with
higher NPV’s. This lack of correlation between process effectiveness and product outcome is
consistent with the other findings from other researchers’ charrettes. For example, when asked
1
Researchers can visit http://www.cafecollab.com, register, and view the class project and
charrette results and download the charrette setup and repeat the experiments presented in this
paper.
138
Chapter 4
Reid Robert Senescu
to select the type of variables (e.g., window area, building geometry, etc.) with the greatest
impact on energy efficiency, professional responses approached random guessing immediately
after they chose the most impactful variable (Clevenger 2010). Similarly, a charrette involving
structural designs found that student participants’ ability to pick optimal designs drops from
96% of optimal when deciding between two variables to 76% for six variables (Flager 2011).
The MSDC required decisions for several dozen variables (the amount varied with the process
chosen), suggesting that any charrette results showing near optimal solutions were probably
due to randomness and not skill. This suggestion does not imply that professional designers
make random design decisions, but that the charrette isolates coordination work from
production work, mitigating the relevance of design decisions. The charrette intends to
measure how designers exchange information, not the quality of the information they
exchange. Thus, the authors only claim that DPCM enables effective process communication,
not necessarily improved product outcomes.
While the authors did not specifically measure collaboration in the student class
projects, the students clearly adopted PIP for collaboration. This adoption demonstrates that
the methodology could be applied in practice, even if the authors cannot claim that
professional projects would see the same dramatic differences between conventional
information management methods and DPCM. The class projects do not demonstrate adoption
of DPCM for sharing across projects. Also, it is beyond the scope of this proposal to measure
the impact of process understanding on investment decisions on process improvement.
5.3
Future Work
Despite the qualifications attached to the external validity, the power of the findings presents a
strong case for future work to make more generalized conclusions. The repeatability of the
charrettes enables future researchers to compare DPCM with other DPM methods while the
139
Chapter 4
Reid Robert Senescu
accessibility of PIP enables its application to future industry case studies to compare with
other PIM research.
This research provides two additional opportunities. First, the application of DPCM to
AEC education and research may enable students and researchers to learn more from each
other when using PIP. Second, DPCM enables the replacement or supplementation of
ethnographic research methods used to study the process of design. Just as social and
professional networking sites have revolutionized the ability of organizational scientists to
apply social network analysis to modern communities, a tool such as PIP enables design
researchers to capture the social interactions and information relationships and apply social
network analysis algorithms to this data. This paper presents a tiny fraction of the results from
the 120,000 actions recorded by 387 different user names in PIP between April 2009 and April
2011. Much opportunity exists to use this data to study how teams exchange information.
6
Conclusion
Architecture, Engineering, and Construction designers (1) struggle to collaborate within
projects, (2) share better processes between projects, and (3) understand processes across
projects to strategically invest in improvement. Overcoming each challenge requires
communication of design processes. The Design Process Communication Methodology
(DPCM) enables effective and efficient design process communication. To test DPCM, the
research maps the methodology to software features in the Process Integration Platform (PIP).
PIP is a web tool enabling project teams to organize and share files as nodes in an information
dependency map that emerges as they work. This paper contributed a set of metrics and an
accompanying test method to measure the impact of DPCM on the effectiveness and
efficiency of four steps required for process communication: (1) Capturing, (2) Structuring,
(3) Retrieving, and (4) Using processes. The authors measure effectiveness and efficiency
140
Chapter 4
Reid Robert Senescu
using PIP and contrast these measurements with results from a conventional information
management method that does not show the dependencies between information.
Using PIP in the Mock-Simulation Design Charrette (MSDC), the authors conclude
that DPCM captures and structures processes effectively, because the student designers
capture 93% of the true information dependencies in the controlled design experiment. The
capturing and structuring is efficient, because it places no measurable burden on the design
teams
DPCM had a statistically significant impact on the number of iterations performed by
designers and the frequency of discussion of multi-disciplinary trends. The design teams
without DPCM did not complete any accurate designs, whereas teams with DPCM completed
four accurate designs. These results provide evidence for the effectiveness and efficiency with
which DPCM enables collaboration within teams.
To select a process, the student design teams viewed the information created by
historic projects. When viewing historic projects employing DPCM, teams selected better
projects to mimic. Once they selected a project, teams with DPCM used the newly shared
processes more efficiently, because they committed few process mistakes. Teams with DPCM
shared processes between teams effectively and efficiently.
DPCM enables the understanding of processes across projects, which provide insights
into: the relationship between information distribution among designers and project
performance; and opportunities for investment in improved information flows.
These results demonstrate the power of DPCM to effectively and efficiently
communicate design processes within projects, between projects, and across projects. DPCM
contributes to filling a gap between two research fields: (1) Project information management
research enables the efficient exchange of information, but does not effectively communicate
141
Chapter 4
Reid Robert Senescu
process; (2) Design process management research effectively communicates processes, but
with methods too inefficient to be adopted in practice. DPCM lays the foundation for
commercial software that shifts focus away from incremental and fragmented process
improvement toward a platform that nurtures emergence of (1) improved multi-disciplinary
collaboration, (2) process knowledge sharing, and (3) innovation-enabling understanding of
existing processes.
7
References
Ayaz, E. (2009). "Global tools gathering and dissemination: A methodology for establishing
best practice for large design and engineering firms." Engineering Sustainability.
Pittsburgh, PA, April 19-21.
Bachman, L. R. (2008). "Architecture and the four encounters with complexity." Architectural
Engineering and Design Management, 4(1), 15-30.
Benkler, Y. (2002). "Coase's penguin, or, Linux and the nature of the firm." Yale Law Journal,
112(3), 367-445.
Beylier, C., Pourroy, F., Villeneuve, F., and Mille, A. (2009). "A collaboration-centred
approach to manage engineering knowledge: A case study of an engineering SME."
Journal of Engineering Design, 20(6), 523-542.
Blessing, L. T. M., and Chakrabarti, A. (2009). DRM, a design research methodology,
Springer-Verlag, London, UK.
Carrillo, P., and Chinowsky, P. (2006). "Exploiting knowledge management: The engineering
and construction perspective." Journal of Management in Engineering, 22(1), 2-10.
142
Chapter 4
Reid Robert Senescu
Clarkson, J., and Eckert, C. (2005). Design process improvement: A review of current
practice, Springer-Verlag, London, UK.
Clayton, M., Kunz, J., and Fischer, M. (1998). "The charrette test method." Stanford
University, Center for Integrated Facility Engineering, Technical Report 120.
Clevenger, C., and Haymaker, J. (2009). "Framework and metrics for assessing the guidance
of design processes." 17th International Conference on Engineering Design, Stanford,
CA.
Clevenger, C. M. (2010). "Design guidance: Assessing process challenge, strategy, and
exploration," PhD Dissertation, Stanford University, Stanford, CA.
Cohn, M. (2004). User stories applied: For agile software development, Addison Wesley
Longman Publishing Co., Redwood City, CA.
Cross, N., and Roozenburg, N. (1992). "Modelling the design process in engineering and in
architecture." Journal of Engineering Design, 3(4), 325-337.
Flager, F. (2011). "Biopt: A method for shape and member sizing optimization of steel frame
structures," Oral Exam Presentation, Stanford University, Stanford, CA, March 10,
2011.
Flager, F., and Haymaker, J. (2007). "A comparison of multidisciplinary design, analysis and
optimization processes in the building construction and aerospace industries." 24th
International Conference on Information Technology for Construction, Maribor,
Slovenia, 625-630.
Ford, J. D., and Ford, L. W. (1995). "The role of conversations in producing intentional
change in organizations." The Academy of Management Review, 20(3), 541-570.
143
Chapter 4
Reid Robert Senescu
Froese, T., and Han, Z. (2009). "Project information management and complexity in the
construction industry." 26th International Conference Managing IT in Construction,
CIB W078, Istanbul, Turkey, 1-10.
Gallaher, M. P., O'Connor, A. C., Dettbarn, J. L., Jr., and Gilday, L. T. (2004). "Cost analysis
of inadequate interoperability in the U.S. Capital facilities industry." National Institute
of Standards and Technology, GCR 04-967.
Garcia, A. C. B., Kunz, J., Ekstrom, M., and Kiviniemi, A. (2004). "Building a project
ontology with extreme collaboration and virtual design and construction." Advanced
Engineering Informatics, 18(2), 71-83.
Goodwin, C. J. (2009). "Control problems in experimental research." Research in psychology:
Methods and design, C. T. Johnson, ed., John Wiley & Sons, Inc., Hoboken, NJ, 205240.
Hargadon, A., and Sutton, R. I. (2000). "Building an innovation factory." Harvard Business
Review, 78(3), 157-166.
Hartmann, T., Fischer, M., and Haymaker, J. (2009). "Implementing information systems with
project teams using ethnographic-action research." Advanced Engineering
Informatics, 23(1), 57-67.
Heiser, J., and Tversky, B. (2006). "Arrows in comprehending and producing mechanical
diagrams." Cognitive Science, 30(3), 581-592.
Hothorn, T., Hornik, K., Wiel, M. A. v. d., and Zeileis, A. (2010). "Conditional inference
procedures in a permutation test framework." version 1.0-18, <http://cran.rproject.org/web/packages/coin/coin.pdf > (Jan. 8, 2011). 26.
144
Chapter 4
Reid Robert Senescu
Javernick-Will, A., Levitt, R., and Scott, W. R. (2008). "Mobilizing knowledge for
international projects." Proceedings of the 2008 ASCE LEED Conference, CIB Task
Group 64 and ASCE Construction Research Council, Lake Tahoe, CA.
Kohavi, R., Henne, R. M., and Sommerfield, D. (2007). "Practical guide to controlled
experiments on the web: Listen to your customers not to the hippo." Proceedings of
the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data
Mining, Association for Computing Machinery, New York, NY, 959-967.
Kreiner, K. (2002). "Tacit knowledge management: The role of artifacts." Journal of
Knowledge Management, 6(2), 112-123.
Marsh, L., and Flanagan, R. (2000). "Measuring the costs and benefits of information
technology in construction." Engineering, Construction and. Architectural
Management, 7(4), 423-435.
Moran, T. P., and Carroll, J. M. (1996). Design rationale: Concepts, techniques, and use,
Lawrence Erlbaum Associates, Mahwah, NJ.
Navarro, M. (2009). "Some buildings not living up to green label." New York Times, New
York, NY, August 30, A8
Ostergaard, K. J., and Summers, J. D. (2009). "Development of a systematic classification and
taxonomy of collaborative design activities." Journal of Engineering Design, 20(1),
57-81.
Otter, A. F. d., and Prins, M. (2002). "Architectural design management within the digital
design team." Engineering, Construction and Architectural Management, 9(3), 162173.
145
Chapter 4
Reid Robert Senescu
Pracht, W. E. (1986). "GISMO: A visual problem-structuring and knowledge-organization
tool." IEEE Transactions on Systems, Man and Cybernetics, 16(2), 265-270.
R Development Core Team and contributors worldwide. "Fisher's exact test for count data."
The R Stats Package, version 2.12.0, <http://stat.ethz.ch/R-manual/Rpatched/library/stats/html/fisher.test.html>( March 25, 2011).
Rittel, H. W. J., and Webber, M. M. (1973). "Dilemmas in a general theory of planning."
Policy Sciences, 4(2), 155-169.
Rosnow, R. L., Goodstadt, B. E., Suls, J. M., and Gitter, A. G. (1973). "More on the social
psychology of the experiment: When compliance turns to self-defense." Journal of
Personality and Social Psychology, 27(3), 337-343.
Scofield, J. H. (2002). "Early performance of a green academic building." TransactionsAmerican Society of Heating Refrigerating and Air Conditioning Engineers,
American Society for Heating, Refrigerating and Air-Conditioning Engineers, Atlanta,
GA, 108(2), 1214-1232.
Senescu, R., Aranda-Mena, G., and Haymaker, J. (2011a). "Relationships between project
complexity and communication." Stanford University, Center For Integrated Facility
Engineering, Stanford, CA, Technical Report 196.
Senescu, R., Haymaker, J., Meza, S., and Fischer, M. (2011b). "Design process
communication methodology: Improving the efficiency and effectiveness of
collaboration, sharing, and understanding." Stanford University, Center For Integrated
Facility Engineering, Stanford, CA, Technical Report 197.
146
Chapter 4
Reid Robert Senescu
Vaus, D. d. (2001). "Tools for research design." Research design in social research, D. d.
Vaus, ed., Sage Publications Ltd., London, UK, 17-33.
Young, N., Jr., Jones, S.A. and Bernstein, H.M. (2007). "Interoperability in the construction
industry." SmartMarket Report: Design & Construction Intelligence, McGraw Hill
Construction.
147
Appendix A
Reid Robert Senescu
Appendix A: Mock-Simulation Design Charrette Instructions
To ensure uniformity between all the student design teams participating in the charrette, I
followed the following script at the beginning of each charrette:
1. Open PowerPoint to “You Can Start This Now” slide.
2. Sign into PIP.
3. Check that everyone has:
a. Windows
b. Microsoft Excel
c. Firefox ver. 3.5 or higher
4. Check that everyone:
a. Enabled Macros in Excel
b. Enabled Popups in Firefox: Tool, Options, Content, uncheck Block Pop-up
Windows
5. Tell students to sign in to Gmail, so they can use Gchat.
a. Confirm everyone accepted your invitation
b. Include them in one conversation and send test message.
6. Go through slides 1-9
7. Go through each step of the instructions.
8. Note that they can search and navigate via hierarchy
9. Remind them that to see others work they just need to hit refresh
10. Tutorial
a. Show the tools they can choose from
b. Show existing projects and the TeamXBoeing797 project
i. Show how they can see comments on projects
ii. Boeing 727 would be a good project to learn from
c. Choose Boeing727 to follow.
d. Show that there are results from design competition
e. Open up 012-Aerodynamics-01.xls from Boeing 727
f. Change values and save to desktop
g. Upload to pip
h. Show that you can also get tools from Tools folder
i. Goto Tools and get structural file
j. Show that they need to meet D/C ratio
k. Save to desktop, emphasize file name convention with iteration
l. Jump ahead to NPV in Boeing 727 project
m. Show how cost estimator can provide feedback to the rest of the team
n. Explain that they can iterate in big global iterations like in the example, or if
they want, they can iterate between two tools, but they need to number their
iterations in the file name.
11. Slides 13-15. Charrette Instructions and Rules
12. Kickoff meeting
148
Appendix A
Reid Robert Senescu
I presented the following slides to each team:
149
Appendix A
Reid Robert Senescu
150
Appendix A
Reid Robert Senescu
151
Appendix A
Reid Robert Senescu
After the kickoff meeting, the students returned to their individual computers and began
designing the classroom.
152