Copyright
by
Andrew Holloman Harthcock
2001
Re-Engineering The Engineering Department
by
Andrew Holloman Harthcock, B.S., M.C.E.
Report
Presented to the Faculty of the Graduate School of
The University of Texas at Austin
in Partial Fulfillment
of the Requirements
for the Degree of
Master of Science in Engineering
The University of Texas at Austin
December 2001
Re-Engineering The Engineering Department
Approved by
Supervising Committee:
Dedication
To Cindy
If patience is a virtue, she must be a saint.
Abstract
Re-Engineering The Engineering Department
by
Andrew Holloman Harthcock M.S.E.
The University of Texas at Austin, 2001
Supervisor: Anthony Ambler
The typical engineering organization must operate in the face of extreme
pressures placed on it to perform at ever increasing levels.
“Faster-Better-
Cheaper” is a phrase that has grown tiresome in its repetition, yet the reality
exists. As design cycles continue to be compressed from an ever widening sphere
of competition, the organization that can not consistently find more efficient
methods of operation faces extinction. Often however, the drive to optimize one
characteristic is at the expense of the others.
Oddly enough, there are
organizations enhancing all three simultaneously. This report examines methods
used by such companies and suggests a generalized process for transforming the
Engineering department into a more efficient business unit.
v
Table of Contents
List of Tables................................................................................................. viii
List of Illustrations .......................................................................................... ix
INTRODUCTION
1
IDEALS IN ENGINEERING
2
Background in Literature ................................................................................. 2
Faster! Better! Cheaper! ................................................................................. 3
Who Is The Customer?..................................................................................... 4
Who Are We? ................................................................................................... 7
How Do We Work?.......................................................................................... 8
Ideals Reprise ................................................................................................... 8
DIAGNOSING ENGINEERING PATHOLOGIES
10
Background in Literature ............................................................................... 10
Commonly Occurring AntiPatterns ............................................................... 11
Analysis Paralysis ................................................................................... 11
Viewgraph Engineering .......................................................................... 11
Death By Planning .................................................................................. 12
Corncob ................................................................................................... 12
Irrational Management ............................................................................ 13
vi
Summary ........................................................................................................ 14
BEST-PRACTICES IN ENGINEERING
15
Background in Literature ............................................................................... 15
A Process For New Products.......................................................................... 16
Team Structure ............................................................................................... 19
Agile Development ........................................................................................ 25
Extreme Programming ................................................................................... 29
Best-Practice Summary .................................................................................. 31
CONCLUSIONS
32
Recommendations .......................................................................................... 33
Product Planning (Who Is The Customer?) ........................................... 33
Team / Process Design (Who Are We?) ................................................ 33
Project Planning & Execution (How Do We Work?) ............................ 34
Further Enquiry .............................................................................................. 34
REFERENCES
36
VITA
37
vii
List of Tables
Table 1 – Demands on Engineering ........................................................................ 3
Table 2 - Success Factors [Wheelwright, pg 14] ................................................................ 9
Table 3 - The 12 Agile Principals [Cockburn, pg 169-173]............................................... 26
viii
List of Illustrations
Illustration 1 – Alignment Dimensions [Labovitz, pg 36]................................................ 2
Illustration 2 – The Development Funnel [Wheelwright, pg 124] .................................... 17
Illustration 3 – Functional Team Structure[Wheelwright, pg 191] ................................... 21
Illustration 4 – Lightweight Team Structure[Wheelwright, pg 191] ................................. 22
Illustration 5 – Heavyweight Team Structure[Wheelwright, pg 191] ............................... 23
Illustration 6 – Autonomous Team Structure[Wheelwright, pg 191]................................ 24
ix
INTRODUCTION
The business model in most industries grew in scope over the last 50 years
from local to regional to international.
This double-edged sword not only
revealed incredible new market opportunities, it enabled competition from
directions never before imagined.
Unfortunately, being “blind-sided” by a
revolutionary new technology or competitor is no longer just a dim specter - it is
standard occurrence in the high-tech fields. The business response? “Faster!
Better! Cheaper!” Not only are we expected to reduce time-to-market by factors
of 2X or even 10X, with exactly the desired product, we must increase the quality
of goods produced and reduce cost to the end consumer. This is daunting if not
downright contradictory at face value, but companies do it – and win. Case
studies show many examples of companies that have optimized all three
requirements, even simultaneously. So what is the trick, and where do we start?
To answer this, I will examine the problem from three directions. First, what is
the ideal? Knowing this will give us the direction to move in. Second, what are
the bad habits or behaviors that are preventing us from competing effectively?
Just as people do not always do the best thing for themselves, companies
frequently engage in pathological behaviors that undermine corporate goals.
Third, what are the most significant best-practices companies are using to win?
We need to learn by example.
I begin with the “idealized” engineering
department.
1
IDEALS IN ENGINEERING
Background in Literature
For “defining ideals”, the single most influential book to this report is The
Power of Alignment by Labovitz and Rosansky. In this work, we are given the
concept of a two-dimensional matrix, as indicated in Illustration #1. The vertical
axis extends from Strategy to People – thus indicating the connection between
policies enacted by top management and the rank-and-file. The ideal in this axis
is to energize and engage the natural creativity at all levels of the organization via
proper upper-management strategies. The horizontal axis extends from Processes
to Customers.
Here we see the connection between corporate methods and
customer satisfaction. This ideal, termed Horizontal Alignment, is one I will
address further in this report.
Illustration 1 – Alignment Dimensions [Labovitz, pg 36]
Strategy
Processes
Customers
People
2
Faster! Better! Cheaper!
What is implied by these three demands? Let us restate them a little more
rigorously in Table 1.
Table 1 – Demands on Engineering
1. Faster - Reduce development cycle time.
2. Better - Enhance customer satisfaction via more targeted functionality
and decreased failure rate.
3. Cheaper - Reduce total cost of ownership in the product.
The benefits of a “Faster” cycle are intuitively obvious. An example of
hard data on this, however would come from an analysis of the release of new
stereo products in the years 1985-1990 by two competitors.
Assuming that
release at the same time as the competition gave ‘x’ profits, it was found that
releasing 6 months late allowed barely a break-even proposition. Releasing just 6
months early, however garnered 3x profits over the product’s lifetime.[Wheelwright p22]
And what is the value of “Better”, or enhanced customer satisfaction?
Ken Blanchard says to “Create Raving Fans; satisfied customers are not good
enough.” Not only do these supremely satisfied customers buy more product,
they essentially become part of the sales force.[Blanchard p-37]
Finally, we are asked for “Cheaper” - a reduction in total cost of
ownership.
Certainly, this includes the initial cost of the product.
But
maintenance, training, and decommissioning costs, for example, often far
3
outweigh the initial investment.
Here a better understand of the customer’s
context provides the necessary knowledge to compete beyond the basic cost of the
product.
In summary, look at Table 1 again. Inherent in point 1 is a knowledge of
what is required – and only the Customer can tell us that. Point 2 is explicitly
Customer centric. And point 3 refers to the Customer’s total cost of ownership.
Thus, we can draw a strong correlation between the customer and how we work.
Inherent in all 3 points is the fact that we are responsible for doing something
better. So the answer to Faster-Better-Cheaper appears to be an intertwined
collection of issues surrounding the customer and our processes as well as
ourselves. Perhaps a simplified model can now be constructed by asking: 1) Who
is the Customer, 2) Who are we, and 3) How do we work? I will address each of
these in turn.
Who Is The Customer?
Bringing the customer inside the development cycle is the point that
Labovitz makes on Horizontal Alignment in the following passage.
Horizontally aligned companies are easy to identify. They are so
“hardwired” to customer requirement that the needs of their customers
resonate with personnel and influence the company’s strategy, processes
and behavior. These companies

Have clear and explicit methods for gathering market data and
disseminating it through the organization.

Link customer needs to their core processes for delivering goods
and services.
4

Base every improvement on changing customer requirements.

Use the customer as the ultimate arbiter of how well they are
doing. [Labovitz, p-109]
Labovitz states that we need to accomplish three things: determine what
the customer most wants, create a shared vision of that need inside the
organization, and understand the needs of the customer’s customer.[Labovitz, pg 109]
These may seem obvious, and in fact, the ideas have been around for 30
years or more.
Such efforts as Total Quality Management (TQM) and Re-
engineering have attempted, with marginal success, to address them. Again,
Labovitz describes the problem:
Most TQM companies left the customer voice outside the processes of the
organization. The job of listening to and interpreting that external voice
fell to that small number of people with direct customer contact: sales and
customer service personnel , and market researchers. Everyone else heard
the customer voice indirectly or not at all.[Labovitz, p-115]
Determining what the customer most wants may be particularly difficult if
it is not clear in the customer’s mind what that is. In the extreme case of B2B
(Business To Business) technologies, our clients are themselves desperately trying
to understand their own customer’s needs.
To better understand these
opportunities for enhancing this flow of knowledge, let us look at the points
where we “touch” our customer. The whole process can be divided into pre-sale,
delivery, and post-sale periods.[Labovitz,
pg 122]
Delivery is not typically an
engineering function and is thus outside the scope of this work. The other two do
have an impact on Engineering and will be discussed further.
5
Pre-sale is the most fruitful in terms of new feature requirements. It is this
point that we can learn what creates value for the customer, and what it will take
to “delight” them. The reason for the loss of a sale, in particular, must be
preserved. Funneling this information back into Engineering, as well as the
remainder of the organization, should be a basic process of the company. Regular
debrief sessions with sales and applications engineers keeps Engineering abreast
of quickly changing market conditions.
The most common scenario during post-sale is an interaction with
Customer Support.
Here, the well-run service department serves as a fire-wall
between Engineering and the customer as basic problems and questions are
resolved without defocusing Engineering. A good goal, but Engineering suffers
from a lack of customer feedback. We see here also that a regular status session,
where Service provides Engineering with a data-reduced summary of issues, links
the designers more closely to the market.
Both of the recommended interactions of Sales-Engineering, and ServiceEngineering fulfill some of the basic needs for Faster-Better-Cheaper. They allow
us to hear what the customer most wants. They create a shared vision of that need
inside the organization.
And they allow us to understand the needs of the
customer’s customer. Now that we know our customer, it is time we understood
some soft-science issues in our own department.
6
Who Are We?
Surely now, if we create a team with the right talent, buy the right tools,
and impose strict development standards, project success is a certainty.
Of
course, this type of managerial naiveté, has provided us a history of failed
projects, unsatisfied customers, and burned out engineers. Once the project has
been identified, vetted, and funded, what are the critical success factors, and what
are the optimal techniques to use?
The following is a hint from Alistair
Cockburn:
Over time, interviewing project teams, I found no interesting correlation
between project successes and process, language, or tools…. I reversed
our assumptions and found that the opposite correlation holds: purely
people factors predict project trajectories quite well, overriding choice of
process or technology…. People issues are the primary issues, technology
and process issues are the secondary issues.[Cockburn, p-44]
So what are these “people issues” Cockburn refers to? The same issues
that make a team successful in one instance and a failure in another. Cockburn’s
point is to create an environment that promotes communication, activity, and
innovation; it leverages basic human tendencies toward innovation; and foremost,
it provides psychological safety within which to operate. Currently, one of the
best sources of study on these topics has been the software industry. This is due,
primarily, to the fact that large development groups are common, and the
intangible nature of software forces communication at multiple levels. Two of the
most recent successful concepts are Agile Development and Extreme
Programming. I will cover these in greater depth later in this paper.
7
How Do We Work?
Successfully bringing a new product to market, in a timely fashion, is the
one most essential function of the Engineering department. To understand what is
important to rapid engineering we need to first look at what factors tend to make
the company more successful in new product launches. Table 2 indicates the
factors Wheelwright has linked with success. Examining these, we see a large
degree of overlap in addressing the prior two concerns of Customer and self
awareness. Here is indicated an early and substantive customer focus by the
product development team. Not only is the team aligned externally with the
customer, however. They are aligned internally in a cross-functional problemsolving team as well. This is supported by strong leadership at a level sufficiently
high enough to function across disciplines. A clear picture of the end result is
evident to all and they each understand their role in achieving it. The successful
product development team creates sufficient prototypes, then validates them, prior
to large-scale development. [Wheelwright, pg 15]
Ideals Reprise
Who is the customer? Who are we? How do we work? Three questions
that must be answered to achieve Faster-Better-Cheaper. Increase the flow of
information about the customer throughout the organization through enhanced
knowledge gathering and inter-departmental communication. Understand that
team-based product development progresses as a conversation and loosen the
8
procedural impediments to the free flow of information.
Create rapid
development environments via: a strong understanding of the desired product and
the time-to-market pressures, internal and external integration, and strong
leadership. [Wheelwright, pg 16]
If only it were all that easy. We turn our attention now from the rosy glow
of the idealized world to the reality of the typical engineering department. The
following section examines the dark-side of people in the business world.
Table 2 - Success Factors [Wheelwright, pg 14]
Item
Success Factors of Outstanding Projects
1
Clear objectives and shared understanding of project’s intent
throughout organization; early conflict resolution at low levels
2
Actively anticipating future customer’s needs; providing continuity in
offerings
3
Maintaining strong focus on time-to-market while solving problems
creatively; system view of project concept
4
Testing and validating product and process designs before hard tooling
or commercial production; “design it right the first time”
5
Broad expertise in critical functions, team responsibility, and integrated
problem solving across functions
6
Strong leadership and widespread accountability
9
DIAGNOSING ENGINEERING PATHOLOGIES
Background in Literature
Identifying “erroneous behaviors” has recently been made easier - and
more of a science. In the late 70’s, Christopher Alexander described the use of
“patterns” in architecture and in the planning of towns. This language allowed the
expression of ideas at a level higher than was previously possible and it captured
expert information that previously required a life-time to accumulate. In 1994
Jim Coplien published a paper titled “A Development Process Generative Pattern
Language.”
[Coplien]
This paper, among others, sparked the use of Patterns in the
software industry. As in architecture, these patterns allowed engineers to capture
the essence of problems that were being solved repeatedly. Not only did this save
time during construction, whether in civil or software engineering, the new
language provided a means of conveying much larger ideas accurately between
engineers. [Brown, 1998, pg 10-11]
In the same way that Patterns provide us good ways to design, the question
arose as to the possibility of “inverse patterns”, or bad behaviors, that we should
be able to recognize and avoid. These oft-repeated bad habits became known as
AntiPatterns. As useful as they are to the designer, there are several that have
been identified as pertinent to project management.
AntiPatterns in Project
Management was written by Brown, McCormick, and Thomas. Based on firsthand observations over multiple case studies, this book identifies basic patterns of
pathological management behavior. Now how do we use them? Although full10
featured templates have been developed
[Brown, 2000],
this report provides a
summary of several common problems and their solutions.
Commonly Occurring AntiPatterns
ANALYSIS PARALYSIS
This pattern occurs when a team gets stuck, typically in the analysis phase
of a project. This may be a result of overstating the required level of detail. It
may also arise as a result of the team’s fear of progressing into the design phase.
This pattern also results from a strict use of the old Waterfall development
process wherein the analysis had to be 100% “complete” before proceeding.
The solution is the use of iterative development techniques – a topic to be
covered later in this paper. In this mode, we admit we do not know everything,
but we do know something, so we get that done and increase both our
understanding of the problem and the volume of useful work accomplished.[Brown,
2000, pg 215-218]
VIEWGRAPH ENGINEERING
In this scenario engineers spend excessive amounts of time creating
presentations, documents, drawings, or anything beside the final work (e.g. – the
software itself.) This can happen as a result of management’s failure to purchase
the correct development tools. As a result, engineers spend more time trying to
express ideas rather than in implementation.
11
If money is not a problem, the obvious solution is to buy the right tools.
Since we are commonly lacking that, a general solution is to spend time
developing prototypes instead of viewgraphs. This gives feedback on critical
design decisions and provides proto-solutions for later use in the actual
product.[Brown, 2000, pg 219]
DEATH BY PLANNING
This pattern emerges as a result of irrational emphasis placed on the
project plan. Two basic forms are referred to as the “Glass Case” plan and the
“Detailitis” plan. The Glass Case results when a fine grain schedule is created,
only to be frozen in place with no update. Since projects shift with growing
knowledge over time, the plan gradually diverges from reality until it is a useless
instrument held as a “sacred cow.” The Detailitis plan suffers from excessive
detail.
Providing the illusion of control, it only serves to defocus Engineering
from more valuable development tasks.
The correct level of detail in a project plan is to show only customer
deliverables, basic technology artifacts, and validation milestones. Examples of
appropriate project line items are: a use-case document, a sellable product, and
Test Plan approval. Note that teams can have sub-projects if it helps them plan
their work through low-level issues, but this should NOT be required or visible to
upper management.[Brown, 2000, pg 221-231]
CORNCOB
This pattern underscores the social nature of engineering. A person who
continues to erect roadblocks to the success of a project is referred to as a
12
Corncob. One of the more image-provoking descriptions, this is indicative of a
department that is blocked due to a personnel issue. This person may be a
manager inside or outside the department, or they may be a team-member. The
diverse causes range from egomania, to greed, to technical bigotry.
It is primarily management’s responsibility to recognize and neutralize the
corncob. This can be done simply in some cases by counseling. Other cases may
require removing the person from the department, or if they are an unrelated
manager, by stating clearly what their role is, what it is not, and backing it up with
upper management intervention, if required. [Brown, 2000, pg 235-242]
IRRATIONAL MANAGEMENT
This pattern includes two basic types of management problems: failure to
make a decision, and making too many decisions. Failing to make the decision
puts the team in gridlock, thus preventing progress. Too many decisions, or
micromanagement, causes a continuous upheaval of the project. This is typically
caused by management’s incapability with managing one or more of the people,
process, or technology issues.
In general, management must admit that it has a problem. It must then
seek help from trusted employees or consultants.
[Brown, 2000, pg 245-249]
Frequently,
this is one or more members of management that have to deal with another
manager. An important aspect of the solution is upper-management with the
strength of character to enact change.
13
Summary
As stated in the introduction, the basic model for improvement suggested
herein is to: 1) Define the ideal, 2) Identify bad behaviors, and 3) Emulate bestpractices.
Antipatterns are a tool for Item #2. Since these issues surround
sometimes murky aspects of human behavior, they have no objective
identification. No scheduled review will pinpoint when a person has ceased to be
a constructive team-member and become a corncob, or when a manager has
transitioned from being effectively involved to micromanaging.
To put this
concept in practice, upper and middle management, as well as the team itself,
would be well advised to first familiarize themselves with this material, then
periodically bring up the topic of patterns and antipatterns for discussion to elicit
multiple viewpoints. Honest and frank discussion of interpersonal or process
problems offers the best chance of solving these problems. The point here is that
any of these may cause an otherwise successful project to delay or to fail
altogether.
To free ourselves to emulate the best-practices, we must first
recognize and remove the worst-practices.
14
BEST-PRACTICES IN ENGINEERING
Background in Literature
On a more positive note, several good elaborations of “good behaviors”
are available. One in terms of organizational structure is found in Revolutionizing
Product Development, by Wheelwright and Clark. Here we are presented with an
“evolutionary chart” of team organization. In this, we are given methods for
increasing team integration across functions. Another “classic” is Peopleware –
Productive Projects and Teams. First published in 1987, and later reprinted in
1999, this is a seminal work in the importance of understanding the human side of
engineering.
Finally, a third book is a work in progress to be published this year (2001)
– Agile Software Development by Alistair Cockburn.
This work places the
sometimes religious debate over “heavy” versus “light” processes in perspective.
In this context, a heavy process has more formality and typically more artifacts in
the form of validation check-points and associated documentation. Conversely, a
lighter process has fewer formal reviews and documents. Is one better than the
other?
It depends.
Is the problem one of low-to-medium criticality that is
solvable with two people? Or is it a mission-critical project using 30 engineers?
At a distance, it seems obvious that a process suitable for one is probably not the
ideal for the other. Yet the typical business paradigm is to force-fit both extremes
within one “corporate process.” Although written for the software profession, the
ideas have obvious applicability across engineering disciplines.
15
A Process For New Products
The first practice is in terms of deciding which projects Engineering
should be working on. How would a process support the need to focus the
Engineering department on the products that will generate the greatest return on
investment (ROI), and the greatest customer satisfaction? Wheelwright describes
several methods for controlling the product lifecycle. These are 1) “Survival of
the Fittest”, 2) “All-the-eggs-in-one-basket”, and 3) an idealized “Development
Funnel.”
Survival of the Fittest is typical of the R&D-centric organization. Here,
many ideas are accepted into the process. Then, depending on the resources of
the backing department, and typically no small amount of political maneuvering,
pet projects are shepherded as far as possible in competition with other projects.
Screens are placed to reduce the number of competing ideas, but these often fail
to sufficiently reduce the number of competitors until a release decision is at
hand. The fundamental problem in these cases is a lack of discipline in shutting
down the projects that do not pass the screen. [Wheelwright, pg 118-120]
The All-the-eggs-in-one-basket development process is typical of start-up
organizations. Here, the company has probably been started due to one good idea,
and so little resources remain that the company can little afford to expend energy
on few, if any, other projects. The first problem inherent with this is that many
otherwise good ideas are prematurely screened out. A secondary issue is that a
“one size fits all” mentality develops as good ideas from rejected projects are
incorporated into the few that pass. This frequently causes products either to fail
16
in development under their own weight, or a market rejection due to an unfocused
product. [Wheelwright, pg 120-123]
The third method takes the best from the prior two to synthesize a more
controlled process over three phases. Refer to Illustration 2. Phase one is idea
generation. As with the first method, we want as many ideas to enter the funnel
as possible. This might be done by creating an award system for submitting ideas.
As an example, Motorola has one of the highest patent rates among technology
firms. This is due largely to its well-structured patent review system that provides
gradually increasing awards to individual contributors based on idea merit.
Illustration 2 – The Development Funnel [Wheelwright, pg 124]
Screen 1
Screen 2
Ship
Phase 1
Product / process
idea generation
and concept
development
Phase 2
Phase 3
Detailing of
proposed project
bounds
Rapid, focused
development
17
Once a product/process idea has been described, it faces screen 1. An
important idea here is that screen 1 is not necessarily a go/kill point. This is more
of a “readiness review” for a later go/kill decision. Here, the idea is reviewed for
fit with the company technology and marketing plans, and for suitability as a
development project. A second role for screen 2 is in identifying competing ideas
that may be combined to reduce waste, or in correctly targeting the idea to new
platform development, enhancement, or derivative product development. Finally,
the idea may not pass this screen until it is found to be complete from these
standpoints.
Phase 2 takes a well-formed idea and “projectizes” it. Typically lasting 12 months, this phase places boundaries around the scope of work, the schedule to
be followed, and the resources to be required. Furthermore, it identifies specific
bodies of knowledge required to accomplish the project. The primary purpose of
this phase, however is to create a standardized project description to be used by
senior management in comparing it to other project candidates.
Screen 2 is the managerial go/kill point. Here, the well-formed project
descriptions from Phase 2 are considered vis-à-vis corporate objectives, each
other, and available engineering resources. Any project passing this point has the
guarantee of full funding and staff allocation in exchange for the expectation that
its chances of taking a product successfully to market are high. All information
gathered for this phase becomes the starting point for rapid, focused development
in Phase 3.
[Wheelwright, pg124-127]
Phase three will be the focus of later sections in
this work.
18
In summary, several factors contribute to successful project selection.
First, the entry of the widest possible array of new ideas must be promoted by the
organization. Second, early management guidance prior to major development
allows the idea to germinate fully. Finally, once the idea has been thoroughly
researched and aligned with corporate goals, it is staffed with sufficient resources
for successful execution.
Team Structure
A huge amount of literature exists on the dynamics of team structure.
What has emerged from an examination of organizational structures over the last
years is that there exists somewhat of a hierarchy of organizational capability.
The curious thing is that it does not follow the age of an organization. In fact, our
own striving for more and more rigid structures, along with some good, oldfashioned turf-wars, has often caused a devolution away from more productive
arrangements. As proof, look at the typical pre-IPO organization (at least the
successful ones…) Here, there is a small, flat organization where a top executive
“runs the show” for a product, since typically at this stage, this is all the small
company can afford to fund. Under this manager are all elements required to get
the product to market – marketing, multiple engineering domains, procurement,
and operations. It is tight, efficient, and very productive. Each team-member
knows what team they are on since there’s only one! Compare this to “megalith
corp.” These corporations are typically split into mini-fiefdoms, complete with
multiple lines of self-defense embodied in the processes. People frequently work
19
on multiple projects so there is only a “development cloud” from which there
occasionally emerges a product – almost by accident in some cases. Between
these extremes lie four basic organizational team structures: Functional,
Lightweight , Heavyweight, and Autonomous.
The Functional team structure is commonly found in the largest of
organizations. Refer to Illustration #3. In this, each functional area is managed
by a practice expert (FM) who has project leads (L) reporting to them. This is the
most “organized” from a corporate perspective, but sadly, it is also the least
productive. In this organization, management runs strictly along functional lines.
This makes it quite convenient in that the same person that assigns work also
gives performance reviews. Unfortunately, a product release generally requires
several of these functional areas to interact closely over time – something this
structure does not readily support. What we need is someone responsible for the
product itself.
20
Illustration 3 – Functional Team Structure[Wheelwright, pg 191]
FM
FM
FM
Function
Function
Function
text
L
L
L
team
team
team
Enter the Project Manager (PM) with responsibility for the product’s
successful completion. Refer to Illustration #4. This person exercises foresight
and facilitates across functional boundaries, thus breaking down some of the
previous communication boundaries. Typically a mid-level manager, this person
emerges from one of the functional areas, so they have more control over this area
as indicated in the illustration. This structure is commonly known as “Matrix
Management.” It is better than Functional, but members may now have confusion
over having “two bosses.” In light of this, it is critical that PMs communicate
with FMs to resolve this confusion, and to get the project done.
21
Illustration 4 – Lightweight Team Structure[Wheelwright, pg 191]
PM
FM
FM
FM
Func
Func
Func
L
L
L
team
team
team
The highest “normal” form of organization has been found to be
“Heavyweight Teams” as indicated in Illustration #5. In this structure, the PM is
a higher member of management – typically a Director or VP level.
This
structure removes doubt about “who the boss is” by officially assigning resources
from the separate teams to work under the PM. The PM has direct market access
via marketing personnel, as well as engineering, operations, and any other
disciplines required. This is easily recognized as that efficient structure used by
start-ups. This structure has some of the greatest corporate challenges, however.
It is critical that upper management bound the responsibilities of the team to
prevent it from straying en masse from the desired program. Other potential
problems exist in trading in-depth technical expertise for good generalists
required for this strategy, and not least, the criticality of the Heavyweight Project
22
Manager’s position. This person must be a seasoned manager with a wide array
of skills. Failure to correctly staff this position will doom the project.
FM
FM
FM
Func
Func
Func
text
L
L
L
team
team
team
PM
Market
Illustration 5 – Heavyweight Team Structure[Wheelwright, pg 191]
The final team structure is even more radical. The Autonomous team
structure, displayed in Illustration #6, takes the heavyweight structure to the
extreme of removing the temporarily assigned workers to a location physically
separate from the rest of the corporation.
When staffed, budgeted, and led
correctly these teams are the most efficient in execution of projects.
The
downside is that people in these groups tend to “go native.” That is, they lose
their corporate identity over time. Once the project is finished, there is often a
problem in bringing them back into “normal” corporate life.
defections are frequent after projects of this nature.
23
As a result,
Illustration 6 – Autonomous Team Structure[Wheelwright, pg 191]
FM
FM
FM
Func
Func
Func
L
L
L
team
team
team
PM
Market
text
In summary, we have seen four different approaches to organizing an
engineering department. Which is best? It depends. Departments engaged in
sustaining activities with a high degree of repetitive activities will best be served
by a Functional or Lightweight Team structure. Likewise, departments with a
higher focus on process improvements over inter-departmental communication
will do better with one of these two approaches. On the other hand, departments
engaged in a large percentage of new development, that require a large degree of
coordination between departments, will benefit from the Heavyweight team
24
structure. And finally, for start-ups, or older companies with a demand for a
radically new design may accept the risk of creating the Autonomous Team.
Agile Development
We now focus on the best-practices of process selection and desired dayto-day team behaviors. The Agile movement was formed for finding better ways
to create software through an understanding of precisely these and other,
software-specific, factors. Its “Manifesto” reads as follows:
We are uncovering better ways of writing software by doing it and helping
others do it. Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan. [Cockburn, pg 165]
These concepts are supported by 12 Principals reprinted in Table 3. Many
of these sound obvious, but they are frequently lost sight of. Early and frequent
delivery of software stresses giving the customer pieces of what they want for
their use and Engineering’s feedback. It addresses the very real occurrence of a
deliverable having its requirements changed on receipt by the customer. It allows
rapid fine-tuning. And it provides frequent design-wins as a team morale booster.
[Cockburn, pg 169-170]
25
Table 3 - The 12 Agile Principals [Cockburn, pg 169-173]
1. Our highest priority is to satisfy the customer through early and
frequent delivery of valuable software.
2. Deliver working software frequently, from a couple of weeks to a couple
of months, with a preference to the shorter timescale.
3. Working software is the primary measure of progress.
4. Welcome changing requirements, even late in development. Agile
processes harness change for the customer’s competitive advantage.
5. Business people and developers work together daily throughout the
project.
6. Build projects around motivated individuals. Give them the environment
and support they need, and trust them to get the job done.
7. The most efficient and effective method of conveying information to and
within a development team is face-to-face conversation.
8. The best architectures, requirements, and designs emerge from selforganizing teams.
9. Continuous attention to technical excellence and good design enhances
agility.
10. Agile processes promote sustainable development. The sponsors,
developers, and users should be able to maintain a constant pace
indefinitely.
11. Simplicity – the art of maximizing the amount of work not done – is
essential.
12. At regular intervals, the team reflects on how to become more effective,
then tunes and adjusts its behavior accordingly.
26
Progress is measured by working software. This appears to be one of
those “obvious” issues, but in the software profession, we frequently allow
ourselves to spend copious amount of time navel-gazing at system specifications,
architectures, flow-charts, and more esoteric portions of the actual coding. This
tends to delay the user-identifiable portions, which, in the hands of a capable
programmer, may be acceptable. Unfortunately, it also allows for tremendous
amounts of hand-waving by a less capable engineer as the schedule slips away out
of control.[Cockburn, pg 169-170]
Changing requirements is not necessarily a bad thing, and may be good.
Engineers frequently blame shifting requirements on failure to perform. But the
reality of the world is that things do change. Given a trust relationship developed
with the customer, and the mutual understanding that change costs, it is possible
to derive a competitive edge via change.
If we can change faster than the
competition, we win.[Cockburn, pg 170]
Bring the customer onto the team. Here, the horizontal alignment concept
comes full-force as we design with the customer. Yes, they will disagree with
some things, but that feedback early on is a lot cheaper than getting it later.
Items 6-8 revolve around what the management industry refers to as “job
design factors.” It is incumbent on management to hire the right people, remove
road-blocks, and stay out of the way. Cockburn states that “we would rather see
motivated, skilled people communicating well, and no process at all, than a welldefined process and unmotivated individuals.”[pg 171] Another job design element
is in facilitating face-to-face communication. This is done by interactive process
27
as well as thoughtful physical placement of personnel in the work environment.
One study indicates that people spend 30% of their time in solitary concentration.
The other 70% is spent communicating ideas.
[DeMarco, pg 62]
We should design the
work place to support this dual need. A final job-design note is in the allowance
for emergent team-behavior.
That is, trust the team to select its preferred
operating approach given an understanding of the requirements.
Items 9 and 11 focus on the correctness and resultant quality of the design
aspect. These items stress technical excellence and simplicity. Focusing on
technical excellence brings people back to the point. Good design is easier to use
and grow than poor design. And simple designs are quicker to achieve and easier
to use than more complex ones.
Another management pointer is provided in Item #10 - that Agile
processes should be sustainable. Although immediate goals may be accomplished
by working overtime, the long-term effect of sweat-shop conditions is a net loss in
productivity.
Left un-pressured by management, good people will drive
themselves far harder – to the point that the good manager must now know when
to reign them in.
Finally, Item #12 reminds us to periodically stop, look back, and assess
how we are doing. The team should then determine what modifications to make
in its behavior to improve.
Agile Development is a new term (circa 2001.) Its ideas are collected
from years of experience in the software industry. And as far as they apply to
28
people performing not just software, but design in general, they are directly
applicable to the Engineering department at large.
Extreme Programming
Another example of best-practice, and another product of the software
industry’s search for developmental nirvana is Extreme Programming, or XP.
This methodology is a refreshingly commonsense approach that values the four
ideals of: communication, simplicity, feedback, and courage.
The practices
described are used at different levels in the project. Practices are recommended
both at the project management scope and in the daily routine of programming.
We will cover basics of the PM practices, then look at the daily programming
practice.
Project planning is quite important, and quite distinctive in XP. One key
concept is that there must be several deliverables throughout the project so the
customer can measure progress in a concrete manner. Other benefits are that the
customer gets partial value for a partial investment, and the team derives
psychological benefits from early design wins. To accomplish this, the project is
broken into a series of 2-4 week iterations wherein each iteration provides a small
set of complete functionality. Since each iteration is a microcosm of the overall
effort, each must have the complete life-cycle in it. Using this method, it’s not
feasible to have the customary rigid phases for requirements, analysis, design,
coding, and testing, Instead, they are all performed in parallel over the iteration.
This more organic approach to development allows us to greatly simplify the
29
work and focus our tasks. Finally, it is important to note that two factors must be
considered at the beginning of each iteration to decide which functionality is to be
included. The customer must be heavily involved to provide insight into relative
business value of different items. And the designers must also be involved to
provide insight into where the highest risk factors are. [Beck, 2000, pg 21-23]
Another critical aspect of project management is in task estimation. By
instrumenting the development process, and lumping all the factors together that
can have an effect on team output (e.g. - people, process, tools, environment,
etc…) we can derive meaningful data to predict future work done by the team.
For this, we use the two concepts of Ideal Time and Velocity.
Ideal Time
provides the most convenient task estimation basis for team members. It is much
easier to think in “idealized” terms of getting work done such as Ideal Days. Note
that this has nothing to do with actual time, it simply gives a relative comparison
of volume of work between tasks.
Velocity gives us a calibrated work output,
per iteration, from a person or team. Its units are Ideal Days/Iteration. Thus,
given an Ideal Time estimate, and a team’s velocity, we can provide a relatively
safe estimate on the time required to execute the task. [Beck, 2000, pg 57-62]
Daily practice is another important set of XP principles.
A pair of
programmers work together at one monitor. They read a use-case, talk to the
customer, and examine the existing framework of software to ascertain how best
to include a new function. First they may need to refactor (rewrite) some of the
system fundamentals to correctly allow for inclusion of the new functionality.
Then they develop the software to test the code to be developed. Then they write
30
the code. Only after the new code passes the new test cases does it get included in
the software used by other developers.
[Beck, 1999, pg 7-9]
Best-Practice Summary
We have examined practices for selecting what projects to focus on, how
to organize the entire department, how to select the best process, and the team
habits to foster. The Development Funnel shows us how to collect a broad range
of ideas and cull them down via a controlled organizational process. The multiple
team structures presented allow us to consciously choose what departmental
organization works best for our business. Agile ideas give us general methods for
aligning engineers with customer requirements and producing items of value early
in development. Finally, XP is one of several new processes that, although
probably not useful for a large safety critical application, in smaller, less critical
designs, produces rapid results with a high level of quality. No promises here for
a silver-bullet or “satisfaction guaranteed”. But these are all good tools, based on
successful projects, to put in the engineering toolbox.
31
CONCLUSIONS
Labovitz says to stay centered on the “main thing.” [Labovitz, pg 3] If the main
thing is to improve the Engineering organization, we need to develop and clearly
communicate the ideals of Engineering to the company. Development cycles can
be reduced. Customer satisfaction can be increased. And total cost of ownership
can be reduced. Simultaneously. But it takes strategic effort on the part of the
company. The customer must be brought into the development cycle. We must
focus on the activities that directly support the customer’s needs versus artificial
attempts at controlling the process. And the “soft” issues of team dynamics must
be used to enable and energize those teams from the inside.
We must be wary of maintaining habits counterproductive to our goals.
By learning to recognize these problems, as well as the corrective actions, we will
insure a move toward better engineering practices.
Finally, examples of best-practices are available at multiple levels for the
Engineering organization. At the highest scope, we must foster the freedom to
generate as wide an array of project ideas as possible, then develop, fund, and
execute the selected ones in a disciplined manner. In creating teams for the
selected projects, we have to move away from either assembly-line, or silo-style
engineering. Project teams of dedicated resources under a strong leader will take
a product to market faster than any other team structure. And at all times during
development, the process used must foster development of items with actual value
to the customer.
32
Recommendations
The following is a summary list of action items based on the prior
observations:
PRODUCT PLANNING (WHO IS THE CUSTOMER?)
1.
Create a promotional system to encourage employee submission of
new ideas.
2.
Hold quarterly Product Roadmap sessions with senior management.
In these, use consistent rules to steer potential projects and decide
whether planned projects should be executed.
3.
Hold a quarterly meeting between customers, sales, marketing, and
Engineering to expose engineers to the market.
4.
Hold quarterly meetings between service and Engineering to expose
engineers to actual product usage data.
TEAM / PROCESS DESIGN (WHO ARE WE?)
1.
Form cross-functional product-centric teams, under the leadership of a
strong Project Manager, to take new products to market.
2. Allow flexible selection of process to suit the demands of each
individual project.
3. Consider the concept of “development as conversation” – that is, take
physical proximity into account when estimating project duration.
4. Where possible, put a customer on the development team.
33
PROJECT PLANNING & EXECUTION (HOW DO WE WORK?)
1.
Plan for iterative development where each 2-6 week cycle provides a
user identifiable benefit.
2. Expect requirements to change – but never compress the schedule
when it does occur unless a de-scope has occurred.
3. The best tool for controlling projects is via scope.
Prioritize
requirements so the PM has flexibility in controlling the scope.
4. Plan for a mid-project reassessment of the scope and schedule based
on progress in the first couple of iterations.
5. Hold a quarterly (or as needed) meeting with team leaders and senior
staff to review projects for anti-pattern behaviors.
Further Enquiry
The topics presented here cover a broad range of issues. As such, further
research could be easily performed by extending either depth or breadth of the
material. Of specific interest to most businesses would be 1) more historical case
studies of successful behaviors, and 2) live data from new implementations of
these techniques. Two additional sources of reference material are available that
were not used in this work:
Constantine on Peopleware, Constantine, Yourdon press, 1995.
The Deming Website:
34
http://deming.eng.clemson.edu/pub/den/deming_philosophy.htm
In particular, the change process of inculcating a new behavior into an
existing culture is a typical source of chaos and fear arising from the uncertainty.
Thus, dealing with the side-effects of the introduction of new ideas has itself
become a topic of consideration. An additional resource for the study of this
phenomenon would be Leading Change, Kotter, HBS Press, 1996.
Leading an engineering department can be a maddening balancing-act of
hard business needs, technological speculation, and individual emotions. It can
also be one of the most rewarding professions imaginable. By studying the
factors that prevent us from succeeding, as well as those practices that history has
proven to be successful, we can re-engineer the engineering department.
35
REFERENCES
Beck, Kent. 1999. Extreme Programming Explained Embrace Change. Addison
Wesley.
Beck, Kent, and Fowler, Martin. 2000. Planning Extreme Programming. Addison
Wesley.
Blanchard, Ken. 1999. The Heart Of A Leader. Honor Books.
Brown, W.H., Malveau, R.C., McCormick, H.W., and Mowbray, T.J. 1998.
AntiPatterns – Refactoring Software, Architectures, and Projects in Crisis.
Wiley.
Brown, W. J., McCormick, H. W., and Thomas, S. W. 2000. AntiPatterns in
Project Management. Wiley.
Cockburn, Alistair, 2001. Agile Software Development, Draft version 2b1.
Addison Wesley Longman.
Coplien, James O. 1994. A Development Process Generative Pattern Language.
PLoP.
DeMarco, Tom, and Lister, Timothy, 1999, Peopleware – Productive Projects and
Teams. 2nd ed. Dorset House Publishing.
Dorf, Richard C., 1999. The Technology Management Handbook, Chapter 14.3 Attributes of Successful New Products and Projects. CRC Press.
Labovitz, George, and Rosansky, Victor, 1997. The Power of Alignment. Wiley.
Wheelwright, S. C., and Clark, K. B. 1992. Revolutionizing Product
Development. Free Press.
36
VITA
Andrew Holloman Harthcock was born in Clarksdale, Mississippi on August 22,
1959, the son of Martin Bates Harthcock and Winona Irene Harthcock. After completing
his work at Jackson Preparatory High School in Jackson, Mississippi, in 1977, he
received his Bachelor of Science degree in Computer Science from the University of
Southern Mississippi in 1982. While employed with Motorola, Inc. in Boynton Beach,
Florida, he received a Master of Computer Engineering degree from Florida Atlantic
University in 1986. In following years, he worked for Mercer University in Warner
Robins, GA, and for Trimble Aerospace in Austin, Texas. Mr. Harthcock is currently the
Software Engineering Manager at Datum-Austin, in Pflugerville, Texas. In January of
2000, he entered the Graduate School at The University of Texas.
Permanent address:
2430 Falcon Dr.
Round Rock, TX 78681
This report was typed by the author.
37