Homework 2: RPV Failure, Due Wed 9/9
CSCI 510, Fall 2015
v3 of 9/3
30 points
Section 3.1 of the ICSM book describes the Remotely Piloted Vehicle (RPV) failure story
about its failures with respect to Principle 3, Concurrent Multidiscipline Engineering.
RPV Failure Story and ICSM Principles (18 points)
Determine the RPV failures with respect to the other three principles:
1. Failures with respect to Principle 1, Stakeholder value-based guidance
(stakeholders not involved in determining the requirements and solutions) (6
points: 3 points for first correct answer, 2 points for second correct answer, 1 point
for the third correct answer—you are allowed to submit more than three answers);
Example failures on Principle 1:
 “Multi-service working group of ... RVP controllers” apparently set all requirements;
many stakeholders were left out of this group (one example: network experts).
 Pseudo-concurrent engineering approach did not meet the maintainability needs of
those who would have to operate the system over many years.
 Priorities of the capabilities were not set per stakeholders’ needs and agreement
prior to the PDR – could have worked on the most pertinent capabilities before PDR.
 At PDR (at least), system did not address the needs of those with existing controllers
(battlefield-based, sea-based, or home-country-based).
 At PDR (at least), planned system did not address several important off-nominal
scenarios, thereby failing to meet the needs of controllers.
 Hardware engineers did not consider or discuss with network experts or other
related engineers before defining interfaces.
2. Failures with respect to Principle 2, Incremental commitment and accountability
(project outcomes totally committed to vs. incrementally) (6 points: 3 points for
first correct answer, 2 points for second correct answer, 1 point for the third correct
answer—you are allowed to submit more than three answers);
Example failures on Principle 2:
 After a brief technology demo, the acquirers totally committed to an approach and a
single contract.
 Even after PDR revealed many issues, contract was not restructured to be
incremental.



After PDR, acquirers and development should have set up accountability measures,
and a reward and penalty system for following and not abiding by the measures
previously established and agreed upon.
Regardless of problems, development could originally have planned for incremental
delivery of capability into a partially-working system, instead of just waiting for a
final integration phase; that would have allowed early termination with partial
success.
Eventual 1:1 capability was not known early in the project, resulting in a missed
commitment (ICSM page 89).
3. Failures with respect to Principle 4, Evidence and risk-based decisions (decisions
made without evidence of their feasibility) (6 points: 3 points for first correct
answer, 2 points for second correct answer, 1 point for the third correct answer—
you are allowed to submit more than three answers).
Example failures on Principle 4:
 Lack of evidence that the budget of $1 billion and schedule of 40 months was
sufficient at the beginning of project, much less a bid promising a smaller budget.
 Failed to require evidence for claims at the PDR.
 Failed to require evaluation by independent expert at the PDR.
 Apparently no overall assessment of risks was ever done.
 Even after the PDR, there apparently were no risk management plans.
Decision at PDR (6 points)
4. (6 points) Which would have been a better decision at the end of the Preliminary
Design Review (PDR), in terms of total cost, total schedule, and resulting system (1
point)? Justify your answer (5 points).
 Declare PDR passed; go with concurrent design and development;
 Declare PDR failed; go with Competitive Prototyping approach described in
ICSM book Section 3.2
1 point for choosing the second option. 5 points for using the justification below; give
partial credit for another reasonable justification (even if the first option was chosen),
especially one that’s quantitative. (Note: look kindly at a result presented in a table.)
Justification (taken primarily from first complete paragraph on page 89 of ICSM; I believe
this information is also available in the lecture slides): “PDR passed” choice: total cost = $3
billion; total schedule = 80 months; resulting system has 1:1 (# RPVs : # pilots) capability,
but with reduced functionality. “PDR failed” choice: total cost = $1.2 billion plus the costto-date of the canceled project (likely around $10 million); total schedule = 42 months;
resulting system has 1:1 capability.
Competition can also introduce or bring about creative solutions, and the desire to win
might motivate teams to perform well.
Identifying and Mitigating Risks (6 points)
5. (6 points) What are some risks with respect to the Competitive Prototyping
approach, and what could be done to mitigate them? Provide at least three risks,
with mitigation (2 points for each risk).
For each good risk (list below is not necessarily complete), 1 point, plus an additional 1
point for any plausible mitigation that’s directed at that risk. (I.e., bad risk gets 0.)
(The first three risks are from the second complete paragraph on page 89 of ICSM):
 Risk: Under-investments in prototype evaluation, leading to insufficient data for
good decision making. Example mitigation: Getting strong expertise (either inhouse or through a consulting contract) in the technical risks together with
expertise in Research & Development project planning.
 Risk: Extra expenses in keeping the prototype teams together and productive
during often-overlong evaluation and decision periods. Example mitigation: Have
contracts be for two phases, where the contractor’s second phase overlaps with
evaluation of their first phase, and evaluation of their first phase determines
whether they’ll go on to the third phase.
 Risk: Choosing system developers too much on prototyping brilliance and too little
on ability to systems-engineer and production-engineer the needed products.
Example mitigation: As part of results evaluation, look at the company’s general
ability to produce actual products. Example mitigation: As much as possible,
require delivery of some products that are required to be production quality.
 Risk: per-phase budgets are not perceived to be sufficient by potential bidders,
leading to too few bids. Example mitigation: (covers only Valuation through
Development phases) Have each contractor provide a proposed project plan for the
next phase as part of their work, and evaluate the proposed project plans before
setting a budget for the next phase.
 Risk: All contractors have technical difficulties on a phase (or maybe multiple
phases). Example mitigation: Have expertise available (either in-house or via a
consulting contract) for evaluating technical risks in contractor proposals.
 Risk: Complex or wide variety of requirements may not be explored or known.
Example mitigation: Plan for prototyping and early user feedback in the complete
project plan.
 Risk: Budget and schedule too small and short. Example mitigation: Prioritize and
agree on priority of capabilities and schedules.