University of Southern California
Center for Systems and Software Engineering
COSYSMO 3.0:
Workshop Results
Jim Alstad USC Center for Systems
and Software Engineering
Annual Research Review
March 17, 2016
Jim Alstad*
03/16
310/344-0894
jalstad@usc.edu
1
University of Southern California
Center for Systems and Software Engineering
Workshop Agenda
• Tentative agenda:
13:00 Attendee introductions
13:15 Detailed introduction to the model and to workshop
techniques
13:30 Delphi round 1
15:00 Break
15:30 Delphi round 2
16:30 Closing discussions
17:00 End
03/16
2
University of Southern California
Center for Systems and Software Engineering
Workshop Materials
•
•
•
•
Main presentation (from this morning)
Workshop presentation (this presentation)
Model document
Delphi voting form
03/16
3
University of Southern California
Center for Systems and Software Engineering
Wideband Delphi Procedure
• Ballots available via:
– E-mail (Excel)
– Memory stick (Excel)
– Paper
• Round 1
– Moderator goes over the parameters in the model
– Each voter fills in a ballot with her opinion as to the correct
value of the parameter
– Ballots are anonymous
– No discussion
– When complete, ballots are turned in to moderator
• Later round
– Moderator assembles results of voting and presents to
group
– Moderator leads group discussion of results
– Voters fill in a new ballot with possibly revised parameter
values
03/16
4
University of Southern California
Center for Systems and Software Engineering
Context for this Delphi (1/2)
• Productivity range (“EMR”):
– Ratio of the numerically highest value for a cost driver to its
numerically lowest value
– Is a measure of the impact of the cost driver
– Recommended voter concept: “Considering the project with
the best rating on this parameter to the project with the
worst rating, what is the productivity ratio due to this
parameter?”
• Some commentators have suggested that the EMRs
as a group are too large
– I.e., the product of all EMRs is too large
– Consequence can be that, for example, going from Nominal
to High (one step) causes too big a change in the estimate
03/16
5
University of Southern California
Center for Systems and Software Engineering
Context for this Delphi (2/2)
• One commentator suggests that a low capability
team (Personnel/Team Capability) may also have low
CONOPS and Requirements Understanding and low
Architecture Understanding, leading to an
unwarranted high value for the product of these.
– Currently, product of EMRs = 19.66
03/16
6
University of Southern California
Center for Systems and Software Engineering
Discussion Topix
• Jim/Winsor: With/without Interoperability
– Jim: Nominal should be the same
• Resolved: Jim’s misunderstanding: two “separate” tables
– Winsor: No further discussion needed
• Jim: Somebody lowered all Cost Drivers
– Resolve: We’ve discussed this. Our opinions will go into
Round 2
• Winsor/Hunter: Scale factors: Lower base, increase
RV
– Resolve: Allow scale factors to be +/-, including RV. Allow
for asymmetry.
• Marilee: # Recursive Levels
– Not for group resolution
• Jim: Statement for RV
– Resolve: Use perdcentages
03/16
7