Evaluate SE Methods, Processes and Tools
Technical Task Plan
USC Workshop
Los Angeles, CA
16 March 2009
Agenda
• Introduction to the MPT task and adjustments
• Status of task
• Planned activities for the workshop
2
SOW Language
Look at current SE methods, processes, and tools (MPTs) as they are applied across the
DoD acquisition life cycle focusing on three different development environments:
individual weapons systems, SoS, and network-centric services. Research will be targeted
at improving current/identifying new SE MPTs that will better support the practice of SE
in these three environments. Specifically, this task will:
1. Define critical attributes of current SE MPTs across the weapons system, SoS, and
network-centric services environments;
2. Identify strengths and weaknesses for these current MPTs and any shortcomings in
their application across DoD;
3. Recommend, in priority order, MPTs for further study to innovate or create improved or
new MPTs to eliminate identified shortcomings;
4. Upon selection by the government of MPTs recommended in sub-task 3 for further
study, perform research to innovate or create improved or new MPTs to eliminate
identified shortcomings, thereby advancing the state of practice of SE within the
community; and
5. For the improvements delivered in sub-task 4 above, propose a methodology for
validating the programs.
3
MPT Task Overview (As proposed)
Establish Criteria and
Validate with Sponsor
Apply Selection
Criteria
Identified Raw MPTs
Select MPTs for
Evaluation
HH
Complete Detailed
Attributes
HH
M M
M
HH M
LL
MPT
Sources
DoD Guideboooks
DoD Programs/Reviews
Service Repositories
Defense Industry
Commercial industry
Queue of selected and prioritized MPTs
Describe MPTs
Fully described
MPTs to evaluate
Apply Evaluation
Criteria
Evaluate MPTs
eWorkshop
Evaluated MPTs
Cumulative MPT
Coverage
BPCH
To Users
Reports and
Recommendations
MPT Analysis
Repeat
Recommended MPTs
Improvements Needed
Overall Gap Analysis
Research Areas
4
Initial MPT Attributes
5
Adjustments
• Scope very broad for task resources
• Task sponsor initially interested in their particular
environmental challenges
– Less interested in the ongoing, repetitive evaluation process
proposed
– Looking for ideas outside traditional MPTs
– Broader survey and gap analysis
6
MPT Task Changes
Define and Validate
Sponsor Environment
Challenges and Needs
Conduct interviews
Revise MPT Attribute
Set
Identify and Mine Comparative
Environments for MPTs
Review
sources
provided
in SoW
Review
literature
and web
Broad
survey of
industry
Select MPTs for
Evaluation
Identify Promising
MPTs and Gaps
Conduct open-ended
expert workshops
Initial MPT Candidate
Templates
Describe MPTs
To Users
BPCH
Evaluate MPTs
eWorkshop
Reports and
Recommendations
MPT Analysis
Repeat
7
Task Status
• Define Sponsor Environmental Challenges and Needs
– Developed and refined initial challenge list
• Validating Environmental Challenges and Needs
– Currently interviewing sponsor personnel
• 5 people to date
• Indications are that environment challenge description valid
– Revise/extend template as needed
– Revise evaluation criteria based on needs assessment
• Revised attribute set for one-page candidate identification
template
Italics indicate tasks of the workshop
8
Environmental Challenges
9
Researcher Template
•
•
•
•
Support data collection on existing MPTs
Includes challenge description
Based on subset of MPT attributes
Intended to winnow down to reasonable number of MPTs to
evaluate
• Primarily used with the open sources review (literature, web)
• Initial feedback good; waiting to see results
10
Broad Industry Survey
• To identify potential gaps and gather MPT candidates from
broader community based on environmental challenges
– Commercial industry
– Defense community
– Research community
• Draft survey developed
• Help needed to get survey to right people
– SERC members
– CSSE Members
11
MPT Activities during workshop
• Pilot broad industry survey with ARR members
– Collect and review data
• Discuss environmental challenges
– MPT types (Process/governance vs. technical/tools)
• Discuss gap/MPT identification workshop strategy
12