Assessing Transition of Security
Operations in Afghanistan
Status Report
18 March 2010
1
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Agenda
• Problem Statement
• Methodology
• System Design Update
• Values and Metrics Update
• Preliminary Results
• Friction Points
• Earned Value Management
2
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Problem Statement
•
•
The goal of the research is to develop a value model that assesses the
transition of security lead from the International Security Assistance Force
(ISAF) and US Forces Afghanistan (USFOR-A) to the Afghani government
and Afghanistan National Security Forces (ANSF)
Deputy Director, Force Integration and Training (CJ7) / CSTC-A defined five
lines of operation (LOOs) that support the goal of transferring security
operations
–
–
–
–
–
•
•
3
Accelerate ANSF growth
Achieve security for the Afghan population
Marginalize malign actors
Achieve legitimate, responsive, and accountable governance
Facilitate community development
Develop metrics and an accompanying decision support tool to measure
progress against the five LOOs
Stakeholders
–
–
–
–
Force Integration and Training cell of NTM-A/CSTC-A (sponsor)
NTM-A/CSTC-A
Coalition military leadership
U.S. government leadership
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Methodology
Project group organized into two subgroups
• Values and Metrics
– Research values and metrics
• “Requirements” to win a counter-insurgency conflict
• Assessments of ANSF, security, Afghani government, and community
– Develop value model with sponsor
• System Design
– Development of user interface, input forms, storage, usable
output
– Integrate values, metrics, and value model from other team
into the system
4
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Technical Approach – System Design
• System Input: the quantitative portion of the value model in a
standardized survey format, completed by military units
• System Processing and Storage: completed survey templates are
configuration controlled and ingested into data storage. User
querying capabilities allow the retrieval of data (by unit and/or
AOR and/or date range) to research trends
• Analysis Output: Condensed and easily understood presentation
for decision makers
5
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Concept of Operation
• Surveys from 5 main military regions
• CJ7 processes surveys and requests status report
CJ7 Processes
Surveys
Requests Status
Report
Military Regions
6
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
System Design Update
• Past Weeks Progress
–
–
–
–
Input Form Prototype
Data Compiler Prototype
Query Prototype
Output Prototype
• Way Ahead
– Obtain weights for Value Model metrics
– Refine interface and status report requirements
– Expand Compiler Capacity
7
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Operational Scenario
8
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Technical Approach – Value Model
• Qualitative Value Model: the identification of an objective
hierarchy relating fundamental and means objectives
• Quantitative Value Model: the articulation of the decision
maker’s preferences towards the attributes, and the means of
measuring each attribute
V(x) = ∑wivi(xi) where wi = weight of attribute i
vi = value of attribute i at score xi
9
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Values and Metrics Update
• Near Term
– Completing value/metric hierarchy without
sponsor input
– Completing weight elicitation without sponsor
input
– Completing input forms
• Mid Term
– Provide completed hierarchy with weights and
input form to system team
10
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Weights - Theory
(1 of 2)
•
Weights represent the relative importance of each parameter vis-à-vis
other parameters at the same node
– must sum to one at each level under each node
•
Bottom Row weights represent lowest level parameter’s importance to
overall decision; used for final check with DM
– Product of the weights on the branch of the tree
– Also must sum to one
Sports Car
.7
.3
Performance
.1
.35
.25
Braking Acceleration
(.7)x(.1)
(.7)x(.35)
.07
.245
11
Comfort
.3
Handling
(.7)x(.25)
.175
Top Speed
(.7)x(.3)
.21
.4
.35
.25
Head
Leg
Shoulder
Room
Room
Room
(.3)x(.4) (.3)x(.35) (.3)x(.25)
.075
.12
.105
Bottom Row Weights
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Weights - Theory
(2 of 2)
• First, define Range of Variation (ROV) as the actual range of
possible values (from worst to best) for a parameter
– For example, in our sports car example suppose that, for the cars we are
examining, the car with the slowest top speed was 140 mph and car with
the highest top speed was 190 mph, with all other cars in between:
• ROV: 140 – 190 mph
• Several methods to elicit weights
– Direct weights: simply ask DM to provide
– Swing weights: “… thought experiment in which DM compares individual
attributes directly by imagining (typically) hypothetical outcomes.”
• Robert T. Clemen & Terence Reilly
– SMARTER: requires only that DM rank order attributes 1 - n
– Rank Reciprocal: requires only that DM rank order attributes 1 - n
– Rank Sum: requires only that DM rank order attributes 1 - n
12
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Swing Weights – Example
(1 of 2)
• Assume you have a node with 3 parameters, each with the
indicated ROV
– Annual Income; ROV: $18K - $40K, where higher is better
– Income Tax Rate; ROV: 1.3% - 8.5%, where lower is better
– Population; ROV: 250,000 – 500,000, where higher is better
• Step 1; Select the one attribute you most want to shift from
worst to best
– Suppose you most prefer to move income from $18K to $40K
• Step 2; Select the second most desired attribute to change, from
worst to best. How important is this to you compared to your
first choice ?
– Suppose you would prefer to move the tax rate from 8.5% to 1.3%, but
only half as important as shifting annual income
• Repeat step 2 for all remaining parameters
– In our example income is ten times more important than population
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Swing Weights – Example
(2 of 2)
• Summary of DM’s feedback on previous slide with K-weight
concept:
– Most important attribute to improve is income: Kincome = 1
– Second most important attribute to improve: Tax Rate
• Half as important as income therefore Ktax = .5 Kincome
– Third most important attribute to improve: Population
• One-tenth as important as income therefore Kpopulation = .1 Kincome
• Convert to scaled weights that will be inserted the appropriate
node of the value model:
• Kincome
• Ktax
• Kpopulation
14
=1
= .5 Kincome
= .1 Kincome
= 1 / 1.6 = .625 weight income
= .5 / 1.6 = .3125 weight tax
= .1 / 1.6 = .0625 weight population
Column Sums to 1.6
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Preliminary Results
• Functioning test system using input forms
15
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Preliminary Results
• Functioning test system using queries
16
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Preliminary Results
• Functioning test system creating an output
from queries
17
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Friction Points
• Distance and interaction of sponsor
– No face-to-face meetings possible
– Flow of information is sporadic
– Use local point of contact for weight elicitation and fabricate
unavailable data
18
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Earned Value Management
BCWS
ACWP
BCWP
$45,000.00
Cost
$40,000.00
$35,000.00
$30,000.00
$25,000.00
$20,000.00
$15,000.00
$10,000.00
$5,000.00
$0.00
Weeks
19
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Cost Index
1.20
1.00
ratio
0.80
CPI
0.60
SPI
0.40
0.20
0.00
Weeks
20
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Questions?
21
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
BACKUP SLIDES
22
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Work Breakdown Schedule
23
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
External System Diagrams
24
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
System Analysis Diagram
25
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Functional Architecture
26
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Functional Decomposition
27
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Value Structure
(1 of 5)
Transition
Assessment
Accelerate
Growth of ANSF
Force Size
Availability of
Equipment
Training and
Leadership
Security for
Afghan People
Marginalize
Malign Actors
Accountable
Governance
Ministry and HQ
Capability
Confidence
Assigned to NATO
Partner
Community
Development
Training
Capability
Weapons
Unit Rating
Number of Units
Fielded
Communication
Equipment
Leader to Soldier
Ratio
Kill Rates
Access to
MEDEVAC
Acquisition
Operations
Average Assigned
Strength
Vehicles
Advanced
Training
Engagements
Access to Fire
Support
Training Plan and
Goals
Unit Facilities
Staff Training
Access to
Intelligence
Force
Sustainment
ETT Availability
Access to
Supplies
Force
Development
Present for Duty
Win/Lose Ratio
Access to Combat
Multipliers
Force
Management
Resource
Management
28
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Value Structure
(2 of 5)
Transition
Assessment
Accelerate
Growth of ANSF
Security for
Afghan People
Economic
Indicators
Marginalize
Malign Actors
Accountable
Governance
Kinetic
Indicators
Price of Exotic
Foods
% of Security
Forces Affected
by Insurgency
New Business
Formations
ISAF+ANSF KIA
New
Construction
Voluntary
Reporting
Capital Flight
Afghan-onAfghan Violence
Assassination
and Kidnapping
Rate
29
Wound/Capture
vs. Kill Rate
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Community
Development
Value Structure
(3 of 5)
Transition
Assessment
Accelerate
Growth of
ANSF
# Districts w/
Functioning
Governance
30
Officials Sleep
in District
Security for
Afghan
People
Detainee Guilt
Ratio
Marginalize
Malign Actors
Captured
Insurgent
Health
Religious
Leaders’
Attitude
Accountable
Governance
Community
Development
Media
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Access to
Internet
#Shadow
Governments
Value Structure
(4 of 5)
Transition
Assessment
31
Accelerate
Growth of
ANSF
Security for
Afghan People
Marginalize
Malign Actors
Accountable
Governance
Community
Development
# New Court
Cases
Participation
in Government
Activities
AntiGovernment
Protests
#Officials
Purchasing
Position
Budget
Execution
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Formation of
Anti-Insurgent
Lashkar
# CIVCAS by
ANSF
Value Structure
(5 of 5)
Transition
Assessment
Accelerate
Growth of
ANSF
Security for
Afghan
People
Marginalize
Malign
Actors
Accountable
Governance
# Afghan
Projects
32
Community
Development
Electricity
Available
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Community
Opinion
Medical Care
Metrics and Range of Variation
(1 of 7)
Accelarate Growth of ANSF:
Increase the growth rate of the ANSF
(ie obtain desired state more quickly
than current plan (when do current
plans indicate that ANSF will be at
desired strength/capability?))
Force Size: What is the size of the ANSF?
Training Capability
metric: capacity of facilities
metric: #trainers available/#trainers
required
See ETT/MiTT Availability belwo of
#trainers available/required.
metric: number of units fielded
ROV: [ANSF = ANA+ ANP]. # troops
provided in Afghan Index : ANA: low
of 6,000 (in Dec 2003) to high of
95,523 (in Nov 2009); ANP: 95,000
(in Nov 2009). Total ANSF: 6,000 (in
2003, ANP # not available) to
190,523 (ANSF in Nov 2009).
metric: average assigned strength
Per Afghan Index: as of May 2009
ANP #assigned/#authorized = 99%;
No comparable number for ANA.
Recruitment Rate
metric: monthly recruitment (percent
of goal)
ROV: [ANSF = ANA+ ANP]. Annual
recruitment in Afghan index: ANA:
low of 9,671 (Mar 2003 - Feb 2004)
to high of ~34,000 (Mar 2008 - Feb
2009). ANP: 17,474 (Mar 2007 - Feb
2008) and 17,191 (Mar 2008 - Feb
2009). Total ANSF: 9,671 (in
2003/04, ANP # Not Available) to 51,
191 (ANSF in 2008/09).
Weapons
metric: #rifles on hand / required
Number of Units Fielded
Average Assigned Strength
Availability of Equipment: Do
ANSF units have access to proper
equipment?
Communications Equipment
Vehicles
Unit Facilities
33
Theoretically Feasible ROV: 0 100%
Theoretically Feasible ROV: 0 metric: #MG on hand / required
100%
Theoretically Feasible ROV: 0 metric: #radios on hand / required
100%
Theoretically Feasible ROV: 0 metric: #vehicles on hand / required
100%
Theoretically Feasible ROV: 0 metric: #company HQ avail / required 100%
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Metrics and Range of Variation
(2 of 7)
Accelarate Growth of ANSF:
Increase the growth rate of the ANSF Training and Leadership: What is Unit Rating
(ie obtain desired state more quickly the level of training, is there enough
leadership for ANSF forces?
Leader to Soldier Ratio
than current plan (when do current
plans indicate that ANSF will be at
desired strength/capability?))
Advanced Training
metric: # complete/required
Theoretically Feasible ROV: 0 100%
Theoretically Feasible ROV: 0 100%
ETT/MiTT Availability
metric: OH/req
ROV: [ANSF = ANA+ ANP]; ANA:
1,062 US Embedded Training Team
(ETT) Assigned / 2,391 US ETT
Required (44%) (Mar 2008), 1,138 /
2,225 (51%) (Nov 2008), 1,175 /
2,663 (44%) (May 2009) so ROV =
44% - 51%. ANP: 921 US PMT
assigned / 2,358 required (39%)
(Mar 2008), 886 / 2,375 (37%) (Nov
2008), 1,050 / 2,375 (44%) (May
2009) so ROV = 37% - 44%
ANSF Independent Units
ROV: per Afghan Index: ANA Units
(Total # units projected to be about
120 by 2014): CM1 (0 to 30), CM2 (5
to 30), CM 3 (25 to 40), CM 4(1 to
10) based on Jun 06 - May 09 data.
ANP Units (Total # units projected to
be about 450 in 2014): CM1 (0 to
30), CM2 (0 to 30), CM3 (5 to 60),
metric: # units operating
CM4 (300 - 450) based on Feb 08 independently in their own battlespace May 09 data.
ANSF Units Conducting
Operations at Battalion Level
metric:# units conducting operations
with/without NATO oversight/support
Present for Duty
metric: percent present for duty
see CM2 above (Battalion level)
ROV: Afghan Index, ANA AWOL
Rate: 7% (Mar 2007/Feb 2008), 9%
(Mar 2008/Feb 2009). ANP AWOL
Rate not provided. Present for Duty
= 1 - AWOL Rate.
metric: reenlistment rate (higher
indicates more confidenc)
ROV: Afghan Index, ANA
Reenlistment rate: Soldiers 50%
(Mar 07/Feb 08) to 57% (Mar
08/Feb09). NCOs: 56% (Mar
07/Feb08) to 63% (Mar 08/Feb09).
ANP Reenlistment rate not provided.
Staff Training
Confidence: Are ANSF forces
confident in their ability to egage
and defeat insurgents, protect the
population?
Reenlistments
34
metric: rating in training categories
metric: avg leader to soldier ratio for
all units
metric: # complete/required
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Metrics and Range of Variation
(3 of 7)
Confidence: Are ANSF forces
confident in their ability to egage
and defeat insurgents, protect the
Accelarate Growth of ANSF:
Increase the growth rate of the ANSF population?
(ie obtain desired state more quickly
than current plan (when do current
plans indicate that ANSF will be at
desired strength/capability?))
Kill Ratio
metric: #enemy/#ANSF
# wins vs # losses
Access to Intelligence
metric: wins/losses
metric: # CO elements that have
access within 10 min
metric: # CO elements that have
access within 10 min
metric: # CO elements with GS
support
Per Afghan Index: ROV:
#ANA killed: 1 (Mar 2007) to
51 (May 2009); #ANP killed:
19 (Feb 2007) to 133 (July
2008). # enemy killed not
available (publicaly)
Not openly available;
SIGACTS
ROV: 0-# of ANSF units
No data avail
ROV: 0-# of ANSF units
No data avail
ROV: 0-# of ANSF units
No data avail
Access to Supplies
metric: # CO elements with unfilled
requests (could specify food, fuel,
and ammunition - Class I,III,V)
ROV: 0-# of ANSF units
No data avail
metric: MiTT assessments
ROV: CM4-CM1 or other
Data not readily avail on higher HQs
Acquisition Operations
metric: MiTT assessments
ROV: CM4-CM1 or other
Data not readily avail on higher HQs
Training Plans and Goals
metric: MiTT assessments
ROV: CM4-CM1 or other
Data not readily avail on higher HQs
Force Sustainment
metric: MiTT assessments
ROV: CM4-CM1 or other
Data not readily avail on higher HQs
Force Development
metric: MiTT assessments
ROV: CM4-CM1 or other
Data not readily avail on higher HQs
Resource Management
metric: MiTT assessments
ROV: CM4-CM1 or other
Data not readily avail on higher HQs
Win/Loss Ratios
Access to Combat Multipliers
Access to MEDEVAC
Access to Fire Support
Ministry and HQ Capability: Can
Ministry/HQ manage the ANSF
effectively?
Force Management
35
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Metrics and Range of Variation
(4 of 7)
Security for Afghan People:
Provide safe environment for all
factions in the Afghan population
metric: total number of ANSF/total
security forces operating in
Afghanistan
metric: total KIA's from
Total KIA's During Preceeding ANSF+ISAF+USFOR-A during past
12 months
12 Months
% of Security Forces that are
from the host nation
Aghan-on-Afghan Violence
Kinetic: Direct, conflict based
measures of security.
Economic: Indirect measures of
security (i.e. higher/lower security
impacts these measures)
36
ROV: 190K (current) to 264K (goal)
ROV: (ISAF) 57-571; (ANSF) annual
data available
Wound/Capture vs. Kill Rate
metric: number of incidents
Not openly available; SIGACTS
metric: number of enemy wounded or
captured / # killed
Not openly available; SIGACTS
Voluntary Reporting
metric: # of reports in a week
Price of Exotic Foods: higher
prices reflect increased risk due
to difficult (unsecure) transport
metric: average price of selected
foodstuff
New Business Formations
New Construction
Capital Flight
metric: # of new business permit
applications
metric: # new construction projects
metric: net change in holdings of
Aghan banks
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Not openly available
Need to select foods
Not readily available; World Bank
has GDP data but only updated
yearly
Not readily available
$60 million to $2285 million
Metrics and Range of Variation
(5 of 7)
Marginalize Malign Actors:
Eliminate influence of malignant
individuals/groups
# Districts with Functioning Governance
metric: number
metric: percentage of officials
who reside within their district
Officials Sleep in District
metric: percentage of detainees
who turn out to be insurgents
Detainee Guilt Ratio
metric: percent of captured
insurgents with moderate to
serious health problems
Captured Insurgent Health
Media: Television/Newspaper
News/Political Opinion Editorials
Internet Access: was forbidden
under Taliban
Theoretically Feasible ROV: 0 - 365
(However, considering the changing
number of districts, we may want this
to be a percentage)
Theoretically Feasible ROV: 0 - 100%
Theoretically Feasible ROV: 0 - 100%
Theoretically Feasible ROV: 0 - 100%
I believe this is a fallable metric, since
it offers equal weight to every media
outlet, despite the number of
Theoretically Feasible ROV: 0 subscribers.
100%
metric: percent sympathetic to
insurgents
metric: percent of population with
internet access
Theoretically Feasible ROV: 0 - 100%
Hostile to whom? Hostility to U.S. is
different from hostility to the Afghan
gov't (neither of which mean support
metric: # hostile religious
for the Taliban). And, if we can't count
services with over 100 attendants attendance, how do we weight them
(counting attendence won't
effectively? We'd be comparing
happen)
apples to oranges.
Religious Leaders
metric: number of districts with
shadow governments operating
Theoretically Feasible ROV: 0 - 365
Number of Shadow Governments within them
37
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Metrics and Range of Variation
(6 of 7)
Accountable Governance: Gov't
officials/institutions are honest (not
corrupt), fair (do not favor one group
over another), legitimate (respected
by the population as the legal
authority).
Number of New Court Cases
Participation in Govt Activities
Government Protests
Possible ROV: 0 to max number over
metric: number of cases opened 2002-2009 for court cases per month
Perhaps we can change this to
"percent of people who turn to state
metric: average number of
courts for various cases," as identified Theoretically Feasible ROV: 0 participants
in the Afghan Index.
100%
metric: number of protests
NumberExecution:
of OfficialsPerhaps
Purchasing Position
metric: number
Budget
money that is allocated, but not
spent, would be a sign of a corrupt metric: money spent / money
govt that holds money for itself
allocated
Formation of Anti-Insurgent LASHKAR
metric:
Formations
number
# CIVCAS by ANSF
38
metric: number
Possible ROV: 0 to max number over
2002-2009 for government protest of
over 100 participants per month
Possible ROV: 0 to total number of
officials identified
Theoretically Feasible ROV: 0 - 1
Possible ROV: 0 to max number over
2002-2009 for formations identified
Possible ROV: 0 to max number over
2002-2009 for CIVCAS
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Metrics and Range of Variation
(7 of 7)
Community Development:
Improvement in infrastructure and
basic services
# Afghan Govt Projects
metric: number started
metric: dollar value of new
projects
Electricity Available
Community Improvement
Medical Care
39
metric: total wattage
metric: hours on per day (from
midnight to midnight)
metric: polling data
metric: number of doctors
available per 1,000 inhabitants
Possible ROV: Min number to max
number over 2002-2009 for projects
started that year
Possible ROV: Min annual US$ to
max annual US$ over 2002-2009 for
projects started that year
Possible ROV: Min annual MW to Max
annual MW over 2002-2009. This can
be either total MW or home-generated
MW (not including imported)
Change this to average number of
hours per day
We have a good amount of polling
data, but none discuss the views
Afghan's hold about their own
community development. Perhaps we
change the wording?
Possible ROV: 0 - 1000
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
MW in 2002 = 243 MW (where 87 is
imported), MW in 2007 = 652 (where
167 is imported). Difference in homegeneration: 156 - 485
Theoretically Feasible ROV: 0 - 24
Theoretically Feasible ROV: 0 100%
Range of Variation (ROV)
aka “Range of a value measure”
• Definition: “The possible variation of the scores of a value
measure” -Gregory S. Parnell
• Important precursor to determining DM value (or utility) function
40
*OR681, GMU
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Weights and Utility Curves
(3 of 3)
• Elicit utility through lottery or certainty
equivalence
• Weights and utility can be linear, piecewise,
exponential, or an S-curve
Top Speed (MPH) vs Utility
1
Utility
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
150
160
170
180
190
200
210
220
230
Top Speed (MPH)
41
*OR681, GMU
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY
Visit Our Project Website
http://mason.gmu.edu/~dugarte/index.html
42
UNCLASSIFIED – FOR INSTRUCTIONAL USE ONLY