Learning and Tracking with Parametric
Performance Measurement
*******
PPMC
Presented by:
Karen McRitchie
Galorath Incorporated
100 North Sepulveda Boulevard
Suite 1801
El Segundo, California 90245
KMcRitchie@galorath.com
 Galorath Incorporated 2004
1
Overview
How project tracking fits into the scheme of SEER tools
Explain some Earned Value metrics and measures
Describe how the earned value measures can be applied to a
SEER-SEM estimate
Introduce PPMC, the SEER-SEM add-on that embodies these
concepts
 Galorath Incorporated 2004
2
se
Ba
lin
e
Measure
&
Analyze
Estimate
&
Plan
ate
-d ls
To ctua
A
Kno
Expwledge
erie &
nce
SEER: Project Management Context
Monitor
&
Control
(s) &
Baseline tuals
Final Ac
 Galorath Incorporated 2004
3
SEER: Process Focus (CMMI)
Project Planning
• Establish Estimates
• Develop a Project Plan
• Obtain Commitment to the Plan
Project Monitoring and Control
• Monitor Project Against Plan
• Manage Corrective Action to
Closure
Measurement and Analysis
• Align Measurement and Analysis
Activities
• Provide Measurement Results
 Galorath Incorporated 2004
4
SEER: Methods
Project Planning
• History-Based Sizing
• Direct Measure Sizing
• Parametric Estimation
• 4-D Planning
Project Monitoring and Control
• 4-D Earned Value Management
• Status Indication
• Performance-based Forecasting
Measurement and Analysis
• Estimation Accuracy
Assessment and Improvement
• Calibration
• Knowledge Base Management
• Benchmarking
 Galorath Incorporated 2004
5
SEER: Tools
Project Planning
• SEER-AccuScope
• SEER-SEM
• Project Miner (SEER-SEM option)
• SEER-H
• SEER-IC
• SEER-DFM
• SEER-SEM Client for Microsoft
Project
Project Monitoring and Control
• Parametric Performance Monitoring
and Control (SEER-SEM option)
Measurement and Analysis
• SEER-CommonRepository
• SEER-ScatterPlot+
currently in development
 Galorath Incorporated 2004
SEER-SEM
Calibration
Mode
SEER-SEM
PPMC
6
What is PPMC?
Parametric Performance Monitoring and Control
A set of add-on features to SEER-SEM that enable project monitoring and
control of software development projects
Features include:
• Inputs for actual effort and project progress as well as project metrics
• Algorithms to combine parametric and earned value
• Reporting to provide information on project status
PPMC employs 4D Earned Value Management
• Activity Completion
• Expenditures
• Artifact Completion
• Defect Discovery/Removal
 Galorath Incorporated 2004
7
Why PPMC?
Earned Value Management is considered a best practice for monitoring
and controlling the progress of a software project
EVM has limitations:
• A single focus on activity completion as the basis for earning value
• Difficulty in providing realistic performance based estimate at completion
(EAC)
PPMC adds monitoring and control capabilities to SEER-SEM
PPMC combines:
• Accepted algorithms currently used for cost and schedule estimation during
the project planning process
• Accepted equations currently used for traditional Earned Value Management
Parametric Performance Monitoring and Control
 Galorath Incorporated 2004
8
The common question
Once a project has started, it is only natural to want to enter your
project metrics
The most common question:
• Can I put in my actual effort once the project is underway and will the
tool recalibrate my estimate?
The right answer is NO - it’s not enough just to give actual hours
• Just because you spent 50% of a planned budget doesn’t mean your
are 50% done
What the project manager needs to know:
• How much more money, how long?
• How can the project be adjusted to meet objectives?
To do this, you need assess the project status in terms of
progress, not just expenditures – Earned Value
Earned value compares work complete to original estimates.
 Galorath Incorporated 2004
9
Traditional EVM – Measures and Metrics
Fundamental Measures
• Budget at Completion (BAC)
• Budgeted Cost of Work Scheduled (BCWS)
• Budgeted Cost of Work Performed (BCWP)
• Actual Cost of Work Performed (ACWP)
Variances
• Schedule Variance (SV)
• Cost Variance (CV)
• Time Variance (TV)
Performance Indices
• Schedule Performance Index (SPI)
• Cost Performance Index (CPI)
• To-Complete Performance Index (TCPI)
 Galorath Incorporated 2004
10
Fundamental EVM Measures - 1
Fundamental Measures can be expressed in terms of cost ($), labor
(hours) or %
Budget At Completion (BAC)
n
BAC (Labor ) = ∑ Activity Planned Hoursi
i =1
Subtle
difference
between these
two: BCWS is
time sensitive
Budgeted Cost of Work Scheduled (BCWS)
BCWS is what
you planned to
get done by time
“t”
 Galorath Incorporated 2004
n
BCWSt (Labor ) = ∑ Activity Planned Hours j
j =1
11
Fundamental EVM Measures - 2
Budgeted Cost of Work Performed (BCWP)
BCWP measures
what you have
‘earned” or
completed
relative to the
original plan.
For BCWP, count
the planned cost of
“completed”
activities at time “t”
m
BCWPt (Labor ) = ∑ Completed Activity Planned Hoursl
l =1
Actual Cost of Work Performed (ACWP)
ACWP is what
you spent by
time “t”
 Galorath Incorporated 2004
n
ACWPt ( Labor ) = ∑ Activity Actual Hours k
k =1
12
SEER-SEM estimates are EVM ready
Computing these EVM measures
requires
• The estimate plan has a break
out of activities
• The estimate plan is time
sensitive
SEER-SEM estimates have these
qualities
• Staffing plan shows the funding
profile over time
• Activity break down provides
effort and schedule by major
milestone
 Galorath Incorporated 2004
13
Applying EVM to a SEER-SEM estimate
Area under the
curve is BAC or the
“plan”
3500
3000
Hours
2500
2000
1500
1000
500
0
1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627
Month
Plan
BAC = Original Estimate
48,200 hours
26.38 schedule months
 Galorath Incorporated 2004
14
Looking at a point in time
Area under red line
is ACWP
3500
Area under the
orange curve is
BCWS
3000
Hours
2500
2000
1500
1000
500
3500
0
3000
2500
Month
Plan
Hours
1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627
Actual Expenditure
2000
1500
1000
500
0
At “t” = 18 months
1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627
Month
ACWP = 28,700 hours
Plan
Actual expenditures
exceed the budget at this
time, but is the project in
trouble?
 Galorath Incorporated 2004
BCWS
At “t” = 18 months
BCWS = 24,707 hours
15
Evaluating what has been “earned”
In order to determine if the project is on track or in trouble, you must
evaluate what has been accomplished, not just what has been spent
SEER-SEM allows you to describe
what you have completed*
Area under the light
blue curve is
BCWP
3500
3000
Hours
2500
2000
1500
1000
At “t” = 18 months
BCWP = 23,856 hours
500
0
1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627
Month
Plan
 Galorath Incorporated 2004
Earned Effort
Actual Expenditure
* Screen from PPMC add-on for SEER-SEM. Similar inputs can be made with the Estimate to Complete
parameters
16
Variances
Variances can be used to analyze the fundamental measures
Positive variances are favorable, negative unfavorable
Cost Variance (CV) is the difference between the earned and actual
CVt (Labor) = BCWPt (Labor) − ACWPt (Labor)
Schedule Variance (CV) is the difference between earned and planned
SVt (Labor) = BCWPt (Labor) − BCWSt (Labor)
Time Variance (TV) is the lateral difference between earned and planned
TVt (Duration) = tBCWS = BCWPt − t
 Galorath Incorporated 2004
17
Performance Indices - 1
Performance indices are tracked over time
Greater than 1 is favorable, less than 1 is unfavorable
Cost Performance Index (CPI)
• The cost efficiency achieved from project start to time t.
CPI t =
BCWPt ($)
BCWPt (Labor)
BCWPt (%)
or
or
ACWPt ($)
ACWPt (Labor)
ACWPt (%)
Schedule Performance Index (SPI)
• The schedule efficiency achieved from project start to time t.
SPI t =
 Galorath Incorporated 2004
BCWPt ($)
BCWPt (Labor)
BCWPt (%)
or
or
BCWSt ($)
BCWSt (Labor)
BCWSt (%)
18
Performance Indices - 2
To Complete Performance Index (TCPI)
• Cost efficiency assumed to complete from time t.
TCPICostt =
Work Remaining
=
Cost Remaining
BAC ($) − BCWPt ($)
or
BAC ($)
− ACWPt ($)
CPI t
BAC (Labor) − BCWPt (Labor)
or
BAC (Labor)
− ACWPt (Labor)
CPI t
BAC (%) − BCWPt (%)
BAC (%)
− ACWPt (%)
CPI t
Composite Performance
TCPICompositet = CPI t × SPI t
 Galorath Incorporated 2004
19
Evaluating the project data
BAC
BCWS
ACWP
BCWP
Cost Variance (CV)
Schedule Variance (SV)
CPI
SPI
TCPI
Composite Index
Negative variances are
unfavorable
 Galorath Incorporated 2004
Fundamental Measures
48,200 Overall Plan
24,707 Budgeted hours at month 18
28,700 Actual hours spent at month 18
23,856 Earned hours at month 18
Variances
(4,844) BCWP - ACWP
(851) BCWP - BCWS
Indices
0.83 BCWP/ACWP
0.97 BCWP/BCWS
0.83 (BAC-BCWP)/((BAC/CPI)-ACWP)
0.80 CPI * SPI
Indices less than 1.0 are
unfavorable
For this example, not only have
you spent more than you
planned to, but you have not
completed the work you should
have
20
Estimate At Complete
Estimate At Complete = Actual + Estimate To Complete
Estimate To Complete = BAC - BCWP
3500
3500
3000
3000
2500
2500
Hours
Hours
The yellow line is Estimate
To Complete = BAC BCWP
2000
1500
2000
1500
1000
1000
500
500
0
0
1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627
1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627
Month
Month
Plan
Earned Effort
Actual Expenditure
Plan
Actual Expenditure
Remaining Effort
BAC = 48,200 hours
BCWP = 23,856 hours
ACWP = 28,700
 Galorath Incorporated 2004
At “t” = 18 months
EAC = 53,044 hours
21
Earned Value and PPMC Terminology
 Galorath Incorporated 2004
Earned Value Terminology
What it means
BCWS - Budgeted Cost of Work
Scheduled
The planned costs (baseline plan)
SEER-SEM calls this the Baseline Plan
ACWP – Actual Cost of Work
Performed
What you paid
SEER-SEM calls this Actual Effort or Actual Cost
BCWP – Budgeted Cost of Work
Performed
The earned value
SEER-SEM calls this Schedule Accomplishments
EAC – Estimate At Complete
The latest estimated cost at completion
ETC – Estimate to Complete
Funds needed to complete the investment
SPI – Schedule Performance
Index
The percent of the investment that has been completed
CPI – Cost Performance Index
The ratio of earned to actual cost of work performed
TCPI - To Complete
Performance Index
The latest cost efficiency assumed to complete
CV - Cost Variance
The difference between earned and actual cost of work
performed
SV – Cost Variance
The difference between actual and planned work
TV – Time Variance
The variance between the actual and planned schedules
22
The PPMC Approach
PPMC extends the use of SEER-SEM to projects underway
 Galorath Incorporated 2004
23
Tracking a Project
Once a project is started, the PPMC process may begin
Start with a SEER-SEM estimate
Establish a baseline estimate
Take project “snapshots” periodically
Evaluate earned value metrics
Replan as required
 Galorath Incorporated 2004
24
The Baseline
A baseline freezes the current
estimate
The baseline provides the
yardstick against which the
project is tracked and evaluated
Allows you to do trades with the
current estimate while
maintaining the planning baseline
A baseline can be updated at any
time
The current estimate may be set
back to the baseline at any time
 Galorath Incorporated 2004
25
The Snapshot
Snapshots are a picture of the project’s metrics at any given point
in time
The snapshot includes two key components:
• Work Complete
• Actual Effort
Work Complete describes the progress in terms of milestone
completion
Actual Effort describes measurable project metrics:
• Actual hours to date
• Defects reported and removed to date
• Size complete to date
Snapshots are flexible in terms of interval or frequency
• Regular snapshots – e.g. monthly or quarterly
• As needed – e.g. before a key review
 Galorath Incorporated 2004
26
Snapshots
Snapshots are entered in matched sets in a dedicated input view
 Galorath Incorporated 2004
27
Reporting
PPMC reports and charts are based on the most recent snapshot
entered
Work complete metrics are used to evaluate remaining effort and
schedule (Estimate To Complete)
Estimate At Complete = Actual Effort + ETC
 Galorath Incorporated 2004
28
PPMC Reporting Options
 Galorath Incorporated 2004
29
Looking at Progress Over Time
See how actual and earned
effort measure up to the
baseline plan
Click on any
month to see
specific
information
 Galorath Incorporated 2004
See CPI and SPI trends over
time
CPI = Cost
Performance Index;
SPI = Schedule
Performance Index;
The ratio of earned
to actual cost of
work performed
The percent of the
investment that has
been completed
30
Other Metrics
Track defect
discovery and
removal rates
against expected
rates
Heath and Status Indicator
shows status and trends from
the previous snapshot
Thresholds are user definable
Increased defect
reporting rate
shows a
worsening trend
 Galorath Incorporated 2004
31
Summary
Parametric Performance
Monitoring and Control enables
powerful evaluation of a project’s
progress
Utilizes EVM best practices
Capitalizes on parametric model’s
ability to replan and forecast
based on current project
assumptions
Flexible – doesn’t require regular
care and feeding, you can use it
only when you need to
Completes the project
management continuum
 Galorath Incorporated 2004
32
Thank you!
For more information:
info@galorath.com
www.galorath.com
 Galorath Incorporated 2004
33