49th Annual NCMA March Workshop What Contracting Professionals Need to Know About Earned Value Management Presenters: Frank Malsbury, CPCM, NCMA Fellow Internal IBR Manager Raytheon Buddy Everage, PMP Director Emergent IT Programs MCR Federal, LLC Agenda • Speaker Introductions • Background and Overview of EVMS • What is it? • ANSI/EIA 748 EVMS Guidelines • Basic Principles • An example of how an EVMS works (EVM Data Analysis) • Information for management decision making • EVM Policies and Directives • OSD / OMB • EVM in Federal Contracting • Where to find EVM requirements in Solicitations • Proper response to EVM requirements • Summary Earned Value Management Definition • Earned Value Management (EVM) is a project management technique that objectively tracks physical accomplishment of work. • EVM provides an early warning of performance problems while there is time for corrective action. • In addition, EVM improves the definition of project scope, prevents scope creep, communicates objective progress to stakeholders, and keeps the project team focused on achieving progress. • Strategic alignment of program objectives with agency mission is essential to obtaining program funding (OMB 300). Once strategic alignment is established, EVM provides the visibility to ensure that the program cost, schedule and technical objectives are met. Very Effective When Used in Conjunction with Other Management Techniques Definitions – Clarification of Terms • Earned Value (EV) – The value of completed work expressed in terms of the budget assigned to that work, often referred to as Budgeted Cost of Work Performed (BCWP). It provides an objective measure of the value of completed work expressed in terms of the budget assigned to perform the work. • Earned Value Management (EVM) – A management methodology that integrates a program’s technical scope, schedule and resources with program risk in a baseline plan against which progress is measured to produce metrics that highlight variances and performance trends, thereby providing program managers, as well as high level management, with objective visibility into program progress and enable more effective management. • Earned Value Management Systems (EVMS) – An integrated set of policies, processes, procedures, systems and practices that meets an organization’s implementation of ANSI/EIA-748. The Language of EVM is Unique EVMS History • The first attempts to view the status of major projects was the DOD, PERT/Cost, in the 1960’s • Then came the DOD Cost Specification (CSpec) requirements • In 1967 the DOD Cost/Schedule Control System Criteria (C/SCSC) • In the late 1980’s the Earned Value Management System (EVMS) became a project management methodology • Currently, ownership of EVM criteria is maintained by industry with adoption of ANSI EIA 748-B standard EVM Has Evolved Over Many Years ANSI/EIA 748 EVMS Guidelines EVMS Guidelines 32 Guidelines establishes the framework within which an adequate integrated Cost/Schedule/Technical Management System will fit The five major Aspects of an Earned Value Management System are required to “play” together Organization 5 Define the authorized work and assign responsibility for the work. Integrate planning, scheduling, work authorization and accounting systems. Planning & Budgeting 10 Plan, schedule, budget and authorize the work. Establish and maintain a time-phased budget baseline. Accounting 6 Accumulate costs for work and material Analysis & Management Reports 6 Compare planned and actual costs, and analyze variances. Identify management actions taken. Horizontal and vertical communications and traceability. Develop revised estimates of costs at completion. Revisions & Data Maintenance 5 Timely incorporation of contractual changes. Maintain traceability from original budgets. Prevent retroactive changes The Questions that EVM Answers • How can a Program Manager get an early warning of performance problems while there is time for corrective action? • How does a Program Manager improve the definition of project scope & prevent scope creep? • How does a Program Manager communicate objective progress to stakeholders? • How can a Program Manager forecast the cost at completion and the end date for the project? Used Correctly, EVM Aids Proactive Management 10 March 2010 7 What is Earned Value Management? Plan Manage Measure EVM Analyze Collect Costs What is Earned Value Management? • EVM is a technique that relates resource planning to schedules and to technical performance requirements • EVM integrates project cost, schedule, technical effort and risk in a realistic, executable plan called the Performance Measurement Baseline (PMB) • Tracks performance/progress to the plan • Provides early warning of developing problems/opportunities • Enables early corrective action when deviating from the plan • EVM is a repeatable process • Standard and consistent methods for progress measurement • Consistent use of performance metrics on all projects • Enables management to easily assess health of the enterprise • Provides project past performance data and lessons learned useful for improving performance and in preparing future estimates for similar work A Disciplined Earned Value Management System is Powerful EVM is a Program Management Process EVM is a Program Management process for integrating Detailed Schedules • Scope Control Account Subcontract Schedules Plans • Schedule • Resources • Cost • Technical Requirements to measure program performance to determine if cost and schedule performance are as planned to determine final program costs EVM is a key factor in winning and managing federal contracts Cost Account Plan (CAP) WBS Name WBS: 1.2.1.1: Design, Fabricate, Test TT&E EDU Task Task 1.2.1.1a EV Method Weighted Milestone Item J F M A M J J A S FY97 FY 98 FY 99 FY 99 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 CAM: Morris Money O N D TOTAL N/A CONTRACT MILESTONES PLAN 50 75 50 50 75 50 ATP 300 50 EARN PDR FCA Lot 1 Lot 2 Lot 3 1.0 Total Contract Task PLAN 1.2.1.1b Fixed Formula 100 1.1 Space Craft 5/9 8/15 EARN Task % Complete Estimates 1.2.1.1c 1.2 PLAN 8/15 4/8 600 100 100 100 100 100 175 1.3 EARN 11/4 12/1 Task 1.2.1.1d Equivalent Units PLAN 75 75 75 300 75 1.4 4/8 2/18 EARN 1.5 Task 1.2.1.1e PLANNING PACKAGE 1.2.1.1 TOTAL COST ACCOUNT PLAN 100 225 225 150 150 150 100 1.6 EARN PLAN EARN 200 225 250 225 225 250 225 225 150 150 150 100 2/18 9/14 1000 2,300 1.7 The Program Manager Owns Earned Value Management 5/21 9/14 1/31 6/25 What Earned Value Management is Not • It is not a software package – Software vendors that tell you their product is ‘validated’ or ‘certified’, do not understand • It is not a set of reports (although reporting is important) • It is not a financial system, but it is maintained by Finance • It is not funds management (but it is an indicator of funds requirements) Earned Value Management is a Program Management Tool Earned Value Management System (EVMS) Features • A disciplined approach to work definition, work organization and assignment, planning, performance measurement, monitoring and control • Work is defined, assigned and planned; plans are approved before work is authorized to begin – Changes are incorporated, planned, approved, and authorized before work begins on the change • EV methods or measurement techniques provide objective assessment of program progress • Variance identification and analysis, summarized through the CWBS, provides performance trends as well as the identification of developing problems – Problems are quickly and easily tracked to their source for immediate corrective action • Performance metrics predict where the program is going and are useful in validating independent estimates or assessments; modern EVM adds use of Technical Performance Metrics The Earned Value Management Process Define the Work Plan the Work Work the Plan Collect Results Measure Performance Change Control External Changes Analyze Deviations Take Corrective Action Earned Value Management is an Iterative Process PMB and the Integrated Baseline? Contract Baseline R I S K & O P P O R T U N I T Y • • • • Contract/Contract Letter Statement of Work (SOW) OR of Objectives (SOO) CDRLs and DIDs System Specification … Work Authorizations (planning) • Primary Work Authorization (PWA) • Secondary Work Authorizations (SWA) • Including Subcontracts & IOTs Program Structure Baseline PMB • • • • Work-Breakdown Structure (WBS) Organizational Breakdown Structure (OBS) Control Account Structure Responsibility Assignment Matrix (RAM) Program Plan & Schedule Baseline EVMS Baseline • Master Phasing Plan • Integrated Master Plan (IMP) • Integrated Master Schedule (IMS) • Control Account Plans • Work Packages & EVTs • Time-Phased Resource Spreads / Staffing Work Authorizations (execution) 14 Program Structuring and EVMS Implementation Work Packages BCWS BCWP ACWP BAC EAC Planning Packages RAM Integrates Work Scope, Budgets & Organizational Responsibility at the Control Account Level 15 IMS/EVMS Planning Define E/A/C (IMP Product Section) Define Control Accts. (RAM) Define IMS Tasks & SubTasks Define & Plan Work/ Planning Packages Sequence Work / Planning Pkg. Activities Control Account Planning IMP/IMS Planning Define Task Relationships Plan WP & PP Resources & Durations Plan IMS Task Resources & Durations Assess Resource Plan Validate EVMS Baseline Assess Critical Path & Schedule Risk Validate IMS Baseline 16 EVM Data Analysis – An Example Earned Value Analysis • % Complete vs. % Spent • Analysis of Variances (SV, CV, VAC) – Schedule Variance (BCWP – BCWS) – Cost Variance (BCWP – ACWP) – Variance At Complete (BAC – EAC) • Graphical Trend Analysis – BCWS/BCWP/ACWP line charts – Cumulative SV and CV line charts – SPi and CPi line charts • To Complete Performance Index (TCPi) – Budget At Complete (BAC) – Estimate At Complete (EAC) • Statistical Forecasting Techniques • Technical Performance Metrics • DCMA Trip Wires (DoD Programs) Key Data Elements and Variances Time Now EAC Forecast Variance At Complete Value BAC Forecast Schedule Slip Schedule Variance (BCWP – BCWS) BCWP Cost Variance (BCWP – BCWS) Time BCWS – Budgeted Cost for Work Scheduled BCWP – Budgeted Cost for Work Performed ACWP – Actual Cost of Work Performed BAC – Budget at Completion EAC – Estimate at Completion ETC – Estimate to Completion Project Objectives • Scope: 200 Drawings • Schedule: 10 Months • Budget: 30 Hours per Drawing • BAC: $300K (6000 hrs x $50/hr) • Plan: 20 Drawings per Month Month 5 Status • 100 Drawings Planned • 70 Drawings Completed • 2450 Hours Spent So, how are we doing? • $52/Hour Average Cost BCWS BCWP ACWP SV CV BAC EAC VAC Hours 3000 2100 2450 (900) (350) 6000 6000 0 Dollars 150.0 105.0 127.4 (45.0) (22.4) 300.0 300.0 0 Percent Variance: Schedule SV Hours BCWS Hours X 100 = % SV (900) Hours SV X 100 = ( 30.0% ) 3000 Hours BCWS 22 Schedule Performance Index Work Completed or Work Planned BCWP BCWS = SPI 2100 Hours BCWP = .70 SPI 3000 Hours BCWS 70% Efficiency to Schedule 23 Percentage Variance: Cost $ Cost Variance X 100 = % CV $ BCWP ($ 22.4) CV X 100 = ( 21.3% ) $ 105.0 BCWP 24 Cost Performance Index Work Completed or $ BCWP = CPI Actual Cost $ ACWP $105.0 K BCWP = .82 CPI $127.4 K ACWP 82% Cost Efficiency 25 Calculated EACs to Assess Realism of Manager’s EAC IEAC* = ACWP + = ACWP + Calculated ETC BAC - BCWP Performance Factor *Independent EAC Analysis Leads to Thresholds and Breach Thresholds require Variance Analysis Reports 26 Current EVM Policies and Directives Defense Spending Changes Losers F-22 Raptor Future Combat Systems Nearly 7 in 10 major U.S. arms programs over budget WASHINGTON (Reuters) – 3/30/09 Nearly 70 percent of the Pentagon's 96 major weapons-buying programs were over budget in 2008 for combined cost growth of $296 billion above original estimates, congressional auditors said in an annual report released on Monday. Marine 1 Pres. Helicopter Littoral Combat Ship F-35 Joint Strike Fighter Reaper-Class UAV Winners Source: Reuters, Popular Mechanics, CNN.com 10 March 2010 28 GAO: Hundreds of federal IT projects are poorly planned and underperforming NextGov.com – 7/31/08 - More than 400 federal information technology projects valued at $25 billion have been poorly planned or performing under expectations — or both, the Government Accountability Office planned to tell a Senate hearing on Thursday •352 projects valued at about $23.4 billion on the management watch list to be poorly planned •87 of their high-risk projects valued at about $4.8 billion were poorly performing •26 projects, valued at about $3 billion, were both poorly planned and poorly performing Earned Value Management is Expected to Help Programs Deliver on Time and Within Cost 10 March 2010 29 Resulting Defense Spending Reform WEAPON SYSTEMS ACQUISTION REFORM ACT OF 2009 - S.454 - A bill to improve the organization and procedures of the Department of Defense for the acquisition of major weapon systems… May 22, 2009: Became Public Law No: 111-23 Section 207 Directs the Under Secretary to review existing (and if necessary, prescribe additional) guidance governing implementation of the Earned Value Management requirements and reporting for DOD contracts. Data Quality EVM Education Contract Execution 30 OSD EVM Policy/Guidance Contracts Thresholds Cost or Incentive Equal to Or Above Threshold > $50M Cost or Incentive Less Than Upper Threshold but Equal to Or Above Lower Threshold < $50M but > $20M Cost or Incentive Less Than Threshold < $20M Requirements - Compliance with industry EVM standard - Formal validation of contractor’s EVM system - Contract Performance Report - Integrated Master Schedule - Integrated Baseline Review - Ongoing surveillance - Compliance with industry EVM standard - Formal system validation not required - Contract Performance Report (tailored) - Integrated Master Schedule (tailored) - Integrated Baseline Review (tailored) - Ongoing surveillance - EVM optional (risk-based decision) - Cost-benefit analysis required OMB EVM Policy/Guidance • Capital Programming Guide v 2.0 – Expands the explanation of concepts in the original guide that was not fully developed, e.g., EVMS • EVMS requirements in contracts or agency in-house project charter • Compliance reviews of agency and contractor EVM Systems • Periodic system surveillance reviews to ensure the EVMS continues to meet the guidelines in ANSI 748 and • Integrated Baseline Reviews (IBRs) to finalize the cost, schedule and performance goals. • NDIA Guides identified as the primary references for information • EVM Federal Acquisition Regulation (FAR) – EVMS identified and formerly linked with ANSI/EIA Standard 748 and OMB Circular A-11, Part 7 – Identifies EVM’s role in the acquisition roadmap – EVMS required for major acquisitions for development in accordance with OMB Circular A-11 EVM in Federal Contracting Fundamental Elements of Contracting for EVM • Identify application of Earned Value Management System in solicitations and contracts • Define reporting requirements and electronic submission • Define evaluation criteria for source selections • Evaluate EVMS – Integration with Award/Incentive plans – EVMS implementation plan for new contracts – Evaluate for compliance with the standard – Support conduct of an Integrated Baseline Review – Contractor’s proposed EVMS Carefully Read the Solicitation (RFP) and Every Attachment (the SOW, CDRLs, DID); Reconcile w/ Contract Awarded Addressing EVM in the Solicitation • Section C – Performance Work Statement/ Statement of Work • CWBS/EVMS/IBR/EVMS Flowdown • Section I – General Provisions • Contract Clauses • Section J – Exhibits and Attachments • Section L - Instructions for Offerors* • Section M- Evaluation Factors for Award* * Note: Sections L and M are included in Solicitations only Carefully Read Everything in the RFP, and Create Spreadsheets and Matrices to Trace Every Requirement Section C Statement of Work • The Statement of Work should contain statements requiring: • Development of a contract work breakdown structure (CWBS) at a level adequate for contract management and control • The contracted technical effort to use an ANSI/EIA-748 compliant EVMS that correlates cost and schedule performance with technical progress • The designation of critical subcontractors by name for EVM compliance and flow down of EVMS compliance to subcontractors • An Integrated Baseline Review (IBR) • Reference to EVMS data items as part of Integrated Program Management reporting and reviews The SOW is the Tasking Document – Know the Tasks Section I General Provisions • New FAR Clauses – July 2006 – FAR 52.234-2 – Notice of EVMS – Pre-Award IBR – FAR 52-234-3 – Notice of EVMS – Post Award IBR – FAR 52-234-4 – EVMS – The Government will conduct an IBR if a pre-award IBR has not been conducted • FAR EVM clauses not tied to dollar thresholds • DFARS Clauses – Solicitation Provision: 252.234-7001 – Notice of Earned Value Management System (Apr 2008) – Contract Clause: 252.234-7002 – Earned Value Management System (Apr 2008) Contracting Officers and Contractors Must Know Everything in Every Clause Section J Exhibits and Attachments • Work Breakdown Structure - MIL-HDBK-881A common WBS for CPRs and IMS through WBS Level 3 • EVM Related Data Item Descriptions (DIDs) • • • • Contract Performance Report – DI-MGMT-81466A Integrated Master Schedule – DI-MGMT-81650 Contract Work Breakdown Structure – DI-MGMT-81334B Contract Funds Status Report – DI-MGMT-81468 • Contract Data Requirements List (CDRL) used to put CPR and IMS reports on contract • EVM CDRLs can and should be tailored within bounds of agency policy • Consider all risk factors when tailoring CPR and IMS DIDs • • • • Type of contract (determined by cost risk) Technology Schedule Past contractor performance Section J – CPR DID DI-MGMT-81466A • The CPR consists of five formats • Uses of CPR Data • The CPR provides timely, reliable summary-level data with which to access current and projected contract performance. • • • • • Format 1 – Work Breakdown Structure Format 2 – Organizational Categories Format 3 – Baseline Format 4 – Staffing Format 5 – Explanations & Problem Analyses • Integrated Cost & Schedule EVM Data • Identify the cost and schedule impact of actual and potential problems • Provide valid, timely program status info for higher management Section J – IMS DID DI-MGMT-81650 • IMS shall include all discrete tasks/activities, work packages, and planning packages identified in the contract Performance Measurement Baseline (PMB) • IMS shall identify “significant” external dependencies (e.g., GFE, test facilities) • Suppliers and government playing a bigger role in producing products • IMS shall be statused and submitted prior to or concurrently with Contract Performance Report (CPR) • Standard scheduling definitions are to be used Section J Award Fee Plan • Government Program managers often wish to tie Contractor award fee to EVM performance – Looks like a score and is advertized as an “objective” measure • But... – Data can be manipulated and may only be discerned by experienced practitioners – If you reward good data, you will get good data • Care should be taken not to mis-incentivize the supplier (Contractor) – Award fee criteria should reference effective management of cost and schedule with EVM data – Criteria should not reference specific values for CV, SV, CPI, SPI, or VAC Award Fee is for Results / Deliveries, not for EVM Scores Section L Instructions for Offerors • Guidance to bidders for the assembly of their proposals • Each offeror’s proposal shall include a description of the EVMS to be used in accordance with the appropriate RFP FAR clauses • Compliance with Validation requires: – Reference to an Advance Agreement, Letter of Acceptance, etc. and a copy of the approved EVM system description; or – A plan to obtain EVM validation to include how and when the system will be validated – Compliance Only requires: a written summary of the proposed EVMS reference in sufficient detail to show how it addresses all ANSI/EIA-748 guidelines Contractor’s Proposed EVMS Addressing the EVMS RFP Requirement • Describes system in sufficient detail to determine compliance • Includes prior Certification/Acceptance • If no EVMS in place, the Contactor must submit a comprehensive plan for Compliance in accordance with FAR/DFARS – Describe the EVMS the offeror intends to use in performance of the contract, and how the proposed EVMS complies with ANSI/EIA 748 – Distinguish between the offeror’s existing management system and modifications proposed to meet the EVMS guidelines – Describe the management system and its application in terms of the EVMS guidelines – Describe the proposed procedure for administration of the EVMS as applied to subcontractors – Describe the process the offeror will use to determine subcontractor compliance with ANSI/EIA 748 • Identify major subcontracts for application of criteria – Verify baseline integrity is maintained Section M Evaluation Factors for Award • Evaluation of the proposed EVMS is normally undertaken as part of the proposal evaluation process to determine the probability of the system meeting the guidelines. – For Existing Systems - evaluation may consist of a confirmation that the referenced validation is accurate and current. The system should be currently in use, and surveillance should not have identified significant, uncorrected problems. – For EVMS (without validation) - the EVM system description should be evaluated for completeness against the guidelines in ANSI/EIA-748. • An on-site examination of a proposed EVMS should not normally be required during proposal evaluation. But if deficiencies are identified, written communications or an on-site visit may be required when approved by the Source Selection Board and Procuring Activity. • The Cognizant Federal Agency should be requested to provide insight regarding EVMS capability, quality, and past performance. Post Award Responsibilities • Assess System Surveillance –Contractor surveillance plan in place –Contractor surveillance is being periodically conducted –Monitor planned activities vs actual activities • Ensure that program office is conducting program analysis on a periodic basis • Schedule, conduct and support periodic Integrated Baseline Reviews • Take corrective action when variances exist EVM and Fixed Price Contracts • OMB Circular A-11 and the FAR do not exempt specific types of contracts from the EVM requirement – Applies to fixed-price, time and material, as well as cost plus • Fixed-price contracts are not a barrier to EVM – Certain guidelines may be not applicable • Indirect costs and material – Actual Costs for fixed-price contracts are simply the prices the customer pays for the work. – Contractor should develop and coordinate a schedule of values that relate performance to progress payments – Will not have a cost variance because ACWP = BCWP Subcontractors and Suppliers Must be Managed so they are Successful Summary • EVM considerations must begin in the early stages of solicitation planning. • Should be addressed in the acquisition strategy and acquisition plan • Post award activities (IBRs and EVMS certification) are especially critical and directly impact the effectiveness of EVM throughout the program lifecycle EVM is a Key Factor in Winning and Managing Federal Contracts Questions? BACKUP INFORMATION EVM Analysis Formulas & Metrics Metric Formula Purpose Cost Performance Index CPI = BCWP ACWP Measurement of cost efficiency of the work accomplished (Cumulative, Average or Current Period). DCMA expectation is => .95 Schedule Performance Index SPI = BCWP BCWS Measurement of work accomplished to the plan (Cumulative, Average or Current Period). DCMA expectation is => .95 Cost Variance CV = BCWP – ACWP Work accomplished cost more or less to complete than planned. SV = BCWP - BCWS Indication of more or less work accomplished than planned. Variance at Completion VAC = BAC – EAC The difference between what the job is planned to cost versus the estimated cost at completion. Percent Complete % Comp = BCWP (cum) EAC Provides percentage of the estimate at completion that has been accomplished Percent Complete % Comp = BCWP (cum )) BAC Provides percentage of the budget at completion that has been accomplished Percent Spent % Spent = ACWP (cum )) BAC Provides percentage of the budget at completion that has been spent to date Baseline Execution Index BEI = Total Tasks/Activities Completed Total Tasks/Activities Planned Measurement indicating efficiency of work accomplished against the baseline plan. DCMA expectation is => .95 Critical Path Length Index CPLI = Critical Path Length + or – Total Float Critical Path Length Calculation that indicates the realism of the program’s ability to complete the program (or major milestones) on time. DCMA expectation is => .95 Schedule Variance 10 March 2010 EVM Analysis Formulas & Metrics Metric Formula Purpose To Complete Performance Index BAC TCPI BAC = (BAC – BCWP (cum )) (BAC – ACWP (cum )) Future efficiency necessary to complete on budget To Complete Performance Index EAC TCPI EAC = (BAC – BCWP (cum )) (EAC – ACWP (cum )) Future efficiency necessary to complete on the current estimate at completion IEAC = ACWP + (BAC – BCWP) CPI Cum) Statistical estimate of the lowest value. It assumes that the remaining work will be performed at the same cost efficiency as the past cost efficiency Independent Estimate at Completion – Minimum Independent Estimate at Completion – Maximum IEAC = ACWP + (BAC – BCWP) SPI (Cum) X CPI (Cum) Statistical estimate of the highest value. It assumes that the remaining work will be performed at a combination of the schedule and cost efficiencies. Weighted Independent Estimate at Completion IEAC = In this formula, the EAC is weighted by 80% of the CPI and 20% of the SPI. 10 March 2010 BAC [0.8 (CPI) + 0.2 (SPI)]