Army Cost Review Board Office Mort Anvari Mort.Anvari@US.Army.mil Steve Loftus Steve.Loftus@US.Army.mil 1 HQ Department of the Army Secretary of the Army Chief of Staff, Army Under Secretary of the Army Vice Chief of Staff, Army Assistant Secretaries of the Army Civil Works Financial Operations Financial Management & Comptroller Installations & Environment Cost & Economics Manpower & Reserve Affairs Budget Acquisition, Logistics & Technology) G8 2 DASA, Cost & Economics Director Acquisition Costing Chief, Cost Review Board Director Programs & Strategy Cost Management C4ISR Costing Weapon Systems Cost Policy & Research Unit Mission Costing Installation Costing 3 Cost Review Board (CRB) • The ASA(FM&C) Formed the Army Cost Review Board to review Cost Estimates for Major Weapon & Information Systems • Created in Response to the Need for: A Comprehensive Army Cost Position, Acceptable to Acquisition & Financial Management Communities & Supports PPBES The CRB Reviews Major Weapon & Information Systems at Their Critical Acquisition Decision Points. All ACAT I Programs & Programs of Special Interest Must Have4 a Recommended ACP Briefed to the CRB. Cost Analysis Requirement MS I MS 0 O L D Concept Exploration MS II MS III Program Definition & Risk Reduction Engineering & Manufacturing Development (EMD) Production, Fielding/ Deployment Cost Estimates Needed A N E W C B Concept Exploration Component Advanced Development Review Concept & Technology Development Pre-Systems Acquisition System Integration System Demo Production Readiness & LRIP Review System Development& Demonstration Rate Production & Deployment Review Support Production & Deployment Systems Acquisition (Engineering and Manufacturing Development, Demonstration, LRIP & Production) Sustainment 5 Cost Estimating Process Definition Data Collect Planning Normalize Estimate Formulation Review/ Presentation Final Document DOCUMENTATION 6 Cost Analysis Model Data/CERs LCC Estimates Cost Drivers Develop Cost Estimate Structure and WBS Start Establish Ground Rules and Assumptions Compile Data Base/ CERs/Models Prepare Cost Estimates for Each Element Test Total System Estimate Engineering Analogy Parametrics Expert Opinion Reasonableness Sensitivity Analysis Cost-Risk Assessment Prepare Documentation 7 Situation • You have just been tasked to develop a cost estimate, that is, a professional opinion about the cost of an item, a service or a thing. • Let’s discuss a process for organizing and developing this estimate. 8 Definition and Planning • Influences the success of the estimate • Understanding the requirements and how you approach the process will establish the guidelines and procedures for the estimate. • Ask lots of questions…They help you understand the requirement. 9 Questions • Why is this cost estimate needed? • What decisions are pending on the results of this estimate? • Will the estimate be briefed and to whom? • Will the results be incorporated into some document? • What does the recipient expect to have included or excluded? • What excursions or variations from the baseline are anticipated? 10 Questions Continued • What are the program and funding constraints especially if the program is a Joint Program? • What are the time constraints for this estimate? • What is the acquisition phase of the program? • Is the program definition mature? • Does technology exist today to design, develop, test and manufacture the system? • What is the interrelationship with other systems? • Are there previous contracts? How many? What type? • How have the contractors performed to date? 11 Definition and Planning Know Purpose of the Estimate • Main purposes of estimates: – Budget Formulation – Comparative Studies – Source Selection 12 Purpose of Estimate Budget Formulation Estimates • • • • • • • Program Office Estimate (POE) Component Cost Analysis (CCA) Independent Cost Estimate (ICE) What-if exercises Rough Order of Magnitude (ROM) Should Cost Estimates CAIV 13 Purpose of Estimate Comparative Studies Estimates • Making cost & benefit comparisons between alternatives – Economic Analysis (EA) – Analysis of Alternatives (AoA) – Force Structure – Trade-off Studies – Source Selection – Prioritization 14 Definition and Planning Defining the System • Adequate description of the technical and program characteristics of the system • What are the physical and performance characteristics? • What are the development, production, and deployment schedules? • How many systems are to be produced? • How will the systems be supported: contract, inhouse, two or three levels of maintenance? 15 Defining the System Integrated System Schedule 1993 ACTIVITY 1994 1995 1996 1997 TODAY 1998 LLTI PEO IPR PROGRAM REVIEWS ENGINEERING & MANUFACTURING DEVELOPMENT LETHALITY ENHANCEMENT LRIP DECISION MRRB TECH SUPPORT DEVELOPM ENT QL TEST & EVALUATION UE FT T QUAL IOTE LIVE FIRE TEST LUT (3)(6) LFT (A) LFT (26) (B,C) (8,4) LOG DEMO II PVT (25 MS L + 10 CLU) ENHANCED PRODUCIBILITY PROGRAM LOW RATE INITIAL PRODUCTION FY96 (28) FY98 FY99 FIELDING AWE QL FTT UE LUT CA MRRB (9) CA MSLS - 698 CLUs 55 MSLS - 872 CLUs 97 MSLS - 1015 CLUs - 108 ARMY M SLS - 1020 CLUs - 206 M SLS - 1080 CLUs - 270 M SLS - 3316 CLUs - 423 LEGEND MSL CLU LLTI CA LEAD TIME PRODUCTION LEAD TIME CA - QUICK LOOK FIELD TACTICAL TRAINER USER EVALUATION LIMITED USER TEST CONTRACT AWARD MATERIEL RELEASE REVIEW BOARD PRODUCTION LEAD TIME CA MULTIYEAR I & II (3 YR CONTRACT) FY97 II, III, FLIGHT I LD II PROD VER TEST FY95 2000 MILESTONE III LITE I / WARHEAD / LOW COST BST FY94 1999 J A S O ND J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D PRODUCTION CA USM C 141 48 194 140 741 133 LEAD TIME PRODUCTION CA M YII CA RANGER 82nd COMPL FIELDING BEGINS HANDOFF FT. HOOD NTC FUE USA 82nd COMPLETE FUE USMC 16 Defining the System Work Breakdown Structure (WBS) • WBS defined in MIL-HDBK-881 as: – product-oriented breakdown of hardware, software, services, data and facilities that define the system. • WBS breaks a total job down into manageable pieces & portrays the way work is to be done. • WBS displays a company’s reporting structure. • Program managers may cite MIL-HDBK-881 “for guidance only” in contract solicitations. 17 MIL-Handbook-881 (WBS) 18 Defining the System Cost Element Structure (CES) 1.0 Research Development Test & Evaluation (RDT&E) 2.0 Production 3.0 Military Construction (MILCON) 4.0 Military Pay and Allowances (MPA) 5.0 Operating and Maintenance Army (OMA) 19 Defining the System COST ELEMENT STRUCTURE – 1.0 RDT&E CES# ELEMENT: 1.01 1.02 1.03 1.04 1.05 1.06 1.07 1.08 1.09 1.10 1.11 DEV. ENG. PEP DEV. TOOL. PROTO MFG. SEPM SYS T&E TRAINING DATA SUPP EQUIP. DEV. FACILITIES OTHER FY00C$M $ 39.039M 0.408 0.457 110.421 78.266 11.112 1.989 3.439 4.897 0.0 0.968 TY$M $38.260M 0.386 0.450 107.724 76.363 10.927 1.954 3.413 4.758 0.0 0.928 20 Defining the System COST ELEMENT STRUCTURE – 2.0 PROCUREMENT CES# ELEMENT: 2.01 2.02 2.03 2.04 2.05 2.06 2.07 2.08 2.09 2.10 2.11 2.12 2.13 2.14 NON REC PROD. REC. PROD ENG. CHG SEPM SYS T&E TRAINING DATA SUPP. EQUIP. OPER./SITE/ACT. FIELDING TRAIN. AMMO/MSLS WAR RESV. MODS OTHER FY00C$M $ 16.110M 1,169.348 0.0 116.637 12.564 28.880 2.073 146.460 0.0 89.525 59.001 0.0 236.619 51.739 TY$M $ 16.583M 1,312.377 0.0 132.258 14.437 31.499 2.300 158.304 0.0 101.635 79.482 0.0 280.729 64.129 21 Defining the System COST ELEMENT STRUCTURE – 3.0 MILCON CES# 3.01 3.02 3.03 3.04 ELEMENT: DEVELOP. CONSTRUCTION PRODUCT. CONSTRUCTION OPERATION/SITE ACTIVATION OTHER MILCON FY00C$M TY$M 22 Defining the System COST ELEMENT STRUCTURE – 4.0 Military Pay&Allowances (MPA) CES# 4.01 4.02 4.03 4.04 4.05 4.06 ELEMENT: FY00C$M CREW MAINTENANCE SYSTEM SPECIFIC SUPPORT SEPM REPLACEMENT PERSONNEL OTHER MPA TY$M 23 Defining the System COST ELEMENT STRUCTURE – 5.0 Operating & Maintenance Army (OMA) CES# 5.01 5.02 5.03 5.04 5.05 5.06 5.07 5.08 5.09 5.10 ELEMENT: FY00C$M FIELD MAINT., CIV LABOR SYS. SPECIFIC BASE OPS REPLENISHMENT DLRs REPLEN. CONSUMMABLES POL END ITEM MAINTENANCE TRANSPORTATION SOFTWARE SEPM TRAINING 5.11 OTHER OMA TY$M 24 Defining the System Cost Analysis Requirements Description (CARD) • Source of a system’s description • Describes important features • Is provided to other groups preparing cost estimates • Helps ensure all groups are costing out the same “program.” • Prepared by program office; approved by DoD Component Program Executive Officer 25 Defining the System Cost Analysis Requirements Description (CARD) Continued 1.0 System Overview 1.1 System Characterization 1.1.1 System Description 1.1.2 System Funcitonal Relationships 1.1.3 System Configuration 1.1.4 Government Furnished Equipment/Information 1.2 System Characteristics 1.2.1 Technical/Physical Description 1.2.1.1 Subsystem Description 1.2.1.2 Functional and Performance Description 1.2.2 Software Description 1.3 System Quality Factors 1.3.1 Reliability 1.3.2 Maintainability 1.3.3 Availability 1.3.4 Portability and Transportability 1.3.5 Additional Quality Factors 1.4 Embedded Security 1.5 Predecessor/Reference System 2.0 Risk 26 Defining the System CARD Continued 3.0 System Operational Concept 3.1 Organizational Structure 3.2 Basing and Deployment Description 3.3 Security 3.4 Logistics 4.0 Quantity Requirements 5.0 System Manpower Requirements 6.0 System Activity Rates 7.0 System Milestone Schedule 8.0 Acquisition Plan or Strategy 9.0 System Development Plan 10.0 Element Facilities Requirements 11.0 Track to Prior Card 12.0 Contractor Cost Data Reporting Plan 27 Definition and Planning Ground Rules & Assumptions • State the conditions which must take place in order for the estimate to be valid • Ground rules and assumptions must be documented since changes in these areas provide an audit trail for changes in the cost estimate. 28 Data Collection and Analysis • Collection and analysis represent a significant amount of the overall estimating task in terms of time. The analysis will include decisions on what programs to include in the data set to whether to truncate lot data on a program for which you are calculating a learning curve. • Document data in your analysis, and any assumptions you make 29 Data Collection and Analysis • The direction we take in collecting historical data will be determined by our choice of estimating methodologies. • This step may also dictate a change in estimating approach due to the availability or non-availability of certain data. • Data collection is not limited to cost data. We must also collect technical and program data if we want the total picture of the historical systems. This will help us ensure the comparability of the systems that we are collecting data on with the system we are estimating. 30 Data Collection and Analysis Most Difficult Task in Cost Estimating • Data Sources – Data Types: Cost/Resource, Technical, Program – Categories: Primary, Secondary • Data Problems – Wrong Format – Matching up – Definition – Temporal Factors - comparability • Normalization • Data Location 31 Army Tools for Total Ownership Cost Estimating Life Cycle Cost Management Tools APPN ACEIT PRICE SEER ACDB AMCOS OSMIS SBC/ISR CO$TAT RDT&E X X X X X X PROC X X X X X X MILCON X X X X X MILPERS X X X X X O&M X X X X X X X 32 Definition and Planning Select the Estimating Approach • Techniques available – Analogy – Parametric – Engineering – Extrapolation – Expert Opinion Select the technique that is most applicable to a specific WBS element 33 Definition and Planning Estimating Methods • Analogy – Basic Comparison – Factors • Parametric – Regression Analysis • Engineering – Detailed • Expert Opinion – Committee – Delphi 34 Life Cycle Cost Estimates Concept Refinement Technology Development A System Development & Demonstration B Production & Deployment Operations & Support C Disposal D Cost Estimating Model x y 0 • • • • • • • Parametric Engineering Actuals x y 0 Parametric Engineering Actuals x y z Parametric Engineering Actuals x y z Parametric Engineering Actuals 0 y z Parametric Engineering Actuals Cost estimates based on confidence intervals Parametric analysis based on similar systems and similar attributes (regression) Engineering data based on reliability projections used for bottoms-up estimate Actual system costs used to extrapolate future system costs Cost estimate revised every two years after production Weights associated with non-actuals decrease as system matures MS B is a true hard stop for systems 35 Cost Estimating Methods Analogy Method • Based on direct comparison with historical information of similar existing activities, systems, or components. • Compares new system with one or more existing similar systems where there is accurate cost and technical data. • Analyst must show validity of comparison. 36 Cost Estimating Methods Analogy Method • Based on known costs of a similar program • Adjustments for complexity, technical, physical • Strengths – Based on representative experience – Less time consuming than others – Can be used as a check on other techniques • Weaknesses – Small sample size – Heavy reliance on judgment – Sometimes difficult to identify analogy and associated costs 37 Analogy Estimating with Factors Cost(New) = Cost(Old) x Adjustment Factor Element Old Sys1 Old Sys2 Old Sys3 New Sys Airframe $500/lb 1.25*S1 Engine 2M/Unit 3M/Unit 5M/Unit .8*S3 Avionics $3K/lb 1.0*S2 Payload 6M/Unit 8M/Unit 7M/Unit $250/lb $2K/lb $750/lb $4K/lb .65*S1 38 Cost Estimating Methods Parametric Method • Known as Statistical Method or Top Down Method • Relates cost to physical attributes or performance characteristics • Uses database of elements from similar systems • Uses multiple systems • Most beneficial in earlier stages of the system or project life cycle 39 Cost Estimating Methods Parametric Method • Statistical relationships between cost and physical or performance parameters of past systems. • Strengths – Captures major portion of cost – Quick what if type estimates • Weaknesses – Less detailed – Getting accurate data 40 Cost Estimating Methods Parametric Method (Extrapolation) • Use historical values to establish a trend for the future. • Example problem: Given the actual productivity and labor rates in the given table. How much will it take to complete a 3-year software development project of 10K lines of code, if 50% is completed in the second year and 25% is completed in first and third years? 41 Cost Estimating Methods Engineering Method • Known as bottom up method • Requires extensive knowledge of system characteristics • Divide into segments; estimate costs for each segment • Combine segments plus integration cost • Uses a combination of cost estimating methods • Detailed knowledge of new technologies may not be available. 42 Cost Estimating Methods Engineering Method • Strengths – Detailed • Best when long stable production process • Weaknesses – Requires a lot of time – Cost – Cannot be used until system well defined 43 Cost Estimating Methods Expert Opinion Method • Subjective judgment of an experienced individual or group • Use if time does not permit a more thorough analysis • Document source(s) of opinion of experts • List attributes of the source(s) Example: Delphi Technique 44 Cost Estimating Methods Expert Opinion Method • Consulting with one or more experts who use their knowledge and experience to arrive at an estimate • Group techniques include – Consensus (Committee) – Delphi • Strengths and weaknesses 45 Expert Opinion Method Delphi Technique • • • • • Query expert opinion from group Seek information from each expert Summarize the results Send report to each expert Gather second opinion after each individual reviews report • Summarize results • Iterative process continues until the experts reach a consensus, or near-consensus. 46 Expert Opinion Method Example Labor type Hours Needed % of Total Hourly Rate Hrs %*rate Senior Engineer 1000 $13 10.5 $1.37 Design Engineer 3000 $11 31.6 $3.48 Tool & Die 500 $11 5.3 $.58 5000 $9 52.6 $4.73 Machinist Totals 9500 $10.16 47 Learning Curve Theory As the quantity of a product produced doubles, the man-hours- per-unit expended to produce the product decreases at a fixed rate or constant percentage (usually 10% to 15%). 48 Learning Curve Theory Factors Contributing to Efficiency • Job familiarization by both production workers and supervisory personnel. • Changes in product design which do not materially affect the product, but result in increased ease and speed of production. • Changes in tooling, machinery, and equipment which simplify or speed up the production process. • Improved production planning and scheduling, and improvements in production techniques and operational methods. • Improvements in shop organization, engineering coordination and liaison. • Improvements in the handling and flow of materials, and in the materials and parts supply systems 49 Learning Curve Theory The table is based on the assumption that the first unit required 100 person-hours to produce. The table indicates a constant rate of reduction of 20% for each doubling of the unit number; the value of the second and each succeeding item in the table is 80% of the value of the preceding item. TABLE FOR FIGURES F-1-1 and F-1-2 Unit No. 1 2 4 8 16 32 64 Unit Person-hours 100.00 80.00 64.00 51.20 40.96 32.77 26.21 50 Learning Curve Theory 80% Unit Curve on Arithmetic Paper 51 Learning Curve Theory 1000 Unit Cost (LOG) 100 90% 85% 80% 10 70% 1 1 10 100 Production Unit Number (LOG) 1000 10000 52 Learning Curve Theory Uses – Evaluating contract production costs. – Assessing impact of production interruptions, product changes and production rate change. – Rate of improvement experienced by a particular contractor on a prior product may be indicative of rate of improvement expected on new product of similar size, complexity, and construction. – Improvement curve pattern experienced in the production of past item can be extended to calculate costs of future items. 53 Estimate Formulation • We have defined our tasks, planned the estimate, assigned cost responsibilities, and performed data collection and analysis. • Here we apply our estimating methodologies and tools: develop the factors, analogies, CERs, and learning curves. • We will aggregate the various cost elements into development, production, and O&S estimates, fiscally spread the costs, and apply inflation. 54 Cost Estimating Methods Laying Out the Estimating Approach Aircraft System – sum of level II elements (cross-check with analogy) Air Vehicle – sum of level III elements Airframe – CERs Air Vehicle Software – Expert Opinion Propulsion – CERs Avionics – Analogy Armament – Catalog Price System Eng/Program Mgt – Factor of Air Vehicle System Test & Evaluation – Factor of Air Vehicle Training – Factor of Air Vehicle Data – Factor of Air Vehicle Peculiar Support Equipment – Factor of Air Vehicle Initial Spares – Factor of Air Vehicle 55 Review and Presentation • We want to ensure that the estimate is reasonable, realistic, and complete. • Reasonableness addresses areas such as: using appropriate and acceptable methodologies; presenting methodologies systematically; the use of relevant data; and ensuring that assumptions are valid and clearly stated. 56 Review and Presentation (Continued) • The realism test checks to see if the assumptions and ground rules are consistent with the statement of work and if the costs are in line with historical data. • We evaluate completeness by determining whether the estimate includes all the work stated in the request for proposal and whether the costs are traceable and reconcilable. 57 Risk and Uncertainty Analysis Risk Analysis Approaches • Detailed Network & Stochastic Effort • Discrete Technical, Schedule, and Estimating Risks • Detailed Monte Carlo Simulation (each WBS) • Bottom Line Monte Carlo Simulation • Add a Risk Factor/Percentage Detail Risk and Uncertainty Analysis Frequency Probability Distributions WBS Cost Distribution Total Cost PDF P.E. Sum PE + P.E. = Confidence + MLC (Mode) CDF 50% 20% P.E. + Many More Sum PE MLC Risk and Uncertainty Analysis Probability Density Function Forecast: Results= Cell B28 Frequency Chart 4,978 Trials Shown .024 117 .018 87.7 .012 58.5 .006 29.2 Mean = $2,198 .000 0 $1,250 $1,750 $2,250 $2,750 $3,250 Risk and Uncertainty Analysis Cumulative Distribution Functions Forecast: Results= Cell B28 Cumulative Chart 4,977 Trials Shown .995 4977 .747 .498 .249 .000 0 $1,250 $1,750 $2,250 $2,750 $3,250 Reasons for Risk Technical Programmatic Cost Schedule Physical Properties Material properties Material Availability Skill Requirement Sensitivity to Assumptions Sensitivity to Technical Risk Degree of Concurrency Sensitivity to Technical Risk Software Complexity Environmental Impact Integration Interface Contractors Stability Sensitivity to Programmatic Risk Sensitivity to Schedule Risk Sensitivity to Programmatic Risk Number of Critical Path Requirement Changes Funding Profile Estimating Errors Operational Environment Political Advocacy Estimating Errors 62 Cost Schedule Curve Parallel Effort More ECP Less Mature Design Typical Life Cycle Cost Fixed Cost Technology Outdate Min Optimal Development Schedule 63 Development Schedule Mean Cycle Time to IOC DoD Pre-1992 Starts 132 months (11 years) Post-1992 Starts 89 months (7.4 years) on-going F-22 216 months (18 years) IOC 2004 Comanche 264 months (22 years) IOC 2006 Commercial Boeing 777 54 months (4.5 Years) 64 Schedule Goals Commercial Drivers • Technology Drives Schedule • Constraint Schedule • Goal is ROI Maximization Cost Defense Commercial DoD Drivers • Funding Drives Schedule • Unconstrained Schedule • Goal is Cost Reduction Development Schedule 65 DoD Program Schedule Drivers Cost Funding Allocation Acquisition Process Program Execution Cycle Time Reduction Goal Development Schedule 66 Final Documentation • Provide the means for other analysts to get the same results that we have in our cost estimate. • Providing good directions and a clear trail to follow are essential in having an estimate that can be replicated. • Provide step-by-step documentation of the methodologies, supporting data, ground rules, and assumptions, equations, examples, etc., • Ability to interpret or evaluate someone else’s cost documentation is as important as ability to prepare good cost documentation. 67 Life Cycle Management Changes Current State ICD Defined, Refined, & Fairly Accurate S&T RDT&E Procurement OPA Construction Personnel MCA MPA Initial Cost Estimate M S A Revised Cost Estimate Revised Cost Estimate M S B Training Operations OMA/MPA Support Multiple bill payer drills to offset actual costs Program for Equipment Only compete procurement costs against Army Investment Strategy over the EPP. Impact of sustainment on Army program ignored. M S C O&S Cost Estimate SR R Disposal Program for O&S Costs Develop SSP Unbalanced process that emphasizes procurement over sustainment: Large unplanned costs Future State ICD/DOTMLPF Change Request B C D Develop Supportability Sustainability Plan DASC SSO LSO OO Rep Cost Estimating Model MM Rep II Rep RDT&E Procurement OPA Construction MCA Personnel MPA Initial Cost Estimate M S A Revised Cost Estimate Support OMA M S B Requirements TT TT Rep Revised Cost Estimate SR R Compete total cost against Army TOA extended over EPP. Disposal FOC Capabilities Based Programming Plan for Sustainment Training Operations Program for Equipment G-3 PAE Rep S&T Periodic Cost Estimate Tri-Chairs of WSR Disposal Resource Providers Integrated Capability (IC) Prioritizers Capability Providers II MM OO DASC M1 DASC M2 DASC MTV Core Teams POM for Sustainment C4ISR UEx UEy TOC Signal Bde BFSB MI Bde x x x x x x FCS G1/G3/G8/G4 A BCT Sustainment Fires UA AVN UA Maneuver Other MACOM HVY MED SBCT LT Sust UA (UEx) Sust UA (UEy) HVY LT UEy HVY MED LT AEB MUA Engineer Bde MP Bde X X x x x x x X x x x x x x x x x x x x 68 Balanced process that considers investment and sustainment: Informed decision makers … S* T* CAIV Process • Set realistic but aggressive cost objectives early in acquisition program • Manage risks • Track progress using appropriate metrics • Motivate Government and industry managers to achieve program objectives • Incorporate incentives to reduce O&S costs for fielded systems 69 Cost As An Independent Variable (CAIV) • Best time to reduce cost is early in the process. • Involves the stakeholders in the process. • Cost tradeoffs must be addressed early in the acquisition process and embedded in program requirement documents, Request For Proposals (RFPs), contract provisions, and source selection process. 70 Cost Analysis Domain Inputs (Descriptions) Requirements Acquisition Assumptions Design Parameters Risk Assessment Process (Models) Outputs (Costs) ASARC CAIG, DAB APB PPBES AOA Cost Analysis Process Databases, Tools, & Models Types of Cost Studies Current Capability Needed Capability Studies Should be Known Unknown Class Cost Estimating Descriptions, Models Costs Analysis CER Development Description, Historical Costs Models Synthesis CAIV Cost Goals, Models Descriptions ( Design and Performance Parameters) Control Performance Based and Design Based Cost Models Currently do not Exist 71 Analysis of Alternatives • Set realistic, aggressive cost objectives early in development • Manage risks • Track progress with appropriate metrics • Motivate government/industry managers to achieve program objectives • Incorporate incentives to reduce O&S costs for fielded systems. 72 Cost Analysis Art or Science? 73