Best Value Business Model Kenneth T. Sullivan PhD, MBA Performance Based Studies Research Group School of Sustainable Engineering and the Built Environment Ira A. Fulton Schools of Engineering Arizona State University www.pbsrg.com 1 Who we are: PBSRG Overview Established in 1994 by Dr. Dean Kashiwagi and Dr. Bill Badger Research effort entails: Information Measurement Theory (IMT): measuring of current conditions to predict future outcomes Clients’ implementation of Best-Value Business Philosophy to improve the efficiency of their organizations and projects/services Organizational Transformation Models Performance Information Procurement System (PIPS) Contracting & Project management model (alignment/leadership instead of management/influence) - PIRMS Performance Information Environment (minimize access and flow of information) for accountability Risk management by using deductive logic, minimization of decision making WWW.PBSRG.COM 2 PBSRG’s Research Results (Performance Based Studies Research Group) Worldwide as a leader in Best-Value Systems Conducting research since 1994 168 Publications 800+ Projects $4.6 Billion Services & Construction 5% Increase in Vendor profit 98% On-time, On-Budget, Customer satisfaction PMI, NIGP, IFMA, IPMA Tests in Netherlands, Botswana/Africa, Malaysia ASU – investments of over $100M due to BV WWW.PBSRG.COM Research Clients General Dynamics University of Minnesota General Services Administration (GSA) Heijmans, Netherlands Ministry of Transportation, Netherlands State of Alaska University of Alberta State of Oklahoma State of Idaho Idaho Transportation Department State of Oregon Neogard Tremco US Solar Arizona Parks and Recreation US Army Medical Command USAF Logistics Command University of New Mexico EVIT School District Arizona State University US Corps of Engineers Arizona Public Service (APS) Salt River Project (SRP) Rochester Public Utility Boise State University University of Idaho Idaho State University Lewis & Clark State College City of Phoenix, AZ City of Peoria, AZ City of Roseville, MN Olmstead County, MN Fann Environmental Brunsfield (Malaysia) Fulbright Program /University of Botswana, Africa US Embassy, Bank of Botswana RMIT, Melbourne Australia Aramark, Canon, Qwest, ISP, Chartwells, AP, Pearson Various Contractors and Consultants WWW.PBSRG.COM 4 Working Commission 117 & Journal International Efforts & Partners 5 years 15 tests for infrastructure Two major GCs Tongji University Brunsfield Complete Supply Chain Fulbright Scholar University of Botswana RMIT PIPS tests Teaching IMT PBSRG platform WWW.PBSRG.COM 5 What makes our research message unusual….. Simplistic Uses logic Efficiency: less decision making, less management, and better results (best value and high profits) It is more important for the vendor who does the work to know what to do than it is for client’s representative to know what the vendor should do Measurement WWW.PBSRG.COM 6 Industry Structure High Performance III. Negotiated-Bid II. Value Based Qualified vendors invited Owner selects vendor Negotiates with vendor Vendor performs IV. Unstable Market Best Value (Performance and price measurements) Quality control Vendor minimizes risk I. Price-Based/Traditional Specifications, standards and qualification based Management & Inspection Client minimizes risk Low Competition WWW.PBSRG.COM High 7 Impact of Minimums & Expectations High Low High Low Vendor 1 Vendor 2 Vendor 1 Vendor 2 Vendor 3 Vendor 4 Vendor 3 Vendor 4 Low High Low High WWW.PBSRG.COM 8 Perception Problems with Traditional Systems Owners Vendors “The lowest possible quality that I want” “The highest possible value that you will get” High High Maximum Minimum Low Low WWW.PBSRG.COM WWW.PBSRG.COM 10 Traditional Management Initial conditions D1 Final conditions M&C Laws Laws Time D1: Client makes decisions on budget, time, and expectation D2: Client consultant/professional makes more decisions to make expectations true D2 D3 D3: Vendors attempt to use the lowest possible price to minimize the risk caused by the decision making of client & consultant/professional M&C: The client attempts to force vendor to make expectations happen WWW.PBSRG.COM Project Management Model Initial conditions C1 Final conditions M3 M2 Laws Laws Time C1: Client Expectations based decisions and various factors – may or may not be “realistic” M1: Measured vendor plan that more accurately describe the M1 initial conditions replaces C1 – converts to a predictive contract M2: RMP/WRR measures deviation & performance to plan M3: Final performance measurement WWW.PBSRG.COM Inefficiency vs Efficiency Can you make the transition? Micro-Management Performance dictated by technical information Specification is the requirement Inspection by client Client’s professional is the expert and has control No performance measurements Increase flow of information Relationships (partnering, deals, give and take) used to solve issues Need more people (inefficient) No accountability Leadership Performance dictated by performance information Specification is only the intent Quality control by vendor Vendor has control Performance measurements Decrease flow of information High performance vendors used to minimize risk Need less people (efficient) Accountability WWW.PBSRG.COM 13 Performance-Based Functions Performance High Low III. Negotiated II. Performance-Based IV. Unstable Market Value & Performance Maximize Profit Vendor Accountability Minimized Management & Inspection Quality Control Vendor minimizes risk I. Price-Based Treat as a Commodity Volume Based No Accountability Finger Pointing Management & Inspection Minimum Standards Client minimizes risk Competition High WWW.PBSRG.COM 14 Best Value Overview Complete business model for organizations & projects A best value selection and management tool (developed and tested over 16 years) It can be applied to any type of system, organization, structure, procurement, project, or need Best Value is not just a procurement method. It is a selection and management tool that can be applied in: Business Services (IT, dining, consultants, equipment, doc mgmt, insurance, etc.) Facility Services (maintenance, roofing, janitorial, landscaping, supplies, etc.) Design, bid, build (DBB), Design build (DB), Construction manager at risk (CMAR) A/E & Design, Job Order Contracting (JOC), Indefinite Delivery Indefinite Quantity BV is not a computer software package, but rather a combination of IMT principles that allows a client to optimize their environment WWW.PBSRG.COM 15 What does the Best Value Model do? Makes things simple (measurement, dominant information) Minimizes the fuel of bureaucracy (decision making, non-dominant information, management, control, and direction) Creates transparency Allows organizations/vendors to be highly efficient and successful Proposes that to accurately identify what “is” and then to have a plan to efficiently meet the needs will minimize risk 16 WWW.PBSRG.COM Best Value System: PIPS & PIRMS PHASE 1 Identification of Potential Best-Value PIPS Performance Information Procurement System PHASE 2 PHASE 3 Pre Planning and Risk Management Measurement of Deviation from the Expectation PIRMS Performance Information Risk Management System 17 WWW.PBSRG.COM 17 What is the model? Identify the expert with as little effort as possible, using measurement and differential Transfer risk and control to the expert through preplanning and risk minimization, focusing on risk that are not controlled Hire the expert Use alignment, planning, & measurement in place of management, control, and direction Create a performance information environment to drive accountability and change Proactive vs. Reactive Supply chain (us mentality) Logic vs. Experience Predictable vs. Chance WWW.PBSRG.COM 18 What are we trying to accomplish from a procurement perspective? Question: If Purchasing wants to buy a “green circle”, in which scenario is hiring the right “green circle” easiest to justify? Scenario 1 Scenario 2 WWW.PBSRG.COM BV Process Filter 1 Filter 2 Past Performance Information Current Capability Filter 3 Filter 4 Filter 5 Filter 6 Interview Prioritization Cost PreKey Personnel (Identify Reasonableness Planning & Risk Min Best Value) Low Award Quality of Vendors High Measurement of Risk & Performance During the Contract Time WWW.PBSRG.COM 20 Evaluation Criteria Past Performance Information Scope Plan Technical Risk Plan Risk Assessment & Value Added Plan (RAVA) Transition Schedule Financials/Cost Interviews WWW.PBSRG.COM Assessment: based on actuals No 1 2 3 4 5 6 7 8 9 10 No 1 2 3 4 5 6 7 8 9 10 Summary Criteria RAVA Plan Transition Milestone Schedule Interview Past Performance Information - Survey Past Performance Information - #/Clients Past Performance Information - Financial Financial Rating Financial Return - Commissions Capital Investment Plan Equipment Replacement Reserve Out of 10 10 25 10 Raw # 10 10 Raw $ Raw $ Raw $ Finanical Totals Summary Criteria Weight/Out of RAVA Plan 28 Transition Milestone Schedule 2 Interview 25 Past Performance Information - Survey 9 Past Performance Information - #/Clients 1 Past Performance Information - Financial 15 Financial Rating 5 Financial Return - Commissions 7 Capital Investment Plan 6 Equipment Replacement Reserve 2 100 $ $ $ $ A 5.91 5.17 15.77 9.80 5.67 7.02 4.00 30,254,170 14,750,000 7,213,342 52,217,512 A 16.55 1.03 15.77 8.82 1.00 10.53 2.00 3.31 4.31 1.77 65.09 $ $ $ $ Vendor B 7.09 6.96 16.78 9.99 3.00 8.67 8.00 60,137,588 20,525,000 4,100,001 84,762,589 $ $ $ $ Vendor B 19.85 1.39 16.78 8.99 0.53 13.01 4.00 6.58 6.00 1.00 78.13 WWW.PBSRG.COM C 6.31 6.33 13.53 9.82 4.42 6.90 8.00 64,000,000 12,340,000 8,171,811 84,511,811 C 17.67 1.27 13.53 8.84 0.78 10.35 4.00 7.00 3.61 2.00 69.04 Identifying the Potential Best-Value Best Best Value Value Prioritization Prioritization Best-Value is within budget Best-Value is the lowest price Best-Value is within [10%] of next highest ranked firm Yes Yes Proceed to highest ranked proposal within budget No Yes No Yes Yes Yes Yes Yes Best-Value can be justified based on other factors No Yes Yes Yes Proceed to Pre-Award No Yes Go with Alternate Proposal or Cancel WWW.PBSRG.COM 23 Best Value System PHASE 1 Identification of Potential Best-Value PHASE 2 PHASE 3 Pre Planning and Risk Management Measurement of Deviation from the Expectation WWW.PBSRG.COM 24 Project Management Model Initial conditions C1 Final conditions M3 M2 Laws Laws Time C1: Client Expectations based decisions and various factors – may or may not be “realistic” M1: Measured vendor plan that more accurately describe the M1 initial conditions replaces C1 – converts to a predictive contract M2: RMP/WRR measures deviation & performance to plan M3: Final performance measurement WWW.PBSRG.COM Traditional Risk Model V C B Buyer Controls Vendor Through Contract WWW.PBSRG.COM Best Value Risk Model V C B Vendor Manages/Minimizes Risk With Contract - Contract is predictive WWW.PBSRG.COM Pre Award Period What is it / Why is it important Period of time allotted to potential best value vendor (aka the Expert) to: Think about and preplan the project Set a plan for its delivery / clarify that your proposal is accurate Identify the risks and issues that could cause the plan to deviate Identify what you don’t know and when you will know it and how the plan could change based upon what you discover Set plans to minimize those risks from occurring Address all the concerns and risks of the client WWW.PBSRG.COM 28 Pre Award Period What is it / Why is it important Period of time allotted to potential best value vendor (aka the Expert) to: Know how they are being successful and adding value (measurement) What metrics you will use and how you will report them What is the current baseline condition we are comparing against Identify what you need from the client and have a plan for getting it Have completely aligned expectations between all parties so everyone knows what is going to transpire and what they are supposed to do WWW.PBSRG.COM 29 BV Process Filter 1 Filter 2 Past Performance Information Current Capability Filter 3 Filter 4 Filter 5 Filter 6 Interview Prioritization Cost PreKey Personnel (Identify Reasonableness Planning & Risk Min Best Value) Low Award Detailed Non-Detailed Non-Detailed Non-Detailed Non-Detailed Non-Detailed Quality of Vendors High Measurement of Risk & Performance During the Contract Time WWW.PBSRG.COM 30 Pre Award Document (Risk Management Plan) 1. Scope & Project/Effort Plan Clear and Detailed Project Scope (what is and what is not included) – Set Baseline Expectation 2. Cost or Financial Model 3. Milestone schedule (linked to performance benchmarks) 4. Risk Minimization Plan Uncontrolled Risks List A list of Risks Proposer does not control with plans to minimize Identified Risks List A list of all previously identified risks (by other bidders, user, and client) with plans to minimize 5. Client Action Item List 6. Weekly Risk Report Set Up 7. Performance Metrics 8. Other: Agreed to Value Adding Options, Original RAVA Plan, Interview Minutes, etc… WWW.PBSRG.COM 31 Award Filter 1 Filter 2 Past Performance Information Current Capability Filter 3 Filter 4 Filter 5 Filter 6 Interview Prioritization Cost PreKey Personnel (Identify Reasonableness Planning & Risk Min Best Value) Low Award Quality of Vendors High Measurement of Risk & Performance During the Contract Time WWW.PBSRG.COM 32 Measurement Filter 1 Filter 2 Past Performance Information Current Capability Filter 3 Filter 4 Filter 5 Filter 6 Interview Prioritization Cost PreKey Personnel (Identify Reasonableness Planning & Risk Min Best Value) Low Award Quality of Vendors High Measurement of Risk & Performance During the Contract Time WWW.PBSRG.COM 33 Weekly Reporting System Excel Spreadsheet that tracks only unforeseen risks on a project Client will setup and send to vendor once Award/NTP issued Vendor must submit the report every week (Friday). The final project rating will be impacted by the accuracy and timely submittal of the WRS WWW.PBSRG.COM 34 Management by Risk Minimization Unforeseen Risks RISK MANAGEMENT PLAN • Risk • Risk Minimization • Schedule METRICS • Time linked • Financial • Operational/Client Satisfac. • Environmental WEEKLY REPORT • Risk • Unforeseen Risks PERFORMANCE SUMMARY • Vendor Performance • Client Performance • Individual Performance • Project Performance WWW.PBSRG.COM Importance of Measurement Measurement is critical Measurement = accountability Accountability = improved performance, change Measurement allows definition of value (expertise) An expert is someone without risk Measurement is a mirror WWW.PBSRG.COM 36 Potential Vision Educate on IMT and Best Value methodology, tool, etc. Transfer and educate a new tool for procurement Transfer and educate techniques for preplanning and contracting Create a measured environment Measure projects and vendors Measure the university, organizations, groups, people, etc. Measurement = accountability = higher performance Integrated into network of similar partners all trying to become better Become educators and leaders of your area WWW.PBSRG.COM 37 How will you know when you have been successful Potential Goals Educated and certified Educating others Measured projects and services Risk Performance Comparison to baseline conditions Preplanning and risk management plans Creation of a leadership environment alignment Less client effort required Fewer lawyers needed Better vendors WWW.PBSRG.COM 38 Stones vs Bricks WWW.PBSRG.COM 39 Stones vs Bricks Best value is a leadership based model Leadership uses measurement and alignment of supply chain participants to optimize efficiency, minimize risk, and reduce costs Traditional approach Micro-management, direction, control, multiple layers of inspection Dependence upon the contract to ensure performance Best Value Approach Leadership, alignment, preplanning, and a supply chain perspective Uses of expertise, measurement, and accountability to optimize performance within the client’s constraints WWW.PBSRG.COM 40 The Event Initial conditions D1 Final conditions M&C Laws Laws Time D1: Client makes decisions on budget, time, and expectation D2: Client consultant/professional makes more decisions to make expectations true D2 D3 D3: Vendors attempt to use the lowest possible price to minimize the risk caused by the decision making of client & consultant/professional M&C: The client attempts to force vendor to make expectations happen WWW.PBSRG.COM Cycle of Learning Perceive Change 100% Information Process Apply WWW.PBSRG.COM 42 Processing Speeds All Individuals Process At Different Speeds Perceive Change 100% Information Process Apply WWW.PBSRG.COM 43 Q: How much do we know about everything? 0% Information This Much? 100% Information This Much? This Much? % Known WWW.PBSRG.COM 44 Proposal: We don’t know very much As for me, all I know is that I know nothing. - Socrates What we know 0% Information 100% Information What we don’t know % of Information WWW.PBSRG.COM 45 Q: How do we solve what we don’t know? (aka Risk) Do we use what we know to solve what we don’t know? OR Do we use logic to solve what we don’t know? What we know 0% Information 100% Information What we don’t know % of Information WWW.PBSRG.COM 46 A: Logic can be applied without knowledge/experience Logic 0% Information 100% Information What we don’t know % of Information WWW.PBSRG.COM 47 What is the model? Identify the expert with as little effort as possible, using measurement and differential Transfer risk and control to the expert through preplanning and risk minimization, focusing on risk that are not controlled Hire the expert Use alignment, planning, & measurement in place of management, control, and direction Create a performance information environment to drive accountability and change Proactive vs. Reactive Supply chain (us mentality) Logic vs. Experience Predictable vs. Chance WWW.PBSRG.COM 48 Case Studies WWW.PBSRG.COM 49 Overview of SHIP Test Objectives Establish contract with SHIP provider for college students in Idaho BSU Objectives: Maximize value (performance and cost) of SHIP Have an environment of risk minimization and performance measurement Minimize client effort in selection and management Minimize decision making Education of PIPS Measurement of differential BSU would like to create a “consortium” of universities/colleges in Idaho for a single SHIP contract WWW.PBSRG.COM 50 Deliverables Major project deliverables include: 1. 2. 3. 4. 5. 6. 7. 8. 9. Set and Educate Project core team and Set BSU Strategic Plan Capture current level of performance and cost Plan providers Cost structure Program structure and details Identify differentials, gaps, and overlaps Create RFP Educate Vendors Run Selection and Interviews Run Pre-Planning and Risk Management Award & Transition Establish and maintain measurement system WWW.PBSRG.COM 51 Overview Create a statewide Student Health Insurance Plan (SHIP) consortium Boise State University (BSU) Idaho State University (ISU) Lewis-Clark State College (LCSC) 3-Year Contract | $36 Million Measurements of Success 1. Reduce internal University program administration costs 2. Maintain or increase Customer Satisfaction (University & Students) 3. Maintain or increase cost-effectiveness of program to students WWW.PBSRG.COM Stat What Should We Include In RFP? Request For Information (RFI) General request to vendors Ask vendors what information they need to see in the RFP to create and provide an accurate proposal Has no contractual implications, just providing information to the client Filter 1 Past Performance Information RFP Filter 3 Interview Filter 4 Prioritize (Identify Best Value) Filter 5 Pre-Award Phase (Pre-Plan) Filter 6 Weekly Report & Post-Rating High Quality of Vendors RFI Filter 2 Proposal & RAVA Plan Low Time WWW.PBSRG.COM Stat Selection Criteria & Weights Responding contractors were evaluated on: Premiums (Student, Spouse, Dependents) (200 Points) Interviews (350 Points) Program Administrator Claims Administrator Waiver Administrator Data Base Manager Marketing Manager Risk Assessment and Value Added (RAVA) plan (250 Points) Risk Assessment – ability to identify and minimize potential risk unique to this project Value Added Option – ability to add value to the project in terms of time, money or quality Scope Plan (50 Points) Concise synopsis of the work that will be performed (major tasks, steps, or work packages). Vendors impression of how they will achieve the objectives of the consortium Past Performance Information (150 Points) Firm Program Administrator WWW.PBSRG.COM Stat Summary of Proposal Submittal Filter 1 Past Performance Information Filter 2 Proposal & RAVA Plan Filter 3 Interview Filter 4 Prioritize (Identify Best Value) Filter 5 Pre-Award Phase (Pre-Plan) Filter 6 Weekly Report & Post-Rating Quality of Vendors High Proposal Includes: 1) 2) 3) 4) Cost/Financial Information RAVA Plan (3) Scope Plan (2) PPI Low Time WWW.PBSRG.COM Stat 55 Coverage/Plan Characteristics Consortium goal was to standardize coverage between all three University's (to maximum extent possible). However, deviations were made as necessary (BSU athletic coverage, ISU RX Coverage, Capitated Fee, etc) Consortium goal was to increase plan characteristics (to provide better coverage for students) NO CRITERIA BSU ISU LCSC CONSORTIUM 1 Deductible Per Academic Year (In-Network) $250 $250 $250 $250 2 Deductible Per Academic Year (Out-Of-Network) $500 $250 $250 $500 $100,000 $50,000 $50,000 $250,000 80% 80% 80% 80% 5 In-Network Max out of Pocket $4,000 No MOP No MOP $4,000 6 Out-Of-Network Coinsurance 50% 60% 80% 60% $6,000 No MOP No MOP $6,000 $400 None $500 $500* 3 Maximum Benefit (Standard) 4 In-Network Coinsurance 7 Out-Of-Network Max out of Pocket 8 RX Drug Coverage (Max) WWW.PBSRG.COM Stat SHIP Analysis NO 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 CRITERIA Cost - Overall Annual Cost to Consortium Cost - BSU Annual Student Premium (per-student per-year cost): Cost - BSU Annual Spouse Premium (per-individual per-year cost): Cost - BSU Annual Dependent Premium (per-individual per-year cost): Cost - ISU Annual Student Premium (per-student per-year cost): Cost - ISU Annual Spouse Premium (per-individual per-year cost): Cost - ISU Annual Dependent Premium (per-individual per-year cost): Cost - LCSC Annual Student Premium (per-student per-year cost): Cost - LCSC Annual Spouse Premium (per-individual per-year cost): Cost - LCSC Annual Dependent Premium (per-individual per-year cost): Interview Rating - The Program Administrator Interview Rating - The Claims Administrator Interview Rating - The Waiver Administrator Interview Rating - The Data Base Manager Interview Rating - The Marketing Manager RAVA Plan Rating Work Plan Rating PPI - Firm - Satisfaction with the associated costs of the service PPI - Firm - Satisfaction with the benefits provided by the service PPI - Firm - Ability to manage the service / program PPI - Firm - Ability to document and provide accurate reports PPI - Firm - Overall customer satisfaction PPI - Firm - Number of different projects PPI - Firm - Number of different customer responses PPI - Administrator - Satisfaction with the associated costs of the service PPI - Administrator - Satisfaction with the benefits provided by the service PPI - Administrator - Ability to manage the service / program PPI - Administrator - Ability to document and provide accurate reports PPI - Administrator - Overall customer satisfaction PPI - Administrator - Number of different projects PPI - Administrator - Number of different customer responses DETAILED WEIGHTS 50 38 6 6 38 6 6 38 6 6 175 70 35 35 35 250 50 14 14 14 14 14 14 14 7 7 7 7 7 7 7 FIRM A FIRM C FIRM D FIRM E FIRM F $12,237,529 $1,772 $2,200 $1,886 $1,267 $1,660 $1,424 $1,228 $1,626 $1,394 6.7 6.6 5.8 5.2 7.8 7.42 6.67 9.7 9.9 9.9 9.8 10.0 10.0 10.0 9.7 9.9 9.9 9.8 10.0 10 10 $11,051,451 $1,552 $3,992 $2,249 $1,185 $3,048 $1,717 $1,244 $3,200 $1,803 7.7 6.1 7.6 5.0 6.4 6.25 7.17 9.1 9.5 9.8 9.9 9.8 20.0 20.0 10.0 10.0 10.0 10.0 10.0 10 10 $11,437,893 $1,685 $2,865 $2,444 $1,113 $2,827 $1,209 $1,298 $3,300 $1,411 7.1 4.6 5.4 3.9 5.0 7.42 6.33 9.9 10.0 10.0 9.9 9.8 9.0 9.0 9.8 9.9 9.9 9.7 9.7 10 10 $12,928,466 $1,806 $3,066 $2,124 $1,394 $2,937 $2,038 $1,484 $3,066 $2,124 7.4 5.3 6.0 4.6 8.1 5.58 5.50 10.0 10.0 10.0 10.0 10.0 10.0 10.0 10.0 10.0 10.0 10.0 10.0 10 10 $13,063,235 $1,914 $3,248 $3,616 $1,259 $3,627 $1,551 $1,615 $2,222 $2,310 7.4 8.3 6.0 4.6 8.1 5.17 5.58 10.0 10.0 10.0 10.0 10.0 10.0 10.0 10.0 10.0 10.0 10.0 10.0 10 10 WWW.PBSRG.COM Stat Analysis of Proposals Total Score: NO CRITERIA 923 916 886 831 840 FIRM A FIRM C FIRM D FIRM E FIRM F 1 Cost - Average Student Premium $1,422 $1,327 $1,365 $1,561 $1,596 2 Cost - Average Spouse & Dependent Premium $1,698 $2,668 $2,343 $2,559 $2,762 3 Average Interview Rating 6.4 6.6 5.2 6.3 6.9 4 RAVA Plan Rating 7.4 6.3 7.4 5.6 5.2 5 Work Plan Rating 6.7 7.2 6.3 5.5 5.6 6 PPI - 1-10 Rating 9.9 9.7 9.9 10.0 10.0 7 PPI - Number of projects and clients 10 17 9 10 10 WWW.PBSRG.COM Stat Overall Best-Value Results • Previous Program: – Student Premiums increased $124/year (past 4 years) – Spouse & Dependent Premiums increased $126/year 2006-2007 2007-2008 2008-2009 2009-2010 Average Increase Per Year ($) Student $1,012 $1,182 $1,263 $1,385 $124 11% Spouse & Dependent $1,843 $2,022 $2,104 $2,220 $126 6% School Premiums Average Increase Per Year (%) Best-Value Results: Student Premium has decreased by 2% (-$26) Spouse & Dependent Premium has decreased by 19% (-$519) In general, Benefits/Coverage have been increased WWW.PBSRG.COM Stat Case Study: ASU Food Services Contract $32 Million Dollars (Over 10 Years) No 1 2 3 4 5 6 7 8 9 10 Summary Criteria RAVA Plan Transition Milestone Schedule Interview Past Performance Information - Survey Past Performance Information - #/Clients Past Performance Information - Financial Financial Rating Financial Return - Commissions Capital Investment Plan Equipment Replacement Reserve Firm A (Incumbent) (1-10) 5.9 (1-10) 5.2 (1-25) 15.8 (1-10) 9.8 Raw # 5.7 (1-10) 7.0 (1-10) 4.0 Raw $ $ 30,254,170 Raw $ $ 14,750,000 Raw $ $ 7,213,342 Finanical Totals $ 52,217,512 Scale Firm B $ $ $ $ 7.1 7.0 16.8 10.0 3.0 8.7 8.0 60,137,588 20,525,000 4,100,001 84,762,589 WWW.PBSRG.COM Firm C $ $ $ $ 6.3 6.3 13.5 9.8 4.4 6.9 8.0 64,000,000 12,340,000 8,171,811 84,511,811 After 1 Year: Monitoring Based on Measurements Increase sale of food by 14% Increased cash to ASU by 23% Minimized management cost by 80% Increased customer satisfaction by 37% Increased capital investment by 100% 1 Total Revenue ($M) FY 06-07 FY 07-08 Difference % Difference Incumbent New Vendor $ 27.02 $ 30.83 $ 3.81 14% 2 Total Return & Commissions ($M) $ 2.17 $ 2.67 $ 0.50 23% 3 Captial Investment Contract ($M) $ 14.75 $ 30.83 $ 18.08 109% 4 Captial Investment 2006 vs. 2007 ($M) $ 0.26 $ 5.70 $ 5.44 2092% No Category 5 ASU Administration (# of People) 7 1.5 -5.5 -79% 6 Customer (Student) Satisfaction (1-10) 5.2 7.1 1.9 37% 7 Myster Shopper Satisfaction N/A 9.6 -- -- WWW.PBSRG.COM 2009 Performance Metrics No Catergory 1 2 3 4 5 6 7 8 9 10 11 Mandatory Meal Plan Sales - Meals ($K) (Meal Swipes) Voluntary Meal Plan Sales - Meals($K) (Meal Swipes) Retail Sales ($K) Catering Sales ($K) Camp/Conference Sales ($K) All Other Sales (Subcontractors & Sushi) ($K) TOTAL REVENUE Commissions on Total Revenue ($K) Subsidy - DPC, West & Polytechnic ($K) Commissions Paid to ASU ($K) (Commission less Subsidy) Commission % No 1 2 3 Catergory Number of Mandatory Meal Plans Sold Number of Voluntary Meal Plans Sold Customer (Student) Satisfaction Survery (1 - 10 (2x/yr) YTD Prior Year $ $ $ $ $ $ $ $ $ $ 8,915.5 294.2 15,408.1 2,329.1 3,030.9 29,977.8 1,902.3 391.7 1,723.3 6.35% Var Act. vs PY YTD Actual $ $ $ $ $ $ $ $ $ $ 8,212.2 404.1 17,320.4 2,526.5 865.2 3,807.2 33,135.6 2,011.3 1,381.6 629.7 6.07% YTD Prior Year YTD Actual 5,361 2,128 7.34 6,159 2,882 7.27 Var Act. vs PY % $ (703.3) -7.9% $ 109.9 37.4% $ 1,912.3 12.4% $ 197.4 8.5% $ 865.2 #DIV/0! $ 776.2 25.6% $ 3,157.8 10.5% $ 109.0 5.7% $ 989.9 252.7% $ (1,093.6) -63.5% -0.28% -4.35% Var Act. Actual vs PY vs PY % WWW.PBSRG.COM 798 755 -0.07 14.9% 35.5% -1.0% Performance Metrics – Combined ASU FY 2010 Financial Performance Metrics No Category YTD Prior Year ADJUSTED wk 6 Sep YTD Actual YTD Budget Var Act. vs PY $ 12,531.2 $ Var Act. vs PY % Var Act. vs Budget Var Act. Vs Budget % $ 6,694.8 84.6% $ 2,078.0 16.6% 264.7 $ (18.1) -4.6% $ 40.7% 1 Mandatory Meal Plan Sales - Meals ($K) (Meal Swipes) $ 7,914.4 $ 14,609.2 2 Voluntary Meal Plan Sales - Meals($K) (Meal Swipes) $ 390.5 $ 372.4 3 Retail Sales ($K) (Sun $, M&G Vol, M&G Mandatory and Cash $ &16,756.8 Credit Cards) $ 17,656.1 $ 22,697.4 $ 899.2 4 Catering Sales ($K) $ 2,475.5 $ 2,502.0 $ 2,733.4 $ 26.5 1.1% $ (231.4) -8.5% 5 Camp/Conference Sales ($K) $ 865.2 $ 822.1 $ 648.9 $ (43.1) -5.0% $ 173.2 26.7% 6 All Other Sales (Subcontractors & Sushi) ($K) $ 3,703.3 $ 3,793.8 $ 5,169.6 $ 90.5 2.4% $ (1,375.8) -26.6% 7 TOTAL REVENUE $ 32,105.7 $ 39,755.5 $ 44,045.1 $ 7,649.8 23.8% $ (4,289.6) -9.7% 8 Commissions on Total Revenue ($K) $ 2,011.2 $ 2,413.2 $ 2,673.5 $ 402.0 20.0% $ (260.4) -9.7% 9 Subsidy - DPC, West & Polytechnic ($K) Commissions Paid to ASU ($K) (Commission on Total Revenue less%Subsidy) Commission $ 1,202.3 $ 604.6 $ 1,157.6 $ (597.7) -49.7% $ (553.0) -47.8% 125.0% $ 292.6 19.3% 10 11 $ 803.7 $ 6.07% 1,808.5 6.07% $ 1,515.9 6.07% $ 1,004.9 107.7 5.4% $ (5,041.3) -22.2% Performance Metrics No Category YTD Prior Year YTD Actual YTD Budget Actual vs PY Var Act. Var Act. Var Act. vs Vs vs PY % Budget Budget % 1 Number of Mandatory Meal Plans Sold 6,133 7,573 7,843 1,455 23.6% -229 -2.9% 2 Number of Voluntary Meal Plans Sold 2,882 4,056 2,215 1,174 40.7% 1841 83.1% WWW.PBSRG.COM Overview of ASU Network The ASU Network consisted of: 64,000 Students and 12,000 Faculty 4 Different Campuses Estimated Cost: $11.1M Number of employees: 18 Full-time employees 8 Students 3 Contract technicians Main Technical Competencies: Core Infrastructure Design & Engineering Edge (Field) Technicians Project Management & Coordination Documentation Network Information/Operations Technology Solutions (network monitoring/measurement) WWW.PBSRG.COM Technology Options As Is University Benchmarking 100% $12M Standard Offering Option Set #1 Option Set #2 Value Add #3 Enhanced Value Add #3 100% 100% 100% 100% 100% $35.7M $35.7M $17M Capital Equipment Investment +15,000 Ports Gig Speed Capability of LAN Devices 7% VoIP Implementation w/new voicemail system +15,000 Ports 35% Over 5 Years 150 $26M +20,000 Ports 50% Over 5 Years 50% Wiring Upgrades $350K Investment In New Technologies 50% Initial Cabling Audit Call Center Application Upgrade & Maintenance Commissions to ASU Additional Services OfferedPM, HPC, Border & Data Center Mgmt 100% $31M +25,000 Ports +30,000 Ports (100%) 80% Over 5 Years 80% $750K 80% +30,000 Ports (100%) 100% in 18-24 100% in 18-24 Months Months 100% 100% $1.375M $1.375M 100% 100% 100% 100% Up to $300K Up to $300K WWW.PBSRG.COM 100% Final ASU IT Networking Contract ASU Maintenance Annual Cost Qwest Maintenance Annual Cost Total Annual Qwest Savings Total Qwest Annual Value Added and Savings $13,981,934 $12,500,000 1,481,934 2,756,934 ASU IT Networking previously performed in-house ASU IT Network Details 76,000 Students and Faculty 5 yr. Contract 4 Different Campuses ASU chose “As-Is” option WWW.PBSRG.COM Vendor Created Information Environment Old Operational Structure New Operational Structure Complicated Management Structure Single Management Structure “Seamless Organization” No measurement Fully measured No accountability Qwest responsible for entire operation Requires more labor 26% less labor WWW.PBSRG.COM Dominant Information Dominant Performance Indicators Overall cost of network Top of the line networking Network Sustainability/Accessibility Customer Satisfaction Dominant Measurements Documentation of Deviations to financials Year 1 Exp. Ex. Risk X Ex. Risk X Ex. Risk X Ex. Risk X New Year 1 Dev. Cap, Exp. Maint. FOE Costs Total $ 100,000 $ 100,000 $ 50,000 $ 25,000 $ 275,000 $ 4,100,000 $ 100,000 $ 100,000 $ $ 25,000 $ 4,325,000 $ 1,652,000 $ $ (25,000) $ 50,000 $ $ 1,677,000 $ 6,818,000 $ $ $ $ $ 6,818,000 $ 12,570,000 $ 100,000 $ 75,000 $ 50,000 $ 25,000 $ 12,820,000 Qwest ASU Current Value Add Overall Cost of Network Annual IT Spend Ratio (new vs maintenance) 17/83 48/52 Top-of-the-line Networking % Converged % Mobility % Equipment not out-of-date 7% 2% 58% 100% 100% 95% Network Sustainability/Accessibility % Equipment not needing replacement (Not at end-of-maintenance) 88% 100% Customer Satisfaction Speed/Quickness Available (Wired / Wireless): % 1Gb - Wired Connections % of 300Mb - Wireless Connections 59% 8% 98% 32% WWW.PBSRG.COM U of MN Objectives The UMN has a goal to be recognized as a top research institution in the world In 2005, CPPM partnered with the PBSRG (ASU) to implement the PIPS Best Value Process CPPM’s Objectives of the Best-Value Program are to: Contract to high performers Respond faster to customer needs Increase performance (on time, on budget, high quality) Increase efficiency of procurement (spend taxpayers money more efficient) Create a fair and open process for all vendors WWW.PBSRG.COM 69 69 CPPM Strategic Plan First organization to establish and follow a Strategic Plan Ultimate Goal: CPPM take over entire program and is successful in implementing and sustaining the program. Year 1 – Pilot Testing Year 2 – Evaluation and Continued Testing Year 3 – Expansion Year 4 – Expansion Year 5 – Infusion & Transition Year 6 – Transition WWW.PBSRG.COM 70 CPPM Strategic Plan Year 1 Identify and educate core group Identify qualified vendors Implement best-value Analyze pilot projects Year 2 Continue testing best-value Evaluate core group and refine Expand test to different trades (General Construction) Educate more internal CPPM staff Implement a weekly project tracking system Refine list of qualified vendors Educate and debrief qualified vendors on initial project results Year 3 Allow other CPPM personnel to test Automated online Directors Report Monitor all CPPM projects (LB & BV) Expand testing (A/E Services) Identify performance of UMN PM’s, Procurement, other critical areas, etc. Train CPPM on all BV components Year 4 CPPM acquire and perform all best-value functions (educate and train) PBSRG assist on areas of weakness CPPM handle analysis and tracking of all weekly reports Implement best-value on a larger scale Educate other UMN groups (Energy, Zones, Permitting, Codes, ect) WWW.PBSRG.COM 71 Transitional Plan No Activity Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 1 Education / Training / Debriefing 0% 0% 25% 50% 75% 100% 2 Data Collection (PPI) on Vendors 0% 0% 0% 0% 0% 0% 3 Proposal Analysis / Review / Rating 0% 25% 75% 75% 100% 100% 4 Modeling / Identification of Best-Value 0% 0% 0% 25% 50% 100% 5 Weekly Risk Reporting System 0% 0% 25% 25% 75% 100% 6 Directors Reporting 0% 0% 0% 0% 0% 100% 7 Documentation and Analysis 0% 0% 0% 25% 50% 75% 8 Modification / Evolution / Updating 0% 0% 0% 0% 0% 0% WWW.PBSRG.COM 72 Current Construction Results Award Analysis: Number of Best-Value Procurements: 161 Awarded Cost: $50.6M (11% below average cost) Average Number of Proposals: 4 Projects Where Best-Value was also Lowest Cost: 53% 85% of projects were awarded to vendor with highest / second highest RAVA Plan (7.3 vs 5.9) Performance Information: Contractor Impacts: 0% Change Orders / 4% Delay Vendor post project rating: 9.6 Average Contractor Increase in Profit: 5% WWW.PBSRG.COM 73 Program Report ViceDirector President DirectorOfficer 1 Procurement 1 PM PM 11 PM PM 22 DirectorOfficer 2 Procurement 2 PM PM 33 PM PM 44 Contractor 1 Contractor 3 Contractor 9 Contractor 4 Contractor 2 Contractor 6 Contractor 7 Contractor 8 Contractor 3 Contractor 1 Contractor 7 Contractor 9 Contractor 4 Contractor 8 Contractor 2 Contractor 2 WWW.PBSRG.COM 74 Report – Overall Program WWW.PBSRG.COM 75 Report - Directors WWW.PBSRG.COM 76 Report - End Users General Overview 1 2 3 4 Total Number of Projects Percent of Projects Procured Using PIPS Total Awarded Cost: Average Number of Risks per Project TEAM 1 (President / University / Admin) 19 79% $5,359,995 3 TEAM 2 Academic Health Center 14 86% $2,821,005 8 TEAM 3 Provost College 7.7% 0.6% 7.2% 63% 68% 41.3% 3.4% 37.8% 36% 50% 41.1% 20.0% 21.1% 80% 80% 8.1% 0.1% 8.0% 95% 79% 19.6% 0.1% 19.6% 93% 79% 14.8% -0.8% 15.6% 100% 60% 4 3 75% 6.75 7.7 10.7 2 2 100% 10 8.5 8.5 1 1 100% 10 8.0 7.0 5 80% $2,353,761 12 Owner Impacts 5 Overall Owner Impacts (Time & Cost) 6 Owner Change Order Rate 7 Owner Delay Rate 8 Percent of Projects without Owner Cost Changes 9 Percent of Projects without Owner Delays Contractor Impacts 10 11 12 13 14 Overall Contractor Impacts (Time & Cost) Contractor Change Order Rate Contractor Delay Rate Percent of Projects without Contractor Cost Changes Percent of Projects without Contractor Delays Satisfaction Ratings 15 16 17 18 19 20 Total Number of Completed Projects Total Number of Client Surveys Returned Percent of Projects Evaluated by Client Average PM Post Project Rating of Contractor Average Client Post Project Rating of Contractor Average Client Post Project Rating of CPPM WWW.PBSRG.COM 77 Report – Internal PM’s WWW.PBSRG.COM 78 Report - Contractors No 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Contractor Contractor 118 Contractor 119 Contractor 120 Contractor 104 Contractor 121 Contractor 105 Contractor 106 Contractor 122 Contractor 107 Contractor 123 Contractor 108 Contractor 124 Contractor 125 Contractor 109 Contractor 126 Contractor 110 Contractor 127 Contractor 128 Contractor 129 Contractor 111 Contractor 112 Contractor 113 Contractor 114 Owner Change Order Rate 721,965 0.3% 220,002 0.7% 269,850 9.4% 459,225 1.6% 241,575 0.0% 1,611,015 0.3% 1,280,362 2.2% 367,650 0.0% 178,440 0.0% 3,227,182 14.9% 327,295 0.0% 69,218 3.5% 1,150,738 1.9% 534,095 2.0% 323,000 3.3% 308,882 1.2% 1,793,355 3.8% 2,956,800 1.3% 1,319,789 2.2% 1,096,707 0.1% 446,100 0.0% 552,815 5.1% 1,841,157 13.0% Total Total Awarded Number of Cost: Projects 3 3 1 3 1 8 9 3 1 2 2 1 3 5 1 1 7 4 6 4 1 3 2 $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ Owner Delay Rate 18.1% 10.4% 303.0% 2.7% 21.9% 32.9% 31.1% 79.1% 0.0% 0.0% 135.4% 0.0% 7.3% 23.2% 3.4% 24.8% 13.6% 1.7% 16.2% 0.0% 6.9% 29.4% 215.8% Vendor Vendor Percent Change Delay of Late Order Rate Reports Rate 0.2% 66.8% 53% 0.0% 0.0% 69% 0.0% 18.2% 47% 0.0% 18.8% 37% 2.7% 50.0% 0% 0.0% 16.3% 32% 0.7% 3.2% 35% 0.0% 1.4% 37% 0.6% 11.4% 25% -0.6% 5.4% 30% 0.0% 0.0% 32% 0.0% 0.0% 31% 0.0% 4.2% 26% 0.0% 0.0% 29% 0.0% 6.8% 22% 0.0% 0.0% 27% 0.0% 0.0% 26% 0.0% 12.2% 11% 0.0% 11.0% 9% 0.0% 9.8% 10% 0.0% 0.0% 15% 0.0% 7.0% 8% W W0.0% W . P B S0.0% RG.COM 13% Vendor Performance 120% 69% 65% 56% 53% 49% 39% 38% 37% 35% 32% 31% 30% 29% 29% 27% 26% 23% 21% 19% 15% 15% 13% 79 Report – Yearly Analysis WWW.PBSRG.COM 80 Report – Top 10 Riskiest Projects No Project Overall Awarded Change Awarded Cost Duration Order Rate Overall Delay Rate Percent of Late Reports Risk Analysis Factor PM Director 1 Mayo Remodel Suite A652 $ 269,850 66 9% 321% 47% 377% Wycliffe Waganda Gary Summerville 2 Barn Clean Renovations $ 269,000 80 2% 166% 60% 229% Wycliffe Waganda Justin Grussing 3 WBOB Remodel Suite 150 $ 273,100 99 1% 96% 37% 134% Pete Nickel Gary Summerville 4 Vet Sciences Third Floor $ 96,930 49 3% 86% 28% 116% Pete Nickel Gary Summerville 5 Weaver Densford College of Pharmacy $ 90,862 28 2% 25% 80% 107% Pete Nickel Gary Summerville 6 PWB Remodel Suite 6-240 $ 127,338 82 17% 23% 64% 104% Steve Bailey Gary Summerville 7 PWB Room 7-158B $ 46,504 30 0% 0% 100% 100% Pete Nickel Gary Summerville 8 Oak Street Parking Surveillance $ 246,802 74 0% 0% 100% 100% George Mahowald Justin Grussing 9 Snyder Bldg Exterior Door $ 219,000 121 -4% 81% 22% 100% Wycliffe Waganda Justin Grussing $ 1,593,561 254 29% 0% 50% 79% Matt Stringfellow Justin Grussing 10 Heller Hall Renovation WWW.PBSRG.COM 81 Report – Analysis of Risks Risk Category 1) Client Impacts Client Scope Change / Decision Client Requested Delay 2) CPPM Impacts 1,200 Percent Impact to Cost 59% Percent Impact to Schedule 46% 976 59% 37% - 224 0% 9% $329,425 885 30% 34% Number of Risks Impact to Cost Impact to Schedule 114 $660,369 111 $ 3 $ 135 660,369 Design Issue 48 $ 189,876 230 17% 9% CPPM Issue (Codes / Permits) 36 $ 46,140 170 4% 7% CPPM Issue (Energy Mgmt) 2 $ 47,533 30 4% 1% CPPM Issue (Hazardous / Health & Safety) 8 $ 35,407 118 3% 5% CPPM Issue (NTS) 8 $ 10,018 64 1% 2% CPPM Issue (Contract / Payment) 11 $ - 132 0% 5% CPPM Issue (Other) 22 $ 451 141 0% 5% 411 2% 16% 3) Contractor Impacts 43 $21,005 Contractor Issue 11 $ - 101 0% 4% Contractor Oversight of Design 9 $ 21,005 38 2% 1% Contractor Issue with Supplier / Sub 23 $ - 272 0% 10% 19 $102,544 111 9% 4% 311 $ 1,113,343 2,607 4) Unforeseen Impacts WWW.PBSRG.COM 82 Research from Contractor Delays Contractor Risks % Delivery of Materials Delayed 28% Installation errors 26% Incorrect material ordered or delivered 11% Alteration of installation needed 9% Manufacture didn't have sufficient materials 9% Misunderstanding of Construction Documents 6% Door Frames incorrect size 4% Soil compaction 2% WWW.PBSRG.COM 52% of risks due to errors in materials delivered 83 Targeted Business Group (Minority & Disadvantaged) Out of 63 qualified contractors, 18 are TGB (29%) Out of 161 PIPS Projects, 26 were awarded to TGB Contractors (16%) Awards were based on best-value, which shows that there are high performing TGB vendors in the MN community WWW.PBSRG.COM 84 Best Value Vendor Characteristics Preplans and Minimizes Risk on Each Initiative Has a plan and knows the risks to the plan Is transparent Communicates clearly Asks good questions, knows what they don’t know Measures Performance and drives accountability Uses Dominant Information to Differentiate themselves/show value added Educates the Client and helps the client be a better client Educates themselves and has a continually enhanced vision Holds themselves and the client accountable Their plan is aligned so that when they win, the client automatically wins WWW.PBSRG.COM 85 Best Value Client Characteristics Ensures their needs and concerns are known by the vendor Ensures the vendor has a plan that addresses each need/concern/risk Is a facilitator to the vendors development of the plan(s) Enforces the best value structure Weekly risk reporting is being done Each risk is given a client satisfaction rating Measurements by vendor are being done Do not be pulled into making decisions you do not need to make Educates themselves and the vendor Avoids reversion Transfers risk and control Holds the vendor and themselves accountable Ensures the plan is aligned so they win and the vendor wins WWW.PBSRG.COM 86 Master of Science Degree Program: Facilities Asset and Project Management Masters program: Available on-line and in-person (site visits) Based on IMT, Leadership, and Best Value Completely integrated with your job/organization Thesis becomes documentation of best value implementation at your organization 30 Credits – Only 7 Classes and a Thesis Classes in IMT, Best Value, and PIRMS Classes in Leadership and FM, and PM (value based) Classes in Research Methods and Data Analysis WWW.PBSRG.COM 87 Comments / Questions WWW.PBSRG.COM WWW.PBSRG.COM 88 WWW.PBSRG.COM 89