Using Data to Optimize Your Contracting Organization Breakout Session # E05 Joanie Newhart - Associate Administrator for Acquisition Workforce Programs Office of Management and Budget Trey Bradley - Program Manager, Strategic Data Initiatives General Services Administration Tuesday, December 4, 2018 2pm 1 Agenda Satisfaction with Acquisitions 2 Competition in the Federal Market Agency Actions to Improve Effective Competition Satisfaction Across the Acquisition Process (2016-2017) How satisfied were you with your overall experience on this acquisition? Source: Acquisition360 The data reveals high levels of satisfaction; The difference in satisfaction between the customer and the contracting office is statistically significant using a joint test Likert Scale 1 Extremely Dissatisfied 2 Moderately Dissatisfied 3 Neither Satisfied nor Dissatisfied 4 Moderately Satisfied 5 Extremely Satisfied 3 Program Office Overall Satisfaction with Acquisitions (2016 – 2017) How satisfied were you with your overall experience on this acquisition? There is very little variation in average satisfaction among agencies Source: Acquisition360 Likert Scale 1 Extremely Dissatisfied 2 Moderately Dissatisfied 3 Neither Satisfied nor Dissatisfied 4 Moderately Satisfied 5 Extremely Satisfied Note: Agencies with less than 10 records have been excluded from this visual. 4 Satisfaction When Part of Integrated Project Team (IPT) (2016-2017) How satisfied were you with your overall experience on this acquisition? At first glance, the presence of an IPT does not appear to drive satisfaction Source: Acquisition360 Integrated Project Team (IPT): a team of professionals representing diverse disciplines with the specific knowledge, skills, and abilities necessary to support the successful execution of projects. Likert Scale 1 Extremely Dissatisfied 2 Moderately Dissatisfied 3 Neither Satisfied nor Dissatisfied 4 Moderately Satisfied 5 Extremely Satisfied 5 Measuring IPT Interaction Using Regression Analysis (2016-2017) Understanding what drives customer satisfaction n = 195 Variable Overall Sat 6 Source: Acquisition360 Definition Overall Satisfaction – Survey Question: How satisfied were you with your overall experience on this acquisition? Integrated Project Team Value of 1 if part of an Integrated Project Team( IPT); value of 0 if not part of IPT. Milestone Schedule Satisfaction with the acquisition milestone schedule when accounting for interaction with the IPT variable. Communication Responsiveness Satisfaction with the procurement office’s responsiveness to questions (communicating in a clear, courteous, timely, and professional manner) when accounting for interaction with the IPT variable. See appendix for complete regression analysis. All variables are statistically significant. Being part of an IPT matters if the Contracting office has the right skills in place—i.e., communication and milestone scheduling. Takeaway from Satisfaction Responses Integrated Project Teams Help reduce redundant work, improve efficiency in the acquisition process and promote resolution of issues. MITRE Integrated Project Team Start-up Guide available here. Move from a Transactional Leader to a Network Leader or “Connector” Per CEB, there is a “fundamental shift to a more fluid business environment, characterized by ubiquitous information and rapid technological advances, where employees’ work has become more collaborative, interdependent, and knowledge based.” Read more here. Collaboration Build relationships with your cross-functional and cross-agency colleagues. When there’s an issue to resolve, having relationships in place makes everything go more smoothly. Forbes has some thoughts on skills needed for improving collaboration here. We are entering a time of unparalleled change in the acquisition world – from modernizing federal acquisition and leveraging innovative practices to incorporating technology into the acquisition process. Hold on and keep up on all the changes at www.performance.gov. 7 Agenda Satisfaction with Acquisitions 8 Competition in the Federal Market Agency Actions to Improve Effective Competition Defining the Rates What are the rates that play a role in Competition? Competition Rate Total Competed Dollars Total Obligations 9 Effective Competition Rate Total Competed Dollars with 2 or more Bids Total Competed Dollars Not Competed Rate Total Not Competed Dollars Total Obligations One Bid Rate Total Competed Dollars with only 1 Bid Total Competed Dollars Small Disadvantaged Business Rate Total Small Disadvantaged Business Dollars Total Small Business Eligible Dollars A Closer look at the Policy of Effective Competition (2014-2017) Understanding the factors that can drive change in Effective Competition Not competed rate is lagged by one period One Bid Rate has 3.5 times the Impact on Effective Competition Rate than Lagged Not Compete Rate* *Standard normal coefficients significant at 1% level All variables are statistically significant. See appendix for complete regression analysis. n = 626 • • • 10 Source: FPDS Moving not competed dollars to effectively competed dollars has a one year lag before impacting effective competition rate - This reflects additional time required for a relatively complex acquisition strategy required to effect this change Agencies should reap the rewards of higher effective competition sooner by focusing on moving one bid dollars to effectively competed dollars Policy tension – Small Business Set-Asides come at the cost of effective competition Effective Competition at CFO Act Agencies* (FY17) Using a 2X2 chart to determine where your Agency stands Effective Competition Rate Competed Dollars Receiving >1 Bid Total Competed Dollars *DoD Excluded n = 8,309,902 Contracting Actions 11 Source: FPDS Agenda Satisfaction with Acquisitions 12 Competition in the Federal Market Agency Actions to Improve Effective Competition Competition at Agency X Raising the Competition Bar Increase this gap n = 89,465 Contracting Actions Source: FPDS-NG 13 Effective Competition Within Agency X (FY17) Using the 2X2 Chart to Focus on Areas for Improvement Components with room for improvement n = 89,465 Contracting Actions 14 Source: FPDS Focus On Easy Wins – One Bid Dollars – Agency X Improving Effective Competition does not necessarily require a broad focus across many contract actions 5% of the One Bid contracts make up 80% of the One Bid dollars in FY17 at Agency X n = 13,907 Contracting Actions 15 Source: FPDS Longer-Term Strategy & Focus – Not Competed Dollars – Agency X Focus on the few not competed contracts with a longer-term strategy to move some of these actions into competitive awards 7% of the Not Competed contracts make up 90% of the Not Competed dollars in FY17 at Agency X n = 4,947 Contracting Actions 16 Source: FPDS Not Competed Awards at Agency X (FY17) More than 35% of the contracts at Agency X did not list a not compete reason Of these dollars, $192M did not utilize a set-aside n = 4,947 Contracting Actions 17 Source: FPDS Breaking Down Not Competed Awards at Agency X (FY17) What categories should we place focus on? Focus on categories with high levels of not competed awards and ease of transition to competed awards (e.g., IT and facilities-related services at Agency X) n = 4,947 Contracting Actions 18 Source: FPDS Expiration Timeline at Agency X A look at upcoming expiring contracts 19 How Can We Help? Potential GSA Contract Portfolio Tool Phase 1 (February 2019) Expiration Timeline Tool allowing for easy exploration of expiring contracts by competition type (Slide 19) Phase 2 (Timeline TBD) • 2X2 Analysis of Components (Slide 14) • Drill Down on reasons & set-asides (Slide 17) • What else would be useful for the audience to see? Helpful Guidance from DOD In 2014, DOD issued guidance on competition that could be useful to other agencies as they refine their approaches. Guidance can be found in this document. 20 Contact Information Joanie Newhart – JNewhart@omb.eop.gov Trey Bradley – Trey.Bradley@gsa.gov 21 Appendix 22 Defining the Variables Acquisition 360 - PMO Responses Variable OVALLSAT Overall Satisfaction: How satisfied were you with your overall experience on this acquisition] ACQMILE Planning: How satisfied were you: [With the acquisition milestone schedule] SCHCHG Planning: How satisfied were you: [With the procurement office’s ability to keep you informed of any changes to the acquisition milestone schedule] ACQASSIST Planning: How satisfied were you: [With the procurement office’s assistance in the Acquisition Plan process, which allowed you to better understand and participate in the procurement] INDENG Planning: How satisfied were you: [With the procurement office’s engagement with industry early in the acquisition process] ACQCOM Communication: How satisfied were you: [With the procurement office’s responsiveness to your questions (communicating in a clear, courteous, timely, and professional manner)] RESISS ELEVPROB ACQRR Integrated Project Team 23 Survey Question Communication: How satisfied were you: [With the procurement office’s effectiveness in resolving any issues or delays encountered during the acquisition process] Communication: How satisfied were you: [With your understanding on how - and to whom – you should elevate problems for resolution] Communication: How satisfied were you: [With early communications describing the roles and responsibilities of the procurement office and of your office (program office)] Value of 1 if part of an Integrated Project Team( IPT); value of 0 if not part of IPT. ACQ360 Regression Analysis 24 One Bid, Not Competed, and SDB effect on Effective Competition Regression Analysis Dependent Variable: EFFCOMPSN Method: Panel Least Squares Date: 10/29/18 Time: 18:08 Sample (adjusted): 2015 2018 Periods included: 4 Cross-sections included: 165 Total panel (unbalanced) observations: 626 Variable C NOTCOMPRATESN(-1) ONEBIDRATESN SDBRATESN R-squared Adjusted R-squared S.E. of regression Sum squared resid Log likelihood F-statistic 25 Coefficient -0.05418 -0.210372 -0.754187 -0.092016 0.902902 0.866624 0.370478 62.4506 -166.7985 24.8882 Std. Error 0.015008 0.034046 0.030928 0.038707 Mean dependent var S.D. dependent var Akaike info criterion Schwarz criterion Hannan-Quinn criter. Durbin-Watson stat t-Statistic -3.610006 -6.179068 -24.38556 -2.377243 -0.072555 1.014433 1.079228 2.291894 1.550395 1.858243 Prob. 0.0003 0 0 0.0179 Driving Factors Behind a Satisfied Program Office The driving factors of overall satisfaction from the Program Office (customer) include: • Procurement Office responsiveness to questions (communicating in a clear, courteous, timely, and professional manner). • Procurement Office ability to keep the Program Office informed of any changes to the acquisition milestone schedule. • Program office understanding on how, and to whom, they should elevate problems to for resolution. 26 Reasons for Not Competing Contracts at Agency X (FY17) A look at the set-asides (or lack thereof) used in not competed awards n = 4,947 27 Source: FPDS