Using Data to Optimize Your Contracting Organization
Breakout Session # E05
Joanie Newhart - Associate Administrator for Acquisition Workforce Programs
Office of Management and Budget
Trey Bradley - Program Manager, Strategic Data Initiatives
General Services Administration
Tuesday, December 4, 2018
2pm
1
Agenda
Satisfaction
with
Acquisitions
2
Competition in
the Federal
Market
Agency Actions
to Improve
Effective
Competition
Satisfaction Across the Acquisition Process (2016-2017)
How satisfied were you with your overall experience on this acquisition?
Source: Acquisition360
The data reveals high levels of satisfaction;
The difference in satisfaction between the customer and the
contracting office is statistically significant using a joint test
Likert Scale
1 Extremely Dissatisfied 2 Moderately Dissatisfied 3 Neither Satisfied nor Dissatisfied 4 Moderately Satisfied 5 Extremely Satisfied
3
Program Office Overall Satisfaction with Acquisitions (2016 – 2017)
How satisfied were you with your overall experience on this acquisition?
There is
very little
variation in
average
satisfaction
among
agencies
Source: Acquisition360
Likert Scale
1 Extremely Dissatisfied 2 Moderately Dissatisfied 3 Neither Satisfied nor Dissatisfied 4 Moderately Satisfied 5 Extremely Satisfied
Note: Agencies with less than 10 records have been excluded from this visual.
4
Satisfaction When Part of Integrated Project Team (IPT) (2016-2017)
How satisfied were you with your overall experience on this acquisition?
At first
glance, the
presence of
an IPT does
not appear
to drive
satisfaction
Source: Acquisition360
Integrated Project Team (IPT): a team of professionals representing diverse disciplines with the specific knowledge,
skills, and abilities necessary to support the successful execution of projects.
Likert Scale
1 Extremely Dissatisfied 2 Moderately Dissatisfied 3 Neither Satisfied nor Dissatisfied 4 Moderately Satisfied 5 Extremely Satisfied
5
Measuring IPT Interaction Using Regression Analysis (2016-2017)
Understanding what drives customer satisfaction
n = 195
Variable
Overall Sat
6
Source: Acquisition360
Definition
Overall Satisfaction – Survey Question: How satisfied were you with
your overall experience on this acquisition?
Integrated
Project Team
Value of 1 if part of an Integrated Project Team( IPT); value of 0 if not
part of IPT.
Milestone
Schedule
Satisfaction with the acquisition milestone schedule when accounting
for interaction with the IPT variable.
Communication
Responsiveness
Satisfaction with the procurement office’s responsiveness to
questions (communicating in a clear, courteous, timely, and
professional manner) when accounting for interaction with the IPT
variable.
See appendix for complete regression analysis.
All variables are statistically significant.
Being part of an IPT matters if the Contracting
office has the right skills in place—i.e.,
communication and milestone scheduling.
Takeaway from Satisfaction Responses
Integrated Project Teams
Help reduce redundant work, improve efficiency in the acquisition process and promote
resolution of issues. MITRE Integrated Project Team Start-up Guide available here.
Move from a Transactional
Leader to a Network Leader
or “Connector”
Per CEB, there is a “fundamental shift to a more fluid business environment, characterized by
ubiquitous information and rapid technological advances, where employees’ work has
become more collaborative, interdependent, and knowledge based.” Read more here.
Collaboration
Build relationships with your cross-functional and cross-agency colleagues. When there’s an
issue to resolve, having relationships in place makes everything go more smoothly. Forbes
has some thoughts on skills needed for improving collaboration here.
We are entering a time of unparalleled change in the acquisition world – from modernizing federal
acquisition and leveraging innovative practices to incorporating technology into the acquisition
process. Hold on and keep up on all the changes at www.performance.gov.
7
Agenda
Satisfaction
with
Acquisitions
8
Competition in
the Federal
Market
Agency Actions
to Improve
Effective
Competition
Defining the Rates
What are the rates that play a role in Competition?
Competition
Rate
Total Competed
Dollars
Total Obligations
9
Effective
Competition
Rate
Total Competed
Dollars with 2 or
more Bids
Total Competed
Dollars
Not Competed
Rate
Total Not
Competed Dollars
Total Obligations
One Bid Rate
Total Competed
Dollars with only 1
Bid
Total Competed
Dollars
Small
Disadvantaged
Business Rate
Total Small
Disadvantaged
Business Dollars
Total Small
Business Eligible
Dollars
A Closer look at the Policy of Effective Competition (2014-2017)
Understanding the factors that can drive change in Effective Competition
Not competed rate is
lagged by one period
One Bid Rate has 3.5 times the Impact on Effective Competition Rate than
Lagged Not Compete Rate*
*Standard normal coefficients significant
at 1% level
All variables are statistically significant. See appendix for complete regression analysis.
n = 626
•
•
•
10
Source: FPDS
Moving not competed dollars to effectively competed dollars has a one year lag before impacting effective
competition rate
- This reflects additional time required for a relatively complex acquisition strategy required to effect this change
Agencies should reap the rewards of higher effective competition sooner by focusing on moving one bid dollars to
effectively competed dollars
Policy tension – Small Business Set-Asides come at the cost of effective competition
Effective Competition at CFO Act Agencies* (FY17)
Using a 2X2 chart to determine where your Agency stands
Effective Competition Rate
Competed Dollars Receiving >1 Bid
Total Competed Dollars
*DoD Excluded
n = 8,309,902 Contracting Actions
11
Source: FPDS
Agenda
Satisfaction
with
Acquisitions
12
Competition in
the Federal
Market
Agency Actions
to Improve
Effective
Competition
Competition at Agency X
Raising the Competition Bar
Increase this gap
n = 89,465 Contracting Actions
Source: FPDS-NG
13
Effective Competition Within Agency X (FY17)
Using the 2X2 Chart to Focus on Areas for Improvement
Components with room
for improvement
n = 89,465 Contracting Actions
14
Source: FPDS
Focus On Easy Wins – One Bid Dollars – Agency X
Improving Effective Competition does not necessarily require a broad focus across
many contract actions
5% of the One Bid contracts make up 80% of
the One Bid dollars in FY17 at Agency X
n = 13,907 Contracting Actions
15
Source: FPDS
Longer-Term Strategy & Focus – Not Competed Dollars – Agency X
Focus on the few not competed contracts with a longer-term strategy to move some
of these actions into competitive awards
7% of the Not Competed contracts make up 90%
of the Not Competed dollars in FY17 at Agency X
n = 4,947 Contracting Actions
16
Source: FPDS
Not Competed Awards at Agency X (FY17)
More than 35% of the contracts at Agency X did not list a not compete reason
Of these dollars, $192M
did not utilize a
set-aside
n = 4,947 Contracting Actions
17
Source: FPDS
Breaking Down Not Competed Awards at Agency X (FY17)
What categories should we place focus on?
Focus on categories with high levels of not
competed awards and ease of transition to
competed awards (e.g., IT and facilities-related
services at Agency X)
n = 4,947 Contracting Actions
18
Source: FPDS
Expiration Timeline at Agency X
A look at upcoming expiring contracts
19
How Can We Help?
Potential GSA Contract Portfolio Tool
Phase 1 (February 2019)
Expiration Timeline Tool allowing for easy exploration of
expiring contracts by competition type (Slide 19)
Phase 2 (Timeline TBD)
• 2X2 Analysis of Components (Slide 14)
• Drill Down on reasons & set-asides (Slide 17)
• What else would be useful for the audience to see?
Helpful Guidance from DOD
In 2014, DOD issued guidance on competition that could be useful
to other agencies as they refine their approaches.
Guidance can be found in this document.
20
Contact Information
Joanie Newhart – JNewhart@omb.eop.gov
Trey Bradley – Trey.Bradley@gsa.gov
21
Appendix
22
Defining the Variables
Acquisition 360 - PMO Responses
Variable
OVALLSAT
Overall Satisfaction: How satisfied were you with your overall experience on this acquisition]
ACQMILE
Planning: How satisfied were you: [With the acquisition milestone schedule]
SCHCHG
Planning: How satisfied were you: [With the procurement office’s ability to keep you informed of any changes to the acquisition milestone
schedule]
ACQASSIST
Planning: How satisfied were you: [With the procurement office’s assistance in the Acquisition Plan process, which allowed you to better
understand and participate in the procurement]
INDENG
Planning: How satisfied were you: [With the procurement office’s engagement with industry early in the acquisition process]
ACQCOM
Communication: How satisfied were you: [With the procurement office’s responsiveness to your questions (communicating in a clear,
courteous, timely, and professional manner)]
RESISS
ELEVPROB
ACQRR
Integrated
Project Team
23
Survey Question
Communication: How satisfied were you: [With the procurement office’s effectiveness in resolving any issues or delays encountered during the
acquisition process]
Communication: How satisfied were you: [With your understanding on how - and to whom – you should elevate problems for resolution]
Communication: How satisfied were you: [With early communications describing the roles and responsibilities of the procurement office and of
your office (program office)]
Value of 1 if part of an Integrated Project Team( IPT); value of 0 if not part of IPT.
ACQ360 Regression Analysis
24
One Bid, Not Competed, and SDB effect on Effective Competition
Regression Analysis
Dependent Variable: EFFCOMPSN
Method: Panel Least Squares
Date: 10/29/18 Time: 18:08
Sample (adjusted): 2015 2018
Periods included: 4
Cross-sections included: 165
Total panel (unbalanced) observations: 626
Variable
C
NOTCOMPRATESN(-1)
ONEBIDRATESN
SDBRATESN
R-squared
Adjusted R-squared
S.E. of regression
Sum squared resid
Log likelihood
F-statistic
25
Coefficient
-0.05418
-0.210372
-0.754187
-0.092016
0.902902
0.866624
0.370478
62.4506
-166.7985
24.8882
Std. Error
0.015008
0.034046
0.030928
0.038707
Mean dependent var
S.D. dependent var
Akaike info criterion
Schwarz criterion
Hannan-Quinn criter.
Durbin-Watson stat
t-Statistic
-3.610006
-6.179068
-24.38556
-2.377243
-0.072555
1.014433
1.079228
2.291894
1.550395
1.858243
Prob.
0.0003
0
0
0.0179
Driving Factors Behind a Satisfied Program Office
The driving factors of overall satisfaction from the Program Office (customer) include:
• Procurement Office responsiveness to questions (communicating in a clear, courteous,
timely, and professional manner).
• Procurement Office ability to keep the Program Office informed of any changes to the
acquisition milestone schedule.
• Program office understanding on how, and to whom, they should elevate problems to for
resolution.
26
Reasons for Not Competing Contracts at Agency X (FY17)
A look at the set-asides (or lack thereof) used in not competed awards
n = 4,947
27
Source: FPDS