Project Selection Advisory Council
Stakeholder Advisory Committee
Joint Meeting
January 15, 2016
1
Agenda
• Introductions
• Progress to Date
• Portfolio/Balancing Strategy
• Path to Improvement
• Discussion
• Next Steps
2
CIP Process – Status
#
Establish PSAC
criteria and
scoring
Establish CIP
Approach
Compile
project
universe
Divisions
Score projects
Set division
priorities
Optimize
Portfolio/
Board Input
Prep for
A&F
Final CIP
for Board
Present
to Board
Process
Look back
Iterate
Dec 23 scores
Jan 4 targets
Feb 10
Final for Board /
Public Outreach
3
PSAC Scoring and Process: Criteria
Draft for Policy Discussion Only
4
Establish CIP Approach/Priorities
• Framework laid out in Council’s Recommendations
• Had to determine how to operationalize
• Chose to apply to all Divisions
• Started with Agile Scrum Process
• Determined universe of projects
• Developed new scoring sheets
• Established what could/should be scored
Draft for Policy Discussion Only
5
Compile Project Universe
• Comprehensive universe of potential transportation investments across
modes (roads, bridges, transit, bicycle, pedestrian) and types
(maintenance and preservation, safety, accessibility, capacity, new
service/expansion) was compiled
• Project Universe includes:
• Current projects under construction/design in each division
• All projects included in long range transportation plans
• Projects included in the STIP and TIPs
• Bond Bill projects
• Projects identified through Capital Conversations
Draft for Policy Discussion Only
6
Project Universe Breakdown
Modernization / Capacity
# Projects
Highway
MBTA
Rail
Transit
IT
Aero
RMV
Other1
Total
165
72
39
78
12
41
3
410
482
37
30
275
38
75
7
1,178
15
28
93
1
44
1,359
527
144
66
117
3
14
871
698
97
34
281
18
28
1,156
306
2
24
369
3,356
274
Underway
6
950
Design
▪
Roughly 950
projects
across
divisions were
targeted to be
scored1
▪
Some of those
projects will
only be partially
scored based
on the available
information
Proposed
Underway
SGR
Projects
targeted for
PSAC
scoring
Design
37
Proposed
194
877
72
191
10
6
4,980
Total
Draft for Policy Discussion Only
1 Includes Office of Planning and Enterprise Services
2 Due to time constraints and available information, only
selected projects (under Modernization/Capacity) were scored;.
SGR was excluded.
PSAC Project Scoring Data
Average
PSAC scores
(out of 100)
Min, max
Number of scores received
38
75
93
40
Rail
11
30
60
MBTA
124
29
73
93
Transit
32
14
48
75
79
Highway
90
1*
Aeronautics
40
50
60
RMV
10
• Divisions have strived to achieve scoring consistency within their own projects, however
•
scoring needs to be calibrated across Divisions
Once scoring is complete, we will determine if they can be scaled or need to be re-scored
* Represents interim aggregate score for all administrative buildings. To be subdivided into one score per building.
Draft for Policy Discussion Only
PSAC Project Scoring Data - Highway
Analysis notes:
Distribution of scoring by project type / PSAC
- 67 projects of 184 projects on the 2016-2019 STIP were
scored using PSAC evaluation criteria
- 13 bicycle projects were scored / median = 30%
- 14 signal projects were scored / median = 54%
- 40 roadway projects were scored / median = 51%
# of projects in quintile ►24
roadway project
distribution ►
13
10
signal project
distribution ►
bicycle project
distribution ►
9
6
5
0
0
0
20
%
40%
0
0
60
%
80%
0
100
%
quintiles of % of 0scores obtained
9
PSAC Project Scoring Data - Highway
Distribution of project scoring / MPO TEC vs. PSAC
# of projects in quintile ► 34
33
Analysis notes:
- 66 projects of 184 projects on the 2016-2019 STIP were
self-scored by the MPOs using their Transportation Evaluation Criteria;
while 67 of 184 projects were evaluated using PSAC scoring
- Projects attained an average of 43% of the maximum score
through MPO TEC and 48% in PSAC
- Projects attained a median of 39% of the maximum score
through MPO TEC and 49% in PSAC
- The MPO TEC scoring results skew further to the right of PSAC
possibly because some of the regions employ scoring schemes that
penalize project characteristics with negative values
23
MPO TEC distribution ►
◄ PSAC distrubition
20
15
6
1
0
2
20%
0
40%
60%
80%
100
0
%
quintiles of % of scores obtained
10
PSAC Project Scoring Data - Highway
MPO score %
PSAC score %
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Projects
Each color marker on
the MPO trend line
represents a different
MPO
11
Balancing – Optimizing Portfolio
• Regional and Social Equity
• Mapping Exercise – Regional and Social Equity
• Program Balance
• Division Goals/Priorities
• Board Input/MassDOT modal priorities
• PfP Tool
Draft for Policy Discussion Only
12
Planning for Performance (PfP) Tool
Draft for Policy Discussion Only
13
Wh
What is a Portfolio Driven Approach?
Projects-first approach
▪ MassDOT and its Boards pick projects,
according to available sources of funding
▪ The final CIP is a list of projects with
projected costs, organized by Division
Portfolio-driven approach
▪ Strategic investment priorities are
articulated by MassDOT and its Boards
▪ These priorities are weighted and
balanced in different investment
portfolios
▪ MassDOT and its Boards pick a portfolio
which best meets the strategic priorities
while providing balance across modes
and geographies
▪ Projects are mapped to specific priorities
and ranked according to PSAC and/or
other Division criteria
▪ The final CIP presents a portfolio with a
set of strategic investment priorities,
including a list of projects which can best
meet those priorities and their projected
costs
Draft for Policy Discussion Only
14
Portfolio Optimization: Strategy/Options
Funding could be concentrated on a few priorities or balanced
across several
Methods
Concentrated
vs
Description
Talking point
▪ Concentrate the dollars
▪ High perfor- ▪ Focuses the
for exceptionally strong
performance on a narrow
set of priorities
mance on
the highest
priorities
▪ Spread the dollars for all- ▪ All-round
round average
performance on a wide
set of performance targets
performance
Pros
organization on
doing a few things
very well
ILLUSTRATIVE
Cons
▪ Underfunds some
priorities
▪ Satisfies the many ▪ Feels least like
massDOT
stakeholders /
public mission
change
Balanced
Draft for Policy Discussion Only
15
Portfolio Optimization: Strategy/Options
Data can be used to select the highest return-on-performance
investment categories and highest scoring projects
Methods
High scoring
Description
Talking point
Pros
▪ Fund the highest PSAC-
▪ A+ projects
▪ Reflects strategic ▪ Not all projects
scoring projects first until
no dollars are left
regardless of project type
first
▪ Spend every dollar where ▪ Best bang
High return
ILLUSTRATIVE
it gives the best possible
marginal benefit, in terms
of asset performance
Draft for Policy Discussion Only
for the buck
priorities set by
PSAC board
▪ Allocates dollars
in the most
efficient manner
Cons
▪
received PSAC
scores this cycle
Subject to critique
that project scores
are not directly
comparable
▪ More complex to
communicate
16
CIP past, ideal, and current processes
Due to time and other constraints, the current FY17 process for CIP portfolio generation is a
transitional approach between past and ideal states
How do we set a
strategic
direction for the
CIP?
Where do
projects in the
CIP come from?
How do we pick
the best projects?
How are MBTA
and DOT CIPs
related?
Past process
Current process
Ideal process

Project list rather than
strategic portfolio

Portfolio to be
informed by
strategic priorities

Portfolio guided by
strategic objectives

Project universe
largely comprised of
projects that have
been advocated for
(either internally or
externally)

Project universe
includes all potential
projects, proactively
gathered from bond
bill, capital
conversations, and
other sources

Project universe
includes all potential
projects, proactively
gathered from bond
bill, capital
conversations, and
other sources

Selection of individual
projects to fund
based on vetted
staff opinion

Selection of individual
projects based on
scores and portfolio
fit as well as staff
opinion

Selection of individual
projects based on
scores and portfolio
fit with staff input

Separate processes
across MBTA and
DOT

Parallel and linked
processes covering
both MBTA and DOT

One process
including both MBTA
and DOT
17
Path to Improvement
In the works:
• Improve economic impact scoring method
• Streamline project intake forms between MPO and MassDOT
• Further expansion of PfP Tool
• Aligning CIP development with other processes
• MPO
• Asset Management
• Performance Management
• Modal Plans
• Adjust criteria/weights based on what we’ve learned
• Developing a gaps analysis and guidebook for how future
iterations of the CIP should be completed
 Will articulate additional items to address to improve
process
18
Discussion Questions
• What feedback to you have on what we’ve presented so far?
• What else should we be thinking about?
19
Next Steps
• Where should the Project Selection Advisory Council and the Stakeholder
Committee go from here?
20