Energy - Utilities
October 2011
Measuring productivity at utilities
Report prepared for Hydro One
Intervener meeting; October 19, 2011
C O N F I D E N T I A L | www.oliverwyman.com
Contents
 Background
 Executive summary
 Industry measurement of productivity
– By regulatory commissions
– At utilities
 Perspectives on productivity metrics
 Activity cost analysis to choose metrics with the biggest impact
– Principal cost driver analysis
– Specific metric recommendations
 Implementation considerations
© Oliver Wyman  www.oliverwyman.com
1
Background
 “In its December 23, 2010 Decision approving Transmission Revenue Requirements for 2011 and
2012, the Ontario Energy Board provided direction and other expectations for further information on
compensation and efficiency comparisons.
 The Board directed “Hydro One to revisit its compensation cost benchmarking study [the Mercer
study] in an effort to more appropriately compare compensation costs to those of other regulated
transmission and/or distribution utilities in North America.”
 Toward that end, the Board directed "Hydro One to consult with stakeholders about how the Mercer
study should be updated and expanded to produce such analyses”.
 The Board went on to describe its expectation that Hydro One “be in a position to provide more
robust evidence on initiatives to achieve a level of cost per employee closer to market value at its
next transmission rate case. The Board will expect compensation increase to be matched with
demonstrated productivity gains”.
Extract from Hydro One RFP # SCO-1000152789, March 2nd 2011
 To satisfy all aspects of the boards request contained in the RFP Oliver Wyman was engaged alongside Mercer.
– Mercer to update the compensation benchmarking study in 2011 and will separately report changes in relative
compensation levels
– Oliver Wyman to provide a perspectives on industry best practices for productivity measurement
© Oliver Wyman  www.oliverwyman.com
2
Outline of approach
Internal and external research was conducted to build a set of recommendations around
how Hydro One could measure productivity.
Research
Contacted commissions
- Contacted 9 US and 8 Canadian
commissions about how they measure
productivity.
Recommendation
Implementation
Outline implementation
considerations
Considerations of productivity metric collection
There are several considerations when using productivity metrics to make decisions
about the performance of the operations.
Surveyed utilities
- Contacted 11 US and 20 Canadian
utilities to establish how they measure
productivity internally
- 13 provided information
Assembled potential metrics
- 10+ meetings held to discuss
productivity with VPs and SVPs
- Aggregated OW knowledge on best
practices for balanced scorecard
creation
Performed cost analysis of provided
financial data
- Identified largest activities to target with
metrics
Reviewed the data capabilities of
Hydro One
- Systems expert interviews and viewed
reporting tools
© Oliver Wyman  www.oliverwyman.com
Balanced approach
required
Difficulty to obtain like
for like comparisons
Metrics could not
capture all productivity
improvements
Cost of metric collection
 Examining productivity in
isolation can cause
misunderstanding and
unintended behaviors
 Even within the same utility
over time, job conditions
change across the territory
and in different years
 Savings from improvements in
working methods, which
eliminate work may not be
captured
 There are potentially large
costs associated with data
collection, retention and
analysis.
 Quality, safety, and long-term
performance should be
measured and examined in
the same context.
– e.g., encouraging faster jobtime could lead to assets
that malfunction and lead to
lower reliability
 It is important to obtain
context of the metrics before
making decisions
– E.g., it is considerably more
difficult to place a pole in
rock than soil; work mix can
mask productivity changes
 For example, AMR has
reduced the need for
personnel to read meters,
which is an improvement in
productivity
 For example, collecting time
associated with a specific task
and associated conditions
either requires
mobile/handheld field systems
or extensive overhead to
collect the information at a
detailed enough level.
Next steps
- Future technology
- Implementation costs
- Reporting and analysis
Despite the challenges utilities continue to look for ways to measure productivity in a way that improves their
business
 Most utilities use a balanced set of metrics to obtain the clearest picture of performance
– Analysis teams place results into context of business cycles and external influences (e.g. weather)
– Trends in headline metrics are explained by the underlying supporting metrics
 Productivity metrics become integrated into the overall management of a utility aligned with corporate goals
– Understand and explain the drivers behind changing costs, internally and to regulators
– Target corporate initiatives at poorly performing areas
 Utilities leverage advanced IT systems such as mobile tracking devices to produce detailed productivity
metrics without creating large indirect costs
13
© Oliver Wyman  www.oliverwyman.com
Implementation considerations
Hydro One will require a detailed plan to develop a set of productivity metrics that are
integrated and aligned with the overall corporate scorecard and direction.
Provided set of metrics
for consideration
High level activities to develop productivity
Productivity metric list, developed, ‘vetted’, iterated,
finalized (aligned with corporate scorecard)
Beta example ‘scorecard’ reports shared with
executives
Report templates signed off
Required system changes identified
System changes implemented
Summary of recommended metrics
Twenty five selected metrics account for approximately twenty two percent of total costs.
 Aggregating the metric choices from
the previous pages, shows a good
coverage of total cost
– 22% of costs covered using 25 main
productivity metrics.
#
Metric
1
Cost of brush control per km of line
Cost per meter install
Cost per pole
4.6%
3.9%
3.7%
 Some metrics cover multiple activities
across different segments (TC, T
OM&A, DC, DOM&A).
6
Cost per new service
Cost per tower constructed
Cost per foundation
Cost per km of line cleared
Cost per meter read
Cost per upgrade
1.1%
0.9%
0.9%
0.9%
1.0%
0.7%
Cost per km of transmission line refurbished
Cost per insulator replaced
Cost per locate
Cost per km for line patrol
Cost per breaker
Cost per transformer
Cost per RTU
Cost per bill
Cost per km of line cleared
Cost per unit – protection replacement
Cost per Transformer Refurbishment
Cost per cancellation
Cost per insulator inspection
Cost per disconnect
0.6%
0.5%
0.6%
0.4%
0.4%
0.4%
0.4%
0.2%
0.3%
0.2%
0.2%
0.2%
0.1%
0.2%
 Some further subdivision of these
metrics may be required to allow
better comparisons.
– e.g. cost per pole could be sub
divided into cost per pole per ground
type.
 Note that estimations of cost coverage
was based on project titles.
– Further validation with business would
be required to confirm the
assumptions made.
– Large amount of projects could not be
understood from titles well enough to
suggest metrics.
2
3
4
5
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Cost per reconnect
Cost per inspection
Total
© Oliver Wyman  www.oliverwyman.com
Rollout to Users (training, access etc)
Beta testing of results and reports
% of total costs
Production state
Ongoing monitoring of productivity improvement
initiatives
© Oliver Wyman  www.oliverwyman.com
19
0.2%
0.1%
~22%
17
3
Executive summary
 Oliver Wyman conducted a broad market survey of US and Canadian utilities and contacted
many regulators directly to assess how common productivity measures were used. In total we
made over 350 contacts to:
– In Canada: 20 utilities and 8 Commissions
– In the US: 11 utilities and 9 Commissions
 This research found:
– No commission was found to routinely measure productivity directly. Instead commissions
focused on ‘outcome’ metrics of overall Service Quality Metrics (SQM) and total costs. In the
majority of cases, the commissions directed us to contact utilities directly as the management of
productivity was considered part of their internal management.
– Most utilities did look at productivity metrics internally as part of a balanced scorecard to
support understanding of trends of the SQM and Cost metrics reported to commissions. The
productivity metrics found suggest that none of the participants track productivity across all
business functions.
 After analyzing Hydro One’s major costs and interviewing many of their senior staff, a number of
metrics have been suggested as candidates to measure productivity, which account for 23% of total
O&M and Capex costs.
© Oliver Wyman  www.oliverwyman.com
4
Contents
 Background
 Executive summary
 Industry measurement of productivity
– By regulatory commissions
– At utilities
 Perspectives on productivity metrics
 Activity cost analysis to choose metrics with the biggest impact
– Principal cost driver analysis
– Specific metric recommendations
 Implementation considerations
© Oliver Wyman  www.oliverwyman.com
5
Primary research at US and Canadian Regulators
A majority of the regulators examined, measured total costs and service quality metrics
instead of productivity metrics
 17 Regulators across the US and Canada were requested to provide any methodologies they had for measuring
productivity
– Contact was made by a combination of calls, e-mails and requests for information
– A review was also performed of publicly filed documents such as rate cases and other dockets
 Most regulators collected service quality metrics with a significant number of cost metrics.
 No regulator regularly measured any productivity metrics
– Nova Scotia Utilities and Review Board measured one productivity metric (Calls handled per agent per day) via an
ad hoc study
 Various additional studies were found during the research
– Ad hoc studies:
- Operational review of Nova Scotia Power, consultant benchmarked NSPI’s corporate services functions finding it
was average to good performers in comparison with the panel (primarily cost benchmarks)
- Kaiser Associates operational review of Nova Scotia Power benchmarked OM&G expenses determining NSPI
performed at a lower cost than other utilities when normalized for size
- CAMPUT review of benchmarking practices for regulators in Canada provided a background on the difficulties in
benchmarking (inconsistent data, definition inconsistencies), as well as high level steps which must be taken
before benchmarking can be adopted into regulatory proceedings (building support and consensus, data
collection and summary, analysis and evolution)
– Regular studies:
- PSEG internal benchmarking survey which benchmarked safety, service quality, cost and reliability across a
panel of electric utilities
© Oliver Wyman  www.oliverwyman.com
6
Contents
 Background
 Executive summary
 Industry measurement of productivity
– By regulatory commissions
– At utilities
 Perspectives on productivity metrics
 Activity cost analysis to choose metrics with the biggest impact
– Principal cost driver analysis
– Specific metric recommendations
 Implementation considerations
© Oliver Wyman  www.oliverwyman.com
7
Overview of methodology for utilities productivity survey
A number of utilities were surveyed to understand which metrics they collect internally
 The purpose of the survey is to gain insight into the different ways in which utilities measure their
performance internally in terms of service quality, cost and productivity
 The survey required participants to provide their cost, productivity and service quality metrics and
for each metric:
– How long they had been measured for
– If they were reported to regulators
– If it was part of penalty
– If they were benchmarked
Respondents
Survey panel recipients
Canada
United States
Note: For a copy of the survey, please see the appendix; PSEG information was extracted from public sources
© Oliver Wyman  www.oliverwyman.com
8
Summary of findings from productivity survey
Performance was measured differently between each utility which participated in the
survey, with most focusing on service quality.
Overall Trends
 Wide disparity in internal performance measurement, each utility defines productivity, service quality and cost
metrics differently.
 Generally each utility is unsure of what percentage of the total cost is being covered by the performance metrics
submitted
Category
Service
Quality
Productivity
Cost
Category findings
 Service quality metrics are reported to regulators more than
cost or productivity
 Common metrics: SAIDI, SAIFI, CAIDI, OSHA rates
 Most productivity metrics were also cost related on a unit
basis
– Cost / KM of line trimmed
 Productivity metrics are not generally benchmarked, part of
a penalty or reported to regulators
 Most metrics detail overall spend in business categories,
many have normalizing factors. E.g. Distribution
O&M/customer
 22% of all cost metrics reported internally are part of a
benchmarking effort
© Oliver Wyman  www.oliverwyman.com
Metrics collected per utility
Median
Max
Min
Total
25
176
4
478
Median
Max
Min
Total
4
59
0
114
Median
Max
Min
Total
6
89
1
213
9
Common metrics
Metrics collected in the surveying process which were measured by at least two utilities.
Cost
3 common metrics
Service quality
34 common metrics
 Net income
 Net Income from operations
 OM&A Costs per customer
 System avg. interruption frequency index
(SAIFI)
 Customer avg. interruption disruption index
(CAIDI)
 % of calls answered in 30s or less
 System avg. interruption duration index (SAIDI)
 % of Calls abandoned
 % of Meters read
 % In-service appointments met
 Customers experiencing multiple interruptions
(CEMI)
 Bill accuracy rate
 Average speed of answer
 Occupational Safety and Health Administration
Incidence Rate
 Momentary avg. interruption frequency index
(MAIFI)
 Emergency response time
 SAIFI – Distribution Only
 # of Off-cycle meter reads/month
 SAIDI – Distribution Only
 Occupational Safety and Health Administration
Severity Rate
 # of Postーfinal adjustment mechanism
processed per month
 New service installation factor
Productivity
6 common metrics






Turnover
Cost per call
Meter reads per FTE
Lost time accident rate
First call resolution rate
Average time per calｌ
© Oliver Wyman  www.oliverwyman.com
 # of Sites billed/month
 # of Sites not billed/month
 DTE cases per 1000 customers
 Damages per 1000 elect. Locate requests
 Customer satisfaction – overall
 Customer experience long interruption duration
(CELID)
 CAIDI – Distribution Only
 CAIDI – Storm
 Average number of energizations per month
 Average number of de-energizations/month
 Average System Availability Index (ASAI)
 % of Meters not read within 6 months
 % of Completed off-cycle meter reads >5 days
 % of Calls answered in under 20s
 Vehicle accident frequency rate
10
Contents
 Background
 Executive summary
 Industry measurement of productivity
– By regulatory commissions
– At utilities
 Perspectives on productivity metrics
 Activity cost analysis to choose metrics with the biggest impact
– Principal cost driver analysis
– Specific metric recommendations
 Implementation considerations
© Oliver Wyman  www.oliverwyman.com
11
Criteria for choosing a set of metrics
Most utilities select metrics to examine using some criteria that best fits with their
business needs.
Metric Criteria
Description
1
Targets principal labour cost
areas
Build an understanding of labour costs and target the
biggest activities first. Choose enough metrics to measure a
large proportion of total costs
2
Covers a wide cross section
of work
Choose metrics which measure the major functions of the
business.
3
Based on Data Capabilities
Only use metrics from data that have high confidence
levels.
4
Allows consistent
measurement over time
Metrics should be precisely defined, so year on year
comparisons are meaningful
5
Appropriate measurement
costs
Metrics should balance usefulness and costs to measure.
6
Applicable over long time
frame
Corporate metrics should not be specific to a particular
project, but rather valid for multiple years
7
Focus on key areas of
customer interest
Metrics should primarily focus on areas of high concern
and/or are important to its customers.
A metric may need to be used in conjunction with other metrics to meet all the criteria
above
© Oliver Wyman  www.oliverwyman.com
12
Considerations of productivity metric collection
There are several considerations when using productivity metrics to make decisions
about the performance of the operations.
Balanced approach
required
Difficulty to obtain like
for like comparisons
Metrics could not
capture all productivity
improvements
Cost of metric collection
 Examining productivity in
isolation can cause
misunderstanding and
unintended behaviors
 Even within the same utility
over time, job conditions
change across the territory
and in different years
 Savings from improvements in
working methods, which
eliminate work may not be
captured
 There are potentially large
costs associated with data
collection, retention and
analysis.
 Quality, safety, and long-term
performance should be
measured and examined in
the same context.
– e.g., encouraging faster jobtime could lead to assets
that malfunction and lead to
lower reliability
 It is important to obtain
context of the metrics before
making decisions
– E.g., it is considerably more
difficult to place a pole in
rock than soil; work mix can
mask productivity changes
 For example, AMR has
reduced the need for
personnel to read meters,
which is an improvement in
productivity
 For example, collecting time
associated with a specific task
and associated conditions
either requires
mobile/handheld field systems
or extensive overhead to
collect the information at a
detailed enough level.
Despite the challenges utilities continue to look for ways to measure productivity in a way that improves their
business
 Most utilities use a balanced set of metrics to obtain the clearest picture of performance
– Analysis teams place results into context of business cycles and external influences (e.g. weather)
– Trends in headline metrics are explained by the underlying supporting metrics
 Productivity metrics become integrated into the overall management of a utility aligned with corporate goals
– Understand and explain the drivers behind changing costs, internally and to regulators
– Target corporate initiatives at poorly performing areas
 Utilities leverage advanced IT systems such as mobile tracking devices to produce detailed productivity
metrics without creating large indirect costs
© Oliver Wyman  www.oliverwyman.com
13
Contents
 Background
 Executive summary
 Industry measurement of productivity
– By regulatory commissions
– At utilities
 Perspectives on productivity metrics
 Activity cost analysis to choose metrics with the biggest impact
– Principal cost driver analysis
– Specific metric recommendations
 Implementation considerations
© Oliver Wyman  www.oliverwyman.com
14
Analysis roadmap for choosing a suite of appropriate metrics
Understanding the breakdown of spend on resource is the first step to determine the
areas to measure.
Total Spend
33%
26%
18%
12%
11%
100%
14%
24%
80%
60%
Costs of materials
from all projects
excluded because
costs are driven by
base commodity
prices.
18%
31%
16%
31%
Costs outside the
top 20 projects of
main functional
areas. Not
considered in first
cut of metrics.
100%
79%
40%
69%
63%
Costs from top 20
projects in each
category. These
costs are broken
out further in the
next few slides.
45%
20%
Top 20 projects in the 4 areas
account for 64% of cost
0%
20%
40%
60%
Trans. - Capital
Dist. - Capital
TC
DC
© Oliver Wyman  www.oliverwyman.com
80%
100%
Dist. – OM&A. Trans. – OM&A. Other
DM
TM
15
Example: Distribution OM&A project metrics
DM
Eight of the biggest twenty distribution projects have suitable metrics to measure part of
their costs
#
100%
Percent
of Total
Cost
50%
Top 20 projects cover 79% of
distribution maintenance cost
0%
1
101
Projects ranked by largest cost
Metric
% Cumulative cost
2
Dx RofW Vegetation Management - Line
Cost of brush control per km of line
Clearing
Inconsistent over time
Dx O&M Trouble Call
3
CSO Sustainment
Outsourced
42%
4
OH Defect Correction & Insulator
Replacement
Cost per insulator replaced
46%
5
Smart Metering - OM&A
Cost per meter read
50%
6
Dx Overtime and Forestry Storm Costs
Inconsistent over time
53%
7
Dx RofW Vegetation Management Brush Control
Cost of brush control per km of line
57%
8
Dx Cable Locates
Cost per locate
60%
9
Dx Vegetation Management - Job Plan &
Inconsistent over time
Notify
62%
10
CSO Service Support - 3rd Party - MR &
Billing
Cost per bill
64%
11 Meter Reading - Prov. Lines
Cost per meter read
67%
CSO Regulatory Compliance - MR &
12
Billing
Inconsistent over time
69%
13 Dx Disconnects / Reconnects
Cost per disconnect
Cost per reconnect
70%
Inconsistent over time
72%
15 Small External Demand (Yearly)
Inadequate frame
73%
16 OPA Programs
Inconsistent over time
75%
17 DS Stations O&M
Inconsistent over time
76%
18 PCB and Other Waste Management
Inconsistent over time
77%
19 Field Special Investigations
Inconsistent over time
78%
CSO Regulatory Compliance 20
Collections
Inconsistent over time
79%
1
Cumulative cost of projects
Project
14
CSO Service Enhancements - MR &
Billing
Totals
19%
31%
79%
Legend
Relevant Metric
Need more info.
Not measurable
Note: Cost does not include materials or projects with negative or zero costs.
© Oliver Wyman  www.oliverwyman.com
16
Summary of recommended metrics
Twenty five selected metrics account for approximately twenty two percent of total costs.
 Aggregating the metric choices from
the previous pages, shows a good
coverage of total cost
– 22% of costs covered using 25 main
productivity metrics.
#
Metric
1
Cost of brush control per km of line
4.6%
2
Cost per meter install
3.9%
3
Cost per pole
3.7%
4
Cost per new service
1.1%
5
Cost per tower constructed
0.9%
 Some metrics cover multiple activities
across different segments (TC, T
OM&A, DC, DOM&A).
6
Cost per foundation
0.9%
7
Cost per km of line cleared
0.9%
8
Cost per meter read
1.0%
9
Cost per upgrade
0.7%
10
Cost per km of transmission line refurbished
0.6%
11
Cost per insulator replaced
0.5%
12
Cost per locate
0.6%
13
Cost per km for line patrol
0.4%
14
Cost per breaker
0.4%
15
Cost per transformer
0.4%
16
Cost per RTU
0.4%
17
Cost per bill
0.2%
18
Cost per km of line cleared
0.3%
19
Cost per unit – protection replacement
0.2%
20
Cost per Transformer Refurbishment
0.2%
21
Cost per cancellation
0.2%
22
Cost per insulator inspection
0.1%
23
Cost per disconnect
0.2%
24
Cost per reconnect
0.2%
25
Cost per inspection
 Some further subdivision of these
metrics may be required to allow
better comparisons.
– e.g. cost per pole could be sub
divided into cost per pole per ground
type.
 Note that estimations of cost coverage
was based on project titles.
– Further validation with business would
be required to confirm the
assumptions made.
– Large amount of projects could not be
understood from titles well enough to
suggest metrics.
% of total costs
Total
© Oliver Wyman  www.oliverwyman.com
0.1%
~22%
17
Contents
 Background
 Executive summary
 Industry measurement of productivity
– By regulatory commissions
– At utilities
 Perspectives on productivity metrics
 Activity cost analysis to choose metrics with the biggest impact
– Overview of methodology
– Principal cost driver analysis
– Specific metric recommendations
 Implementation considerations
© Oliver Wyman  www.oliverwyman.com
18
Implementation considerations
Hydro One will require a detailed plan to develop a set of productivity metrics that are
integrated and aligned with the overall corporate scorecard and direction.
High level activities to develop productivity
Productivity metric list, developed, ‘vetted’, iterated,
finalized (aligned with corporate scorecard)
Beta example ‘scorecard’ reports shared with
executives
Report templates signed off
Required system changes identified
System changes implemented
Rollout to Users (training, access etc)
Beta testing of results and reports
Production state
Ongoing monitoring of productivity improvement
initiatives
© Oliver Wyman  www.oliverwyman.com
19
© Oliver Wyman  www.oliverwyman.com
20