Research Plan:
2015 Upstream and Residential
Downstream Lighting Program
Impact Evaluation
(ED_I_Ltg_4)
California Public Utilities Commission
Date: July 22, 2016
LEGAL NOTICE
This document was prepared as an account of work sponsored by the California Public Utilities Commission.
It does not necessarily represent the views of the Commission or any of its employees except to the extent,
if any, that it has formally been approved by the Commission at a public meeting. For information regarding
any such action, communicate directly with the Commission at 505 Van Ness Avenue, San Francisco,
California 94102. Neither the Commission nor the State of California, nor any officer, employee, or any of its
contractors or subcontractors makes any warranty, express or implied, or assumes any legal liability
whatsoever for the contents of this document.
DNV GL - Energy – www.dnvgl.com/energy
1.1.1 Table of contents
1.1.1
1.1.2
1.1.3
Table of contents
List of figures
List of tables
2
2
2
1
INTRODUCTION ............................................................................................................ 4
1.1
Background
1.2
Objectives
4
1.3
2015 overview
5
1.4
Measure groups
7
1.5
Key research questions
9
2
STUDY METHODS ....................................................................................................... 12
2.1
Overview
12
2.2
2.2.1
2.2.2
2.2.3
2.2.4
2.2.5
Data sources
Consumer surveys
Supplier interviews
Retail lamp stock inventories and shopper intercept surveys
Lamp Choice Model
Program tracking data and other secondary data sources
12
13
15
18
19
21
2.3
2.3.1
2.3.2
2.3.3
2.3.4
Approach
Measure quantity adjustments
Gross Savings
Net savings
NTG analyses
22
22
23
28
37
3
REFERENCES.............................................................................................................. 38
4
1.1.2 List of figures
Figure 1. Conventional gross and net savings overview ...................................................................... 28
1.1.3 List of tables
Table 1. Quantity and share of upstream and residential downstream lighting measures by delivery
mechanism, sector and IOU, 2015 .................................................................................................... 5
Table 2. Quantity and share of residential and nonresidential upstream lighting measures by measure group
and IOU, 2015 ............................................................................................................................... 6
Table 3. Quantity and share of residential downstream lighting measures by measure group and IOU, 2015 7
Table 4. Uncertain measures to be prioritized - 2015 upstream and residential downstream lighting program
impact evaluation ........................................................................................................................... 8
Table 5. Quantity and share of upstream residential lighting measures by measure group and IOU, 2015
(high-priority measure groups highlighted) ......................................................................................... 9
Table 6. Research questions and associated data sources - 2015 upstream residential lighting impact
evaluation ................................................................................................................................... 13
Table 7. 2015 California actual market shares for lamps in the A-lamp replacement category ................. 16
Table 8. Replacement lamp categories and included technologies......................................................... 17
Table 9. 2015 California “counterfactual” market shares for lamps in the A-lamp
replacement category . 17
Table 11. Number of lamps addressed by intercepted lamp purchasers by measure group, 2013—2016 ... 19
Table 12. Ex post share of residential versus nonresidential upstream lighting measures by IOU, 2015 ..... 23
Table 13. Overview of 2015 upstream and residential downstream impact evaluation inputs and data sources
................................................................................................................................................. 24
DNV GL - Energy – www.dnvgl.com/energy
Table 14. Residential lighting HOU estimates by upstream lighting measure group and IOU ..................... 25
Table 15. Residential lighting peak CF by measure group and IOU ....................................................... 25
Table 16. CFL HVAC interactive effects factors by IOU ........................................................................ 27
Table 17. Net savings calculation example for A-lamp replacements (≤ 30 W) ...................................... 31
Table 18. Illustration of net savings calculation for A-lamp replacements (≤ 30 W) relative to traditional
incandescent/halogen lamps .......................................................................................................... 33
Table 19. Illustration of net savings calculation for A-lamp replacements (≤ 30 W) – all positive and
negative savings allocated to efficient technologies ............................................................................ 34
Table 20. Illustration of net savings calculation for A-lamp replacements (≤ 30 W) using different baseline
technologies ................................................................................................................................ 36
DNV GL - Energy – www.dnvgl.com/energy
1
INTRODUCTION
1.1 Background
This document presents the research plan for an impact evaluation of the California’s investor-owned utilities’
(IOU) 2015 upstream and residential downstream lighting energy-efficiency programs. The January 2016
Energy Division (ED) & Program Administrator Energy Efficiency Evaluation, Measurement and Verification
(EM&V) Plan, (Version 6) refers to this study as Lighting-16: 2015 Lighting Impact Evaluations and
Efficiency Savings and Performance Incentive (ESPI) evaluations. 1 This research plan addresses
nonresidential upstream and all residential lighting measures included in the IOUs’ 2015 lighting programs. 2
DNV GL will closely coordinate this work with other relevant studies.
This plan presents the overarching goals and objectives of the 2015 Upstream and Residential Downstream
Lighting Program Impact Evaluation included in work order (WO) ED_I_Ltg_4 (2013-2015 Lighting Impact
Evaluation and Market Research Studies). The plan also presents details regarding the scope, budget, and
timeline associated with this effort. This draft version of the plan currently represents collaboration between
the California Public Utilities Commission (CPUC) ED and DNV GL staff along with input from the CPUC ED’s
ex ante team and California’s electric and dual-fuel IOUs (PG&E, SCE, and SDG&E). Section 3 of this
document provides complete references for the reports and other sources cited herein.
1.2 Objectives
The January 2016 EM&V Plan states that “the goal of this study is to collect impact evaluation data on
selected lighting measures for the 2015 IOU lighting programs (and provide results as rapidly as possible).”
Objectives include:
1.
Verify the quantity of IOU-discounted upstream lighting products that were shipped, sold and
installed by residential and nonresidential customers within the PG&E, SCE and SDG&E service
territories during the 2015 program period;
2.
Estimate the gross energy and peak coincident demand impacts from these measures; and
3.
Determine an appropriate estimate of net energy and demand impacts.
The impact evaluation will verify and validate the IOU-reported energy savings and peak demand reduction
claims for upstream and residential downstream lighting measures and programs through a combination of
pass-through approaches (relying on information gathered in previous evaluation cycles) and primary data
collection. The evaluation will provide feedback to the IOUs regarding program performance, report on the
outcomes of long-term market transformation efforts, and provide results that will support future
improvements to IOU program design. To support these efforts, the evaluation team will prepare and
present timely and accurate reporting to the CPUC and/or IOUs.
1
2
CPUC ED, 2016. We refer to this document herein as the “January 2016 EM&V Plan” or “the Plan.”
Work order ED_I_Com_1 (Non-Residential Downstream Impact Evaluation) will address nonresidential, non-upstream lighting impact evaluation
issues. Another work order (TBD) will address nonresidential, non-upstream lighting market research issues.
DNV GL - Energy – www.dnvgl.com/energy
1.3 2015 overview
The tables below summarize the IOUs’ 2015 upstream (residential and nonresidential) and residential
downstream lighting program activities. As shown in Table 1, they provided incentives for nearly 15.7 million
lighting units via these three combinations of delivery mechanism and market sector in 2015. At 90% of the
total, residential upstream measures composed the largest share of total relevant lighting units, followed by
nonresidential upstream and residential downstream (at 6% and 4%, respectively). SCE provided incentives
for nearly three times as many units as either of the other IOUs at nearly 10.3 million (65% of total units
compared to 22% for PG&E and 13% for SDG&E).
Table 1. Quantity and share of upstream and residential downstream lighting measures by
delivery mechanism, sector and IOU, 2015
Measure Quantity
Delivery
Mechanism/Sector
Upstream – residential
Upstream – nonresidential
Downstream – residential
Total
Percent of Column
PG&E
SCE
SDG&E
Total
PG&E
SCE
SDG&E
Total
3,133,197
9,182,908
1,804,935
14,121,040
91%
90%
92%
90%
208,751
579,401
132,517
920,669
6%
6%
7%
6%
98,312
496,518
32,041
626,871
3%
5%
2%
4%
1,969,493 15,668,580
100%
100%
100%
100%
3,440,260 10,258,827
Percent of Total
22%
65%
13%
Source: 2015 IOU program tracking data (preliminary analyses)
DNV GL - Energy – www.dnvgl.com/energy
100%
Table 2 shows the quantity of residential and nonresidential upstream lighting measures for which the IOUs
provided incentives in 2015 as well as the share comprised by each measure group by IOU. Again, indoor
LED A-lamps comprised the largest share of total measures across IOUs at 25%, followed by indoor CFL
reflector lamps (17%), indoor LED reflector lamps (17%) and indoor high-wattage CFLs (> 30 W; 17%).
Together, these four measure groups comprised approximately three-quarters of all residential upstream
lighting measures in 2015 (76%).
Table 2. Quantity and share of residential and nonresidential upstream lighting measures by
measure group and IOU, 2015
Measure Group
Measure Quantity
Percent of Column
(Measure Quantity)
SDG&
Grand
PG&E
SCE
E
Total
PG&E
SCE
SDG&E
Grand
Total
Indoor CFL > 30 Watts
-
2,315,789
199,135
2,514,924
0%
24%
10%
17%
Indoor CFL 3-way
-
356,432
24,093
380,525
0%
4%
1%
3%
91,306
1,126,146
265,251
1,482,703
3%
12%
14%
10%
499,902
98,892
29,773
628,568
15%
1%
2%
4%
Indoor CFL fixture
-
8,973
351
9,324
0%
<1%
<1%
<1%
Indoor CFL globe
-
140,577
6,243
146,820
0%
1%
<1%
<1%
Indoor CFL reflector
-
2,540,155
88,564
2,628,719
0%
26%
5%
17%
665,456
-
213,596
879,052
20%
0%
11%
6%
1,381,811
1,548,699
808,637
3,739,147
41%
16%
42%
25%
694,575
1,626,451
299,753
2,620,779
21%
17%
15%
17%
-
195
5
200
0%
<1%
<1%
<1%
8,898
-
2,051
10,949
<1%
0%
<1%
<1%
3,341,948
9,762,309
1,937,452
15,041,709
100%
100%
100%
100%
Indoor CFL A-lamp
Indoor CFL basic
Indoor LED fixture
Indoor LED A-lamp
Indoor LED reflector
Outdoor HID
Refrig. case LED lighting
Total
Source: 2015 IOU program tracking data (preliminary analyses)
DNV GL - Energy – www.dnvgl.com/energy
Table 3 shows the total quantities of residential downstream lighting measures in the IOUs’ 2015 programs
by measure group and IOU as well as the share of total units comprised by each measure group. Forty
percent of the IOUs residential downstream lighting units in 2015 were indoor LED A-lamps. All other
measure groups accounted for much smaller shares of the residential downstream lighting sales; at the top
of that list are indoor basic CFLs and indoor CFL fixtures (at 13% and 12%, respectively).
Table 3. Quantity and share of residential downstream lighting measures by measure group and
IOU, 2015
Measure Group
Percent of Column
(Measure Quantity)
Measure Quantity
PG&E
SCE
SDG&E
Grand
Total
PG&E
SCE
SDG&E
Grand
Total
-
2
-
2
0%
<1%
0%
<1%
75,139
1,336
3,047
79,522
76%
0%
10%
13%
Indoor CFL fixture
2,768
71,378
1,783
75,929
3%
14%
6%
12%
Indoor CFL reflector
1,651
72
-
1,723
2%
<1%
0%
<1%
1
53,548
-
53,549
<1%
11%
0%
9%
641
53,235
795
54,671
1%
11%
2%
9%
11,776
218,024
20,141
249,941
12%
44%
63%
40%
-
-
1,738
1,738
0%
0%
5%
<1%
1,752
23,020
-
24,772
2%
5%
0%
4%
801
22,264
1,147
24,212
1%
4%
4%
4%
2,315
1,254
-
3,569
2%
<1%
0%
1%
Outdoor CFL fixture
956
33,911
2,820
37,687
1%
7%
9%
6%
Outdoor CFL reflector
303
26
-
329
<1%
<1%
0%
<1%
Outdoor LED fixture
209
393
570
1,172
0%
0%
2%
0%
-
18,055
-
18,055
0%
4%
0%
3%
98,312
496,518
32,041
626,871
100%
100%
100%
100%
Indoor CFL > 30 Watts
Indoor CFL basic
Indoor controls*
Indoor LED fixture
Indoor LED A-lamp
Indoor LED night light
Indoor LED reflector
Indoor other
Outdoor CFL basic
Outdoor LED other
Grand Total
* Complete measure group name is “Indoor Controls Wall or Ceiling Mounted Occupancy Sensor.”
Source: 2015 IOU program tracking data (preliminary analyses)
1.4 Measure groups
As described above, the IOUs provided incentives for nearly 15.7 million lighting units via their upstream
and residential downstream lighting programs in 2015. Because upstream lighting measures comprise more
than 96% of these measures delivered via the upstream mechanism and/or to residential IOU customers
(see Table 1), the 2015 evaluation will focus on upstream measures. As such, the evaluation team will “pass
through” all of the ex ante savings estimates associated with residential downstream units in the 2015
impact evaluation. Similarly, because upstream residential measures alone comprise 90% of all units
delivered via the upstream mechanism and/or to residential IOU customers, the 2015 impact evaluation will
focus exclusively on residential upstream measures. The evaluation team will also “pass through” all of the
ex ante savings estimates associated with nonresidential upstream units in the 2015 impact evaluation—
however, as we will discuss in Section 2.3.1.2 below, we will assess how the IOUs allocate upstream lighting
DNV GL - Energy – www.dnvgl.com/energy
units between the residential and nonresidential sectors and update this parameter as part of the evaluation
as necessary.
For residential upstream measures, the January 2016 EM&V plan notes that “the Energy Savings
Performance Incentive [ESPI] is a key driver in schedules for prioritizing and delivering impact studies for
uncertain measures as noted earlier. 3” The EM&V Plan identifies lists of measures to be prioritized in 2015
impact evaluations as a result of the uncertainty associated with specific impact evaluation parameters for
these measures. The list of uncertain lighting measures relevant to this evaluation includes high-wattage
CFLs, LED lamps, LED downlight fixtures, and plug-in LED night lights. Table 4 provides more details from
the ESPI measure list.
Table 4. Uncertain measures to be prioritized - 2015 upstream and residential downstream
lighting program impact evaluation
Measures
IOUs
Areas of Uncertainty
Screw-in CFLs of all types with PG&E, Update of the gross baseline assumptions to account for the type and
wattage of the lamp being replaced. Previous net-to-gross (NTG)
wattage values greater than
SCE,
30 Watts
SDG&E studies have not focused on high wattage CFL lamps.
Screw-in LED lamps including
downlight replacement kits
Update baseline assumptions and net savings including replaced lamp
PG&E,
for early retirement versus standard practice for normal replacement
SCE,
and replace-on-burnout; market move to LED technology and rapidly
SDG&E
changing products and pricing requires verification of assumptions.
LED night lights
SCE
Assumptions of replaced equipment and use uncertain.
Source: CPUC ED, 2016. Table 44 (2015 Uncertain Measures to be Prioritized). Pages 181—182.
The evaluation team also examined the upstream residential lighting measure groups included in the IOUs’
2015 program tracking data to help develop priorities for the 2015 impact evaluation (Table 5). Because the
“Lighting indoor LED fixture” measure group includes LED downlight replacement kits and other measures,
and because LED downlight replacement kits are the specific measures of interest for the uncertain
measures list, we have disaggregated the “Lighting indoor LED fixture” measure group into two subgroups in
the table to distinguish between downlights and other fixture types. 4
As shown, the IOUs’ 2015 residential upstream lighting programs included relatively large quantities of
measures for three of the ESPI measure groups—high-wattage CFLs (> 30 W) and LED lamps (which
includes LED A-lamps and LED reflector lamps). LED downlight replacement kits comprised only 6% of total
upstream residential lighting program units across IOUs but approximately 20% of PG&E’s total units. And
although LED night lights appear on the 2015 ESPI list, none of the IOUs provided incentives for these
3
4
CPUC ED, 2016. Page 6.
For the purposes of concise display in Table 5, we disaggregated the “Lighting indoor LED fixture” measure group into its component measures
and then grouped these into two subgroups: “LED recessed downlight” and “LED interior fixture – 28W.” The “LED recessed downlight” subgroup
includes the following measures: LED RECESSED DOWNLIGHT <10 W LED; LED RECESSED DOWNLIGHT >=10 W TO 12 W LED; LED RECESSED
DOWNLIGHT >12 W TO 25 W LED; LED RECESSED DOWNLIGHT 10 WATT; LED RECESSED DOWNLIGHT 11 WATT; LED RECESSED DOWNLIGHT
12 WATT; LED RECESSED DOWNLIGHT 15 WATT; LED RECESSED DOWNLIGHT 16 WATT; LED RECESSED DOWNLIGHT 21 WATT; LED RECESSED
DOWNLIGHT/RETROFIT 9 WATT; COMMERCIAL-LED RECESSED DOWNLIGHT 12 WATT; COMMERCIAL-LED RECESSED DOWNLIGHT 16 WATT. The
“LED interior fixture – 28W” subgroup includes only LED INTERIOR FIXTURE 28 WATT.
DNV GL - Energy – www.dnvgl.com/energy
measures through their upstream residential lighting programs in 2015. As such, DNV GL does not plan to
conduct specific impact evaluation activities to address LED night lights.
In addition to these, three other measure groups comprise at least 10% of total units for at least one of the
IOUs, including CFL reflector lamps, CFL A-lamps, and basic spiral CFLs. Because of the relatively large
quantities of these measures in the 2015 portfolio, we consider these additional measure groups as high
priorities for the 2015 impact evaluation in addition to the ESPI measure groups of high-wattage CFLs, LED
A-lamps, LED reflector lamps, and LED downlight replacement kits. These 7 measure groups comprise
approximately 96% of all residential upstream lighting units included in the IOUs’ 2015 programs and will be
the focus of the 2015 residential upstream lighting impact evaluation.
Table 5. Quantity and share of upstream residential lighting measures by measure group and IOU,
2015 (high-priority measure groups highlighted)
Measure Quantity
Measure Group
Percent of Column
(Measure Quantity)
SDG&
Grand
PG&E
SCE
E
Total
PG&E
SCE
SDG&E
Grand
Total
1,298,902
1,457,390
751,725
3,508,017
41%
16%
42%
25%
-
2,389,085
84,362
2,473,447
0%
26%
5%
18%
653,235
1,531,653
261,222
2,446,110
21%
17%
14%
17%
-
2,177,284
191,316
2,368,600
0%
24%
11%
17%
85,828
1,058,807
249,352
1,393,987
3%
12%
14%
10%
LED recessed downlight*†
625,324
-
210,514
835,837
20%
0%
12%
6%
Indoor CFL basic
469,908
93,042
24,280
587,230
15%
1%
1%
4%
Indoor CFL 3-way
-
335,070
24,093
359,163
0%
4%
1%
3%
Indoor CFL globe
-
140,577
5,979
146,556
0%
2%
<1%
1%
Outdoor LED fixture
-
-
2,051
2,051
0%
0%
<1%
<1%
LED interior fixture – 28W†
-
-
37
37
0%
0%
<1%
<1%
Outdoor CFL fixture
-
-
5
5
0%
0%
<1%
<1%
9,182,908
1,804,93
5
14,121,040
100%
100%
100%
100%
Indoor LED A-lamp
Indoor CFL reflector
Indoor LED reflector*
Indoor CFL > 30 Watts
Indoor CFL A-lamp*
Grand Total
3,133,197
Source: 2015 IOU program tracking data (preliminary analyses)
* 2015 uncertain measure (ESPI)
† Included in the “Lighting indoor LED fixture” measure group in previous tables.
1.5 Key research questions
The January 2016 EM&V Plan lists one key research question for the 2015 impact evaluation:
1.
What are the ex post savings results?
DNV GL staff thus worked with the stakeholders listed in Section 1.1 to identify additional relevant research
questions for the evaluation. These include:
2.
What is the appropriate baseline for residential upstream LED lamps?
3.
What is the appropriate baseline for residential upstream CFLs?
DNV GL - Energy – www.dnvgl.com/energy
4.
What is the freeridership level for residential upstream LED lamps?
5.
What is the freeridership level for residential upstream CFLs?
6.
In the absence of updated metering data, what are the best estimates of hours of use (HOU) by IOU?
7.
Is there any evidence that consumer satisfaction differs with LED lamps that meet the California
Quality LED Specification 5 versus LED lamps that do not meet the spec?
Question 1 (What are the ex post savings results?) addresses the key research question in the January 2016
EM&V Plan as well as the 2015 uncertain measure list’s requirement to “update … net savings” for screw-in
LED lamps for PG&E, SCE, and SDG&E (as well as updating net savings for other measures). 6
Question 2 (What is the appropriate baseline for residential upstream LED lamps?) addresses the 2015
uncertain measure list’s requirement to “update baseline assumptions … including replaced lamp for early
retirement versus standard practice for normal replacement and replace-on-burnout” for PG&E, SCE, and
SDG&E. To clarify, however, the evaluation team’s interpretation of the phrase “replaced lamp for early
retirement versus standard practice for normal replacement and replace-on-burnout” is that the uncertainty
lies in the extent to which “normal replacement” activities (or “standard practice”) involve “early retirement”
or “replace-on-burnout.” As such, the evaluation team will focus evaluation efforts (in part) on
understanding the share of installed LED lamps that replace functioning lamps (early retirement) versus
lamps that have stopped working (replace-on-burnout). The evaluation will also address the baseline
technology mix for LED lamps.
Question 3 (What is the appropriate baseline for residential upstream CFLs?) addresses the 2015 uncertain
measure list’s requirement to “update … the gross baseline assumptions to account for the type and wattage
of the lamp being replaced” for “screw-in CFLs of all types with wattage values greater than 30 Watts” for
PG&E, SCE, and SDG&E.
Question 4 (What is the freeridership level for residential upstream LED lamps?) and Question 5 (What is the
freeridership level for upstream CFLs?) address concerns raised by program administrators and other
stakeholders during discussions regarding results from the Impact Evaluation of 2013-14 Upstream and
Residential Downstream Lighting Program 7 and discussions of study priorities for the 2015 impact evaluation.
Question 6 (In the absence of updated metering data, what are the best estimates of HOU by IOU?) and
Question 7 (Is there any evidence that consumer satisfaction differs with LED lamps that meet the California
Quality LED Specification versus LED lamps that do not meet the spec?) arose from the same discussions
and IOU comments on the 2013-14 upstream and residential downstream lighting program impact
evaluation report.
•
5
6
7
Question 6 (regarding HOU estimates) will address (at least in part) concerns from SDG&E regarding
the differences in average hours of use for replacement lamps in SDG&E’s service territory versus
the other IOUs’ service territories. We expect a more definitive answer to this question from the
CEC, 2014.
CPUC ED, 2016. Table 44 (2015 Uncertain Measures to be Prioritized). Pages 181—182. The same source serves as the basis for all remaining
quotations in this section of the research plan (Section 1.5).
DNV GL, 2016.
DNV GL - Energy – www.dnvgl.com/energy
forthcoming in-home California lighting inventory and metering study. 8 As such, efforts to address
this question as part of the 2015 upstream and residential downstream lighting impact evaluation
will be limited.
•
The same is true with regard to Question 7 (regarding the California Quality LED Specification): the
in-home inventory and metering study will provide a more concrete opportunity to investigate
satisfaction with LED lamps in use in IOU customer households. That study will thus provide a more
appropriate vehicle for addressing the IOUs’ concerns regarding the possible market
transformational effects of their incentives for LED lamps that meet the spec. The 2015 upstream
and residential downstream lighting impact evaluation will provide high-level insights regarding IOU
customer satisfaction with LED lamps in general (as we will describe further in Section 2.2.2 below).
Note that there is no specific research question to address “assumptions of replaced equipment and use” for
LED night lights for SCE as mentioned in the 2015 uncertain measure list. SCE did not provide incentives for
LED night lights through its 2015 upstream residential lighting program (nor did PG&E or SDG&E).
8
DNV GL is currently engaged in the scoping phase (Phase 1) of this study and expects to deliver a finalized research plan for the study in early
2017. The January 2016 EM&V Plan provides more detail on the Phase 1 study (CPUC ED, 2016).
DNV GL - Energy – www.dnvgl.com/energy
2
STUDY METHODS
2.1 Overview
The 2015 impact evaluation will build on the methods from 2010-12 and 2013-14 upstream lighting impact
evaluations. The proposed approach for 2015 will still result in estimates of gross savings, net savings, and
NTG ratios but will attempt to calculate net savings at the market level. The approach relies upon marketlevel estimates of energy consumption by technology and measure group (including technologies such as
incandescent and halogen lamps, which are not included in the IOUs’ programs). Based on market-level
estimates of net savings along with estimates of gross savings, we propose to calculate NTG ratios for each
of the evaluated upstream lighting measure groups. Below we describe the impact evaluation data sources
(Section 02.3) and overall approach (Section 2.3).
2.2 Data sources
The 2015 impact evaluation relies upon seven data sources. Table 6 below shows the data sources aligned
with the research questions described above. We provide more details on these sources and the overall
study approach below.
DNV GL - Energy – www.dnvgl.com/energy
Table 6. Research questions and associated data sources - 2015 upstream residential lighting
impact evaluation
Retail Lamp Stock
Inventories
Shopper Intercept
Surveys
Lamp Choice Model
Program
Tracking Data
Secondary Data
Sources
1. What are the ex post savings results?
Supplier Interviews
Research Question
Consumer Surveys
Data Sources
X
X
X
X
X
X
X
2. What is the appropriate baseline for residential upstream LED
lamps?
X
X
X
3. What is the appropriate baseline for residential upstream CFLs?
X
X
X
4. What is the freeridership level for residential upstream
LED lamps?
X
X
X
X
X
5. What is the freeridership level for residential upstream CFLs?
X
X
X
X
X
6. In the absence of updated metering data, what are the
best estimates of HOU by IOU?
X
7. Is there any evidence that consumer satisfaction differs with
LED lamps that meet the California Quality LED Specification
versus LED lamps that do not meet the spec?
X
X
X
X*
* Supplier interviews will not focus on consumer satisfaction with LED lamps explicitly but instead focus on suppliers’ perspectives
regarding the influence of the CEC specification on their LED lamp sales – e.g., whether they would sell lamps that meet the CEC
spec in absence of the program.
2.2.1 Consumer surveys
DNV GL will conduct surveys with residential electric customers of PG&E, SCE, and SDG&E to obtain several
key inputs to the 2015 upstream and residential downstream lighting program impact evaluation. These
include:
1.
Installation rate. The consumer surveys will obtain details regarding the installation rates for CFLs
and LED lamps. The surveys will do so at the technology level (i.e., one installation rate for CFLs and
another for LED lamps). As part of this discussion, we will address the ESPI priority related to the
share of LED lamps that replace functioning lamps (early retirement) versus LED lamps that replace
lamps that have stopped working (replace-on-burnout).
2.
Satisfaction with LED lamps. While the energy savings associated with LED lamps that meet the
California Quality LED Specification do not differ from the savings for LED lamps that do not meet
the spec, the IOUs have suggested that the superior quality of the lamps that meet the spec will
have market transformational consequences in California. The theory is that the higher quality lamps
will result in high consumer satisfaction with LED lamps, ultimately leading to repeat purchases.
DNV GL - Energy – www.dnvgl.com/energy
Conversely, LED lamps that do not meet the spec are of lower quality, which could result in
consumer dissatisfaction that could reduce or eliminate their future purchases of LED lamps.
As described above in Section 1.5, the forthcoming in-home inventory and metering study will
provide a more concrete opportunity to investigate satisfaction with LED lamps in use in IOU
customer households, and thus provide a better vehicle to address the IOUs’ concerns than the 2015
impact evaluation. However, the consumer survey will briefly address satisfaction with LED lamps in
general. While we will be unable to distinguish during the survey whether the LED lamps purchased
by consumers do or do not meet the CEC spec, consistent and widespread satisfaction with LED
lamps may suggest that the specification does little to affect consumer satisfaction. Conversely, if
survey results suggest that consumer satisfaction with LED lamps is inconsistent, this may indicate
that there are differences in satisfaction with LED lamps that could be attributable (at least in part)
to the CEC spec. Such a result could demonstrate the need for more focused consumer research on
this topic.
3.
HOU assessment. Results from the 2010 California residential lighting metering study suggest
lower lighting HOU for SDG&E than for PG&E or SCE. 9 Using the consumer survey, we will test the
hypothesis that average daily HOU are as different among the IOUs as suggested in the 2010
metering study, and in the direction suggested by the study (i.e., that average daily HOU for SDG&E
is approximately one-third lower than the other IOUs’ estimates). Specifically, we will test whether
consumer-reported HOU are as different as indicated by the metering study. This test will not
confirm actual HOU, nor will we use consumer-reported values as an alternative HOU estimate.
However, the test will offer an indicator to either support or refute the differences across IOUs from
the metering study.
To construct the test, we will estimate the average daily HOU for three high-use rooms (e.g., kitchen,
living room, and master bedroom) for each IOU using 2010 metering study results. This will
establish the magnitude and directionality of differences in average daily HOU by IOU for the
purposes of this assessment. During the consumer surveys, we will ask respondents to estimate the
average daily HOU for lamps in each of these three rooms. Once we complete the consumer surveys,
we will average the HOU estimates from each respondent across the three rooms and then
aggregate by IOU.
The test will have one of 2 outcomes:
a) Null hypothesis is not rejected: customer-reported HOU do not provide strong evidence that
the difference among IOUs is less than that found in the metering study. In this case, we will
continue to use IOU-specific HOU values based on the 2010 metering study.
b) Null hypothesis is rejected: customer-reported HOU provide strong evidence that the
difference among IOUs is less than that found in the metering study. In this case, we will
develop statewide estimates of HOU based on the 2010 metering study and use these statewide
estimates for SDG&E in the 2015 impact evaluation. For PG&E and SCE, we will use the IOUspecific HOU estimates based on the 2010 metering study.
As stated in Section 1.5 above, we do not intend this assessment to represent the best and final
estimate of average daily HOU by IOU. Instead, the goal is to expend a reasonable level of effort in
the near-term to reassess the validity of the 2010 estimates in advance of updated (and more
precise) estimates from the forthcoming in-home lighting inventory and metering study.
4.
9
Baseline technology mix and wattage for high-wattage CFLs. During the 2016 consumer
surveys, DNV GL will attempt to identify purchasers of high-wattage CFLs (> 30 W). We will ask
these consumers to identify the technology and wattage of the lamp replaced with high-wattage
CFLs. DNV GL recognizes that it will be challenging to obtain meaningful point estimates of lamp
wattage over the phone. While we would likely obtain more reliable information from face-to-face
interactions with consumers, no such interactions are planned in support of the 2015 impact
evaluation. As such, the consumer survey represents the best-available opportunity to attempt to
address this issue. Nonetheless, it may be necessary to rely on secondary data sources (or a
KEMA, Inc., 2010, Volume 1. See Table 56: Recommended Average Residential Daily HOU by IOU with Confidence Intervals (Appendix B, page
112) which suggests average daily HOU estimates of 1.3 hours/day for SDG&E and 1.9 hours/day for both PG&E and SCE.
DNV GL - Energy – www.dnvgl.com/energy
combination of secondary sources and other supporting information) to estimate baseline wattage
for high-wattage CFLs for the 2015 impact evaluation. DNV GL notes that the forthcoming in-home
lighting inventory and metering study provides an ideal opportunity to address baseline technology
and wattage with consumers for installed high-wattage CFLs and have incorporated this objective
into preliminary plans for that study. The inventory and metering effort will provide more concrete
estimates of baseline wattage for high-wattage CFLs.
5.
Inputs into LCM. Section 2.2.4 below describes the LCM in detail as well as the expected inputs to
and outputs from the model for this study. From the consumer surveys, we will obtain:
a.
Distribution of lamp purchases by retail channel. One challenge in using the LCM in
previous impact evaluations is that the model relies, in part, on results from the in-store
shopper intercept surveys. The intercept surveys are, by necessity, based on a convenience
sampling approach. To improve the LCM’s ability to represent the distribution of lamp
purchases by retail channel within the purchaser population, we will include questions in the
consumer surveys to address recent purchase locations (retail channels).
b.
Customer demographics together with recent lamp purchase information. As
described in the detailed LCM discussion, we will use the consumer survey respondents,
rather that intercept survey respondents, to represent the universe of lamp purchase
decisions.
We will first review the 2015 consumer telephone survey results to determine whether they provide
a sufficient level of detail to serve as LCM inputs. If so, will use the 2015 consumer telephone survey
results regarding the distribution of lamp purchases by retail channel to ensure that the LCM reflects
this distribution rather than the distribution of intercepted purchasers. If not, we will collect
information in the 2016 consumer survey to serve this purpose.
6.
Insights into channel shift. The consumer survey will include questions to address whether
shoppers typically choose from among the lamps available to them in a preferred retail channel or if
they will move to a different retail channel to purchase a preferred type of lamp. The supplier
interviews will also get their perspectives as to which retail channels and which lamp technologies
are most likely to be experiencing channel shift.
As part of the consumer survey task, we will also investigate:
•
Which mode or modes may be most appropriate (for example, a telephone survey, online survey,
etc.)
•
Whether we need to rake survey results to balance demographics
•
Whether it is necessary to address these objectives in one or more consumer survey efforts (to
minimize the length of individual surveys).
2.2.2 Supplier interviews
DNV GL will conduct approximately 40 interviews with lamp supplier representatives with the primary goals
of assessing the 2015 program’s influence on lamp sales and estimating market-level sales of replacement
lamps to California consumers in 2015. The most efficient respondent population for the supplier interviews
is likely the manufacturers’ representatives (because there are many more retailers than manufacturers)
and retail buyers for large retail chains. However, there may be occasions when interviews with retail buyers
for smaller retail chains or even store managers may be appropriate. For example, if our interviews with
lighting manufacturers or retail buyers leave certain retail channels underrepresented, we may want to add
some interviews with smaller retail buyers or store managers from these underrepresented channels. We
refer to all of these representatives collectively as “suppliers” herein. DNV GL will contact these
DNV GL - Energy – www.dnvgl.com/energy
representatives as appropriate using one general interview guide that we will modify to suit the specific
respondent characteristics. 10
Starting from scratch (rather than modifying interview guides used in prior upstream lighting program
evaluations), DNV GL will create an interview guide to address:
•
Share of 2015 lamp sales represented by upstream lighting program sales. In advance of
each interview, we will prepare a summary of the quantity of lamps for which each supplier received
2015 upstream lighting program incentives, by technology and measure group. During the interview,
we will ask the representative to confirm the quantity and to estimate the percentage of total lamp
sales for that measure group represented by the program-discounted quantity.
•
Total 2015 lamp sales. For each replacement lamp category, we will ask each supplier
representative to estimate the proportion of his or her company’s 2015 California sales represented
by each common alternative lamp type within that replacement lamp category. For example, for the
A-lamp replacement category, we will ask respondents to estimate the proportion of their 2015
California sales represented by each lamp type shown in Table 7. Since we will know the actual
quantities of their lamp sales through the upstream program from the program tracking data, we will
be able to estimate their sales volumes for the other lamp types not discounted by the program
based on their relative market shares.
Table 7. 2015 California actual market shares for lamps in the A-lamp replacement
category
A-Lamp Replacements (≤ 30 W)
% of Supplier’s
2015 Lamp Sales
in California
CFL A-lamps/basic spiral CFLs through the upstream program
__%
CFL A-lamps/basic spiral CFLs not through the upstream program
__%
LED A-lamps through the upstream program
__%
LED A-lamps not through the upstream program
__%
EISA-compliant incandescent/halogen A-lamps
__%
Traditional incandescent/halogen A-lamps
__%
Total 2015 California sales of A-lamp replacements
100%
We will repeat this exercise for two other replacement lamp categories (reflector replacements and
high-wattage replacements) although in these cases, the lamp technologies in each category will be
more limited than for A-lamps. Table 8 shows all three relevant replacement lamp categories and
their included technologies.
10
For example, if a small independent retailer received a large volume of lamps through the program, it may be most effective to speak with the
store manager regarding these lamps (possibly in addition to the manufacturer’s representative[s]). If a major retail chain received a large
volume of lamps through the program, it may be most effective to speak with the corporate-level retail buyer regarding these lamps (again,
possibly in addition to the manufacturer’s representative[s]).
DNV GL - Energy – www.dnvgl.com/energy
Table 8. Replacement lamp categories and included technologies
Technology
•
Replacement Lamp Category
A-lamp
Reflector
High-Wattage
X
CFL spiral
X
CFL (same shape as category name)
X
X
LED
X
X
EISA-compliant incandescent/halogen
X
Traditional incandescent/halogen
X
X
X
X
Estimated 2015 lamp sales in absence of the upstream program. Next, we will ask the
supplier representative to provide similar market share estimates for the counterfactual scenarios—
in other words, what their market shares would have been in absence of the program. Table 9 shows
the specific line items for which we will request market share estimates using the example of the Alamp replacement category.
Table 9. 2015 California “counterfactual” market shares for lamps in the A-lamp
replacement category
A-Lamp Replacements (≤ 30 W)
% of Supplier’s
2015 Lamp Sales in California
without Upstream Program
CFL A-lamps/basic spiral CFLs
__%
LED A-lamps
__%
EISA-compliant incandescent/halogen A-lamps
__%
Traditional incandescent/halogen A-lamps
__%
Total 2015 California sales of A-lamp replacements
100%
It is possible that the absence of the program would have led not only to a shifting of the relative
sales shares of each lamp type but also to a change in the total volume of 2015 California sales. To
account for this, we will ask the interviewees whether there would be any change in the total volume
of their 2015 California sales for lamps in a given replacement lamp category (e.g., A-lamps,
reflectors, high-wattage lamps) in the absence of the program. If so, we will obtain details regarding
the directionality (larger or smaller) and the magnitude (% change).
It is also possible that some suppliers or retail buyers will be unwilling to estimate their relative sales
shares with and without the program. In such cases, we will give them the option of providing a
more generic estimate for the 2015 California market as a whole.
•
Insights into channel shift. We will also ask the supplier representatives whether they would
expect sales to shift among channels for any lamp types in absence of the program. For example,
they may report that the absence of the program might lead to a significant shift in sales in LED Alamps from the discount channel to the home improvement channel due to discount stores not
stocking these lamps at full price. In such cases, we would ask the suppliers or retail buyers to try to
estimate the direction and magnitude of these channel shift effects.
•
LED lamps that do not meet the CEC spec. DNV GL also recognizes the importance of addressing
LED lamps that do and do not meet the California Quality LED Lamp Specification in the supplier
interviews. This issue is particularly important given that during previous interviews, some suppliers
suggest that they would not have sold any LED lamps that met the spec in absence of the upstream
lighting program. We will ask supplier representatives whether they sold any LED lamps that met the
CEC spec without program incentives. We will also ask them whether they would have sold any LED
DNV GL - Energy – www.dnvgl.com/energy
lamps that met the spec in 2015 if the program did not exist. Finally, we will ask them to describe
any other influences of the CEC specification on their LED lamp sales in California.
The supplier sample will include major lamp manufacturers whether or not the manufacturers received
incentives from the IOUs through the 2015 upstream lighting program, and whether or not the
manufacturers produced lamps that were eligible for program incentives. In other words, if there are major
manufacturers that produce only (for example) traditional incandescent lamps or program-eligible lamps
that did not receive IOU discounts in 2015, we will attempt to interview them to obtain estimates of 2015
lamp sales. The ultimate goal is to best represent market-level sales of replacement lamps through brickand-mortar retail stores to California consumers.
2.2.3 Retail lamp stock inventories and shopper intercept surveys
In late 2015 through early 2016, DNV GL staff conducted stock inventories of lamps for sale in California
retail stores throughout PG&E, SCE, and SDG&E service territories. We have conducted similar inventories
roughly every 6 to 12 months since 2011. During the last several phases of stock inventories, we conducted
concurrent shopper intercept surveys with lamp purchasers. The stock inventories gathered detailed
information regarding all residential replacement lamps stocked in the stores other than linear fluorescent
lamps, and the shopper intercept surveys focused on shopper purchasing decisions and installation
intentions for the newly-purchased lamps. 11
DNV GL field staff conducted surveys in chain and independent retail stores, including stores that
participated in the IOUs’ 2015 upstream lighting programs as well as non-participating stores. Field staff
spent a minimum of four hours in each store completing the lamp stock inventories and attempting to
intercept shoppers. We completed 207 lamp stock inventories purchaser 433 intercept surveys during the
winter 2015-16 period.
Field staff completed purchaser intercept surveys opportunistically—that is, with individuals who were
shopping during the time periods during which we conducted intercept surveys in specific stores. As such,
results from the intercept surveys may not represent the broader population of shoppers purchasing
replacement lamps at various stores throughout the year. Nonetheless, given the range in timeframes and
store types in which we conducted these surveys, results provide general indications of shopper preferences,
price sensitivity, lamp installation intentions, and so on. Additionally, the 2015 impact evaluation will
leverage consumer survey results to better reflect the distribution of lamp sales by retail channel in our
analyses.
To support the 2015 impact evaluation, DNV GL will leverage the retail lamp stock inventory results in two
ways:
1.
To support the LCM. The LCM reflects the lamp prices and availability that DNV GL staff observed
in retail stores during the retail stock inventories. We will update the LCM to ensure that it
represents the mix of lamp stock found on retail shelves during the most recent phase of stock
inventory research conducted by DNV GL (winter 2015-16).
11
Field researchers also conducted shopper intercept surveys with respondents who were not purchasing lamps (non-purchaser shopper intercept
surveys), but the impact evaluation focuses on surveys with lamp purchasers only because these surveys included detailed questions regarding
lamp replacement intentions.
DNV GL - Energy – www.dnvgl.com/energy
2.
To support the supplier interviews. We use the retail stock inventory results to estimate the
share of lamp stock by technology and measure groups for specific retail chains and manufacturers
to support the supplier interview effort described above.
We will also use the purchaser intercept survey results in two ways:
1.
To support estimates of baseline technology mix. During the purchaser intercept surveys, we
asked respondents to identify the lamp technologies they intended to replace with their new lamps
(for up to two lamps per respondent). Error! Reference source not found. shows the number of
lamps in each measure group addressed by purchaser intercept surveys during the summer 2013,
winter 2014-15, and winter 2015-16 data collection periods. As shown, we have robust data from
the winter 2014-15 and winter 2015-16 periods (which include the program period) for many
measure groups, but if include one additional period (summer 2013), we have robust data on which
to base these analyses for six of the seven groups. The exception is high-wattage CFLs, for which
the purchaser intercept surveys addressed only 13 lamps during the relevant data collection periods.
For high-wattage CFLs, we will thus attempt to obtain details regarding the baseline technology mix
from the 2016 consumer surveys.
Table 10. Number of lamps addressed by intercepted lamp purchasers by measure group,
2013—2016
Measure Group
Quantity of Lamps Addressed by Purchaser Intercept Surveys
Summer
2013
Winter
2014-15
Winter
2015-16
Overall
180
181
101
462
CFL A-lamp ≤ 30 W
22
12
13
47
CFL reflector ≤ 30 W
31
21
9
61
4
4
5
13
LED A-lamp (all wattages)
54
94
190
338
LED reflector (all wattages)
57
93
81
231
348
405
399*
1,152
Basic spiral CFL ≤ 30 W
High-wattage CFL (> 30 W)
Total Lamps
Note that the total lamps shown here is smaller than the total quantity of intercepted purchasers during the winter 2015-16
period because some survey respondents purchased lamp types not shown in this table (e.g., traditional incandescent lamps).
2.
To support the LCM. The intercept surveys ask lamp purchasers to rank a set of hypothetical lamp
technologies as most-to-least likely to buy given specific price points. The LCM uses these data to
estimate model coefficients, as described in Section 2.2.4. This evaluation will update model
coefficients using 2015-16 purchaser intercept survey data.
2.2.4 Lamp Choice Model
DNV GL developed the LCM for the 2010-12 impact evaluation to quantify consumer responses to upstream
lighting incentives. The underlying program theory is that providing discounts for a CFL or LED lamp makes
that CFL or LED lamp a more attractive choice than the alternatives. The program’s effects include providing
lower-priced lamps in retail stores than would be available without the program and enabling specific retail
DNV GL - Energy – www.dnvgl.com/energy
stores (such as those in the discount channel) to stock lamps that otherwise would not meet their price point
requirements. Discrete choice models are the analytical framework designed to address these types of
effects. These models combine the relevant information about each possible choice— for example, the lamp
price and consumer characteristics—and assign a probability to each of the choices. To estimate attribution,
we use the model to estimate the market share for each lamp type with and without the program in place.
Our approach is as follows:
1.
Re-estimate the LCM. We will re-estimate the LCM with 2015-16 purchaser intercept survey data.
Re-estimation ensures that the model reflects consumer price sensitivities regarding the different
lamp technologies available in brick-and-mortar retail stores during the 2015 program period.
2.
Estimate market shares under three scenarios by channel. We will estimate market shares
using a simulation-based approach. The simulation takes two inputs. The first input is a
representation of consumers. We will use the respondents in the 2016 consumer survey to form this
input. Unlike the intercept survey data, the consumer survey data are a representative sample of
consumers in the lamp market. The second input is a representation of available lamp choices based
on retail lamp stock inventory data. We will run the simulation against three scenarios:
•
With-program scenario. This scenario reflects the lamp prices and availability that DNV GL
observed in retail stores during the retail lamp stock inventories conducted in winter 201516. This scenario results in an estimate of the share of program lamp sales for each
modelled technology.
•
No-discount scenario. This scenario reflects the lamp prices that consumers would have
seen in California retail stores in 2015 in the absence of IOU discounts. DNV GL estimated
price differences based on clearly-labeled IOU discounts in the stores or by matching lamps
to program tracking data. This scenario results in a counterfactual estimate of market shares
that would have occurred if only prices on program-discounted lamps changed due no
program activity. This scenario represents the first of two “no program” scenarios.
•
Constrained scenario. In addition to the price effects described in the no-discount scenario,
the constrained scenario reflects stocking changes that would have occurred in absence of
the program. During our supplier interviews, when a respondent indicated that his or her
company would not have sold a lamp through a specific retail channel in the absence of the
program, we consider those lamps to be program-reliant. (For example, if a supplier
representative told us he or she would not have sold LED A-lamps to discount stores without
upstream lighting program incentives, we would consider the presence of these lamps in
discount stores to be program-reliant.) This scenario results in a counterfactual estimate of
market shares if program-reliant lamps were not in stores and if the IOUs did not discount
lamps. This scenario represents the second “no program” scenario.
In prior evaluations, we have used these results to directly estimate a NTG ratio that represents the relative
percent changes in a given technology’s market shift. However, this approach leads to some challenges in
the presence of competing efficient technologies. For the 2015 evaluation, the team proposes to use the
LCM to estimate the overall difference in market shares with and without the upstream lighting program
considering all technologies jointly. The market share differences together with market volume and unit
energy consumption (UEC) will yield market-level net savings. We will combine the market shares generated
from the LCM with the market shares estimated by supplier interviews to produce the best estimate for
program impacts on replacement lamp market share, and thus net savings (see Section 2.3.3 for greater
detail).
DNV GL - Energy – www.dnvgl.com/energy
The evaluation team recognizes that this approach depends on a robust estimate of total market volume for
each set of competing technologies. In the event that a market volume sales estimate cannot be produced
with sufficient rigor in the time and budget available for this evaluation, the evaluation team will leverage
the vetted and proven methodology implemented in the 2013-14 upstream lighting impact evaluation. This
methodology uses supplier and consumer data to estimate NTG ratios, and applies those ratios to gross
savings, which produces net savings. Nevertheless, a market-level interpretation of net savings is the most
comprehensive method to evaluate upstream program performance, and will be the preferred approach for
this evaluation.
2.2.5 Program tracking data and other secondary data sources
The 2015 impact evaluation will also rely on program tracking data as well as the result of other EM&V
efforts (including the California Lighting and Appliance Saturation Study [CLASS] 12).
2.2.5.1
Program tracking data
Each of the IOUs uploads program tracking data onto a centralized server to be downloaded by the
evaluation team. The team will analyze, clean, re-categorize, reformat, and merge these separate datasets
into one program tracking database. 13 The tracking data will provide details regarding the quantity of
lighting measures shipped as well as details regarding the manufacturers and retailers involved in the 2015
programs, the latter of which will provide the sample frame for the in-depth lighting supplier interviews. It
will also produce estimates of average discounted wattage per measure group, and will be used to pass
through ex-ante results for specific parameters not addressed in the 2015 evaluation.
2.2.5.2
Other EM&V reports
We will mine data from other EM&V studies to support the overall evaluation efforts described herein for
work order ED_I_Ltg_4. These data sources may include:
12
13
•
Impact Evaluation of 2013-14 California Upstream and Residential Downstream Lighting
Programs (DNV GL, 2016). This study included all lighting measures associated with upstream
delivery mechanisms and all downstream lighting measures targeted at the residential sector. The
impact evaluation focused on seven measures that collectively accounted for over 90% percent of
each IOUs’ ex ante net savings from upstream and residential downstream measures. These
measures included basic spiral CFLs, CFL A-lamps, CFL globes, CFL reflectors, LED A-lamps and LED
reflectors. Several of the impact evaluation parameters and methodologies used in the 2013-14
program cycle will be “passed through” and utilized in the current evaluation.
•
Impact Evaluation of the 2010-12 Residential/Advanced/Upstream Lighting Programs
(DNV GL, 2014b). As with the 2013-14 impact evaluation, this study included all lighting measures
associated with upstream delivery mechanisms and all downstream lighting measures targeted at
the residential sector. This evaluation focused heavily on four measure groups that each individually
made up at least 1 percent of each IOU’s portfolio-level kWh or kW claims: CFL A-lamp, CFL basic,
CFL globes, and CFL reflectors. Many methodologies that were used in the 2013-14 evaluation and
that will continue to be used in this evaluation were developed under WO28.
•
CLASS (DNV GL, 2014a). The CLASS study updates and augments saturation and efficiency
characteristics from previous CLASS studies conducted in 2005 and 2000 for use in understanding
future energy savings potential and past accomplishments in the residential sector. The 2012 CLASS
DNV GL, 2014a.
Note that we have completed some preliminary analyses of these data in support of this research plan.
DNV GL - Energy – www.dnvgl.com/energy
study included onsite observations on a sample of 1,987 single-family, multi-family and mobile
home residences with individually-metered electric accounts across the service territories of PG&E,
SCE and SDG&E. The 2015 impact evaluation will rely on CLASS to update the delta Watts, HOU,
and peak coincidence factors for CFLs and LED lamps (see Section 2.3.2 for more detail).
2.3 Approach
As described in Section 1.2 above, the 2015 impact evaluation has three main objectives:
1.
Verify the quantity of IOU-discounted upstream lighting products that were shipped, sold and
installed by residential and nonresidential customers within the PG&E, SCE and SDG&E service
territories during the 2015 program period
2.
Estimate the gross energy and peak coincident demand impacts from these measures
3.
Determine an appropriate estimate of net energy and demand impacts
As such, the evaluation approach includes three key components:
1.
Adjust rebated measure quantities
2.
Develop gross savings inputs
3.
Develop net savings estimate
We explore these components in more detail below. As part of this evaluation, we will attempt to estimate
the program’s net impacts relative to changes in the California market for residential replacement lamps (as
noted in Section 2.1 above). We will then produce a NTG ratio for each measure group by dividing the net
savings associated with the measure group by the gross savings associated with the measure group. In the
event that collected data are unable to produce a rigorous estimate of total market volume, the evaluation
will instead rely on the vetted approach used in the 2013-14 evaluation, and apply a NTG ratio to the gross
savings estimate. 14
2.3.1 Measure quantity adjustments
In recent evaluations of the upstream and residential downstream lighting programs, evaluators have
applied three adjustments to the quantity of rebated measures claimed by the IOUs as having been sold to
their residential and nonresidential customers during the program period. As with the 2013-14 impact
evaluation, 15 the 2015 evaluation will maintain the values associated with each of these adjustments,
including:
1.
Quantity of IOU-discounted products shipped by participating manufacturers to retailers as
determined through the verification of a sample of program invoices/applications
14
15
2.
Percent of IOU-discounted products purchased by residential versus nonresidential customers
3.
Percent of IOU-discounted products purchased by non-IOU customers (i.e., leakage)
Note that the net savings methodology from the 2013-14 evaluation requires a gross savings estimate without efficient-to-efficient lamp
replacements. To accommodate the mixed baseline, the methodology estimates net savings by applying a NTG ratio to the legacy gross savings
estimate. It then calculates the alternative, mixed-baseline net-to-gross ratio by holding net savings constant, and dividing it by the mixedbaseline gross savings. See the 2013-14 impact evaluation for more details and an example of that calculation (DNV GL, 2016).
DNV GL, 2016.
DNV GL - Energy – www.dnvgl.com/energy
We provide more detail regarding measure quantity adjustments below.
2.3.1.1
Invoice verification
The 2013-14 residential and upstream lighting impact evaluation relied upon the same invoice verification
rate as the 2010-12 evaluation. 16 For the 2010-12 evaluation, evaluators verified the quantity of IOUdiscounted units shipped by participating manufacturers to retailers based on their review of a sample of
program invoices and applications. The evaluation estimated an ultimate verification rate of 100% for all
IOUs and retail channels. As such, we will apply the 100% verification rate in the 2015 impact evaluation.
2.3.1.2
Residential versus nonresidential
To estimate the portion of upstream CFLs that are installed in nonresidential applications, the 2013-14 and
2010-12 evaluations relied on the results of two onsite survey studies conducted during the 2010-12
period—the CLASS 17 and the Commercial Market Share Tracking Study. 18 These efforts yielded the
residential versus nonresidential shares of total upstream lighting program measures shown in Table 12. We
will apply these estimates in the 2015 impact evaluation.
Table 11. Ex post share of residential versus nonresidential upstream lighting measures by IOU,
2015
Share of Upstream Measures
IOU
Nonresidential
Residential
PG&E
7%
93%
SCE
6%
94%
SDG&E
6%
94%
Overall
7%
93%
Source: DNV GL, 2016.
2.3.1.3
Leakage
Leakage is defined as the quantity of program-discounted upstream lamps that “leak” out of the collective
IOU service territories—in other words, IOU-discounted lamps that are ultimately installed outside of the
IOUs’ service territories. Due to the lack of strong data supporting leakage, no adjustment to quantity was
applied in the impact evaluations of the IOUs’ 2013-14 or 2010-12 programs. We will therefore apply the
same 0% leakage rate in the 2015 impact evaluation.
2.3.2 Gross Savings
This section describes the methodology for analyzing gross impacts. There are six elements to these
analyses:
16
17
18
DNV GL, 2014b.
DNV GL, 2014a.
Itron, Inc., 2014.
DNV GL - Energy – www.dnvgl.com/energy
1.
Average daily HOU
2.
Average percent of installed measures operating at peak (coincidence factor, or CF)
3.
Wattage displaced by IOU-discounted products (delta Watts)
4.
HVAC interactive effects (IE)
5.
Unit energy savings (UES) estimates (kWh/year and peak kW)
6.
Installation rate
Table 13 provides an overview of these inputs and their associated data sources for the 2015 impact
evaluation by market sector (residential and nonresidential) and program delivery mechanism (upstream
and downstream).
Table 12. Overview of 2015 upstream and residential downstream impact evaluation inputs and
data sources
Market Sector and Program Delivery Mechanism
Evaluation Input
Residential Upstream
HOU
2010 metering study; 2016 consumer survey
(to test difference in 2010 HOU results for
SDG&E)
Peak coincidence factor
2010 metering study
Delta Watts
a. Baseline technology
mix
b. Baseline wattage
c. Replacement
technology mix and
wattage
a. 2016 consumer survey for high-wattage
CFLs (> 30 W); Purchaser intercept
surveys for all other measures
b. 2016 consumer survey for high-wattage
CFLs (> 30 W); 2012 CLASS inventory
for other measures; all with assumed
baseline technology replacement ratios
from purchaser intercept surveys
c. 2015 program tracking data
HVAC interactive effects
Pass through from ex ante
Installation rate
2016 consumer survey
Residential
Downstream
Nonresidential
Upstream
Pass through from ex ante*
* With exception of residential/nonresidential split.
2.3.2.1
HOU
Estimates of the average daily HOU for residential lighting in this evaluation will use the same approach as
in the 2013-14 impact evaluation with the exception of a 2016 consumer survey exercise to test the validity
of IOU-specific estimates for SDG&E (as detailed in Section 2.2.1 above). As explained, should the consumer
survey exercise conclude that the customer-reported HOU do not provide strong evidence that the difference
among IOUs is less than that found in the metering study, we will apply IOU-specific HOU values based on
the 2010 metering study (as shown in Table 14). If the customer-reported HOU do provide strong evidence
that the difference among IOUs is less than that found in the metering study, we will develop statewide
estimates of HOU based on the 2010 metering study and use these statewide estimates for SDG&E in the
2015 impact evaluation. In this case, we will still use the IOU-specific HOU estimates based on the 2010
metering study for PG&E and SCE.
DNV GL - Energy – www.dnvgl.com/energy
Table 13. Residential lighting HOU estimates by upstream lighting measure group and IOU
Evaluated Upstream Lighting
Measure Group
PG&E
SCE
HOU
90%
CI
MSB CFL basic spiral ≤ 30 W
1.6
MSB CFL A-lamp ≤ 30 W
MSB CFL reflector ≤ 30 W
SDG&E
HOU
90%
CI
±0.1
1.9
1.5
±0.2
1.7
MSB CFL high-wattage (> 30 W)
Overall
HOU
90%
CI
HOU
90%
CI
±0.2
1.4
±0.2
1.7
±0.1
1.9
±0.2
1.3
±0.3
1.6
±0.2
±0.3
1.9
±0.2
1.2
±0.4
1.7
±0.2
*
*
*
*
*
*
1.9
±0.2
LED A-lamp, all wattages
*
*
*
*
*
*
2.1
±0.2
LED reflector, all wattages
*
*
*
*
*
*
2.1
±0.2
* Sample sizes were too small to produce IOU-specific estimates. We will apply the overall estimates in calculating impacts. Please refer to
DNV GL, 2014b for more details regarding metering study sample sizes.
2.3.2.2
Peak coincidence factor
Peak CF represents the average percent of time that a lamp is used during the peak period. The peak
periods vary by climate zone. Similar to the HOU estimates, the estimates for CF will be passed through
from the 2013-14 impact evaluation. We derived these estimates based on the logger data collected for the
2010 metering study 19 and applied to in-home lamp inventory data collected during CLASS. 20 Table 15
shows the peak CF values that we will pass through in the 2015 impact evaluation.
Table 14. Residential lighting peak CF by measure group and IOU
Measure Group
PG&E
SCE
SDG&E
Overall
Peak
CF
90%
CI
Peak
CF
90%
CI
Peak
CF
90%
CI
Peak
CF
90%
CI
MSB CFL basic spiral ≤ 30 W
0.05
±0.01
0.07
±0.01
0.04
±0.02
0.06
±0.01
MSB CFL A-lamp ≤ 30 W
0.05
±0.02
0.06
±0.02
0.04
±0.02
0.05
±0.01
MSB CFL reflector ≤ 30 W
0.05
±0.02
0.06
±0.02
0.04
±0.03
0.06
±0.02
MSB CFL high-wattage (> 30 W)
*
*
*
*
*
*
0.06
±0.01
LED A-lamp, all wattages
*
*
*
*
*
*
0.06
±0.02
LED reflector, all wattages
*
*
*
*
*
*
0.06
±0.02
* Sample sizes were too small to produce IOU-specific estimates. We will apply the overall estimates in calculating impacts. Please refer to
DNV GL, 2014b for more details regarding metering study sample sizes.
2.3.2.3
Delta Watts
For each evaluated measure group, the delta Watts calculation consists of replacement lamp wattage,
baseline technology mix, and baseline wattage. We will calculate delta Watts as the difference between a
measure group’s technology-weighted baseline wattage and its average replacement lamp wattage. We
provide more details on these inputs below.
Replacement lamp wattage
19
20
DNV GL, 2014b.
DNV GL, 2014a.
DNV GL - Energy – www.dnvgl.com/energy
We will calculate the average replacement lamp wattage for each measure group based on the 2015
program tracking data.
Baseline technology mix
As described in Section 2.2.3 above, the 2015 impact evaluation will rely upon estimates of baseline
technology mix by measure group from in-store purchaser intercept survey data.
Baseline wattage
We will use the baseline technology mix (described above) and the average baseline lamp wattage from the
CLASS in-home lamp inventory data to estimate baseline wattage all measure groups except high-wattage
CFLs (> 30 W). For high-wattage CFLs, we will use 2016 consumer survey data to estimate baseline wattage.
For all measure groups, we will estimate baseline wattage based on the average lamp wattage for each
baseline technology weighted by the proportion of all baseline measures represented by each technology.
Because we collected the CLASS data prior to extensive adoption of EISA-compliant general purpose lamps,
we will perform a fairly simple calculation to account for an increased adoption in EISA-compliant lamps. We
will consider the midpoint between the average CLASS incandescent wattage and the average 2015-16 Shelf
inventory EISA wattage as the incandescent/EISA baseline wattage. The forthcoming in-home lamp
inventory and metering study will obtain more accurate and detailed information regarding EISA-compliant
lamp adoption among the California IOUs’ electric customers.
2.3.2.4
HVAC interactive effects
The Database for Energy Efficient Resources (DEER) includes savings factors for kWh, kW and therms
savings for indoor CFLs and LED lamps. The DEER team develops HVAC interactive effects (HVAC IE)
estimates for internal load changing measures such as interior lighting and appliances and other plug loads
measures. These effects can alter the "direct" electric and gas impacts for those measures due to resulting
changes in heating and cooling HVAC system energy use. As such, these savings factors are applied to the
direct impacts as a multiplier for both kWh and kW and factor of therm/kWh for therm impacts.
Table 9 shows the multipliers used for the upstream CFL measures evaluated in the 2013-14 upstream and
residential downstream impact evaluation report, which we obtained from DEER 2011. 21 DEER does not
currently distinguish among different CFL measures (e.g., basic spiral CFLs versus CFL A-lamps), therefore
we apply the same HVAC IE factors to all evaluated CFL and LED lamp measure groups. We show the factors
in Table 9 by IOU and building type. The residential and commercial factors are the weighted average of all
existing building types for residential or commercial buildings with a given IOU.
21
DNV GL, 2016.
DNV GL - Energy – www.dnvgl.com/energy
Table 15. CFL HVAC interactive effects factors by IOU
Building
Type
CFL HVAC
Interactive Effect
Adjustment
PG&E
SCE
SDG&E
kWh
1.02
1.07
1.03
kW
1.33
1.40
1.23
Therms
-0.025
-0.019
-0.018
kWh
1.06
1.12
1.12
kW
1.21
1.24
1.23
Therms
-0.0061
-0.0032
-0.0028
Residential
Commercial
IOU
Source: DEER 2011.
2.3.2.5
Installation rate
In the 2015 impact evaluation (as in the 2013-14 evaluation), the installation rate is the percentage of all
upstream lamps that that will ultimately be installed. DNV GL will develop updated installation rates for CFLs
and LED lamps in the 2016 consumer survey (described in Section 2.2.1 above).
2.3.2.6
Unit energy savings
For measures discounted through the IOUs’ 2015 upstream lighting programs, we will compute unit energy
savings (UES) using Equation 1 below and peak demand reductions using Equation 2 below.
Equation 1. Unit energy savings
1 𝑘𝑘𝑘𝑘ℎ
365 𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑
𝑘𝑘𝑘𝑘ℎ
� = Δ𝑊𝑊𝑊𝑊𝑊𝑊𝑊𝑊𝑠𝑠𝐿𝐿 [𝑊𝑊 ] ∗ 𝐻𝐻𝐻𝐻𝑈𝑈𝐿𝐿 [ℎ] ∗
𝑈𝑈𝑈𝑈𝑆𝑆𝐿𝐿 �
∗
∗ 𝐼𝐼𝐸𝐸𝐿𝐿 [𝑘𝑘𝑘𝑘ℎ]
1000 𝑊𝑊ℎ
1 𝑦𝑦𝑦𝑦𝑦𝑦𝑦𝑦
𝑦𝑦𝑦𝑦𝑦𝑦𝑦𝑦
Where:
ΔWattsL = average displaced (delta) wattage for IOU-discounted lamp measure group, L, in Watts (W)
HOUL = annual average HOU for IOU-discounted lamp measure group, L, in hours (h)
IEL = DEER HVAC interactive effects in kilowatt-hours (kWh)
DNV GL - Energy – www.dnvgl.com/energy
Equation 2. Peak demand reduction
1 𝑘𝑘𝑘𝑘
𝑘𝑘𝑘𝑘
� = Δ𝑊𝑊𝑊𝑊𝑊𝑊𝑊𝑊𝑠𝑠𝐿𝐿 [𝑊𝑊 ] ∗ 𝐶𝐶𝐹𝐹𝐿𝐿 ∗
𝑈𝑈𝑈𝑈𝑆𝑆𝐿𝐿 �
∗ 𝐼𝐼𝐸𝐸𝐿𝐿 [𝑘𝑘𝑘𝑘 ]
1000 𝑊𝑊
𝑦𝑦𝑦𝑦𝑦𝑦𝑦𝑦
Where:
ΔWL = average displaced (delta) wattage for IOU-discounted lamp measure group, L, in Watts (W)
CFL = average percent on at peak for IOU-discounted lamp measure group, L
IEL = DEER HVAC interactive effects in kiloWatts (kW)
2.3.2.7
Estimate gross savings
We will calculate gross savings as energy savings, peak demand reduction, and increased natural gas usage
(based on interactive effects). Figure 1 illustrates the components used to calculate gross savings.
Figure 1. Conventional gross and net savings overview
2.3.3 Net savings
We will build our net savings analysis on methods used in the previous two cycles, with some improvements
based on our experiences with that work. As in past evaluation cycles, we rely on supply-side and demandside estimates of program influence and integrate the results based on the strengths and limitations of each
approach. Our approach to estimating net savings involves the following steps:
1.
Market share estimation. We will determine the market share of each competing technology
with and without the program in place separately for 3 categories of replacement lamps:
a.
A-lamp replacements (≤ 30 W): CFL A-lamps/basic spiral CFLs; LED A-lamps; EISAcompliant incandescent/halogen A-lamps; traditional incandescent/halogen A-lamps
b.
Reflector lamp replacements (≤ 30 W): CFL reflector lamps, LED reflector lamps,
incandescent/halogen reflector lamps
c.
High-wattage lamps: high-wattage CFLs (> 30W), equivalent EISA-compliant
incandescent/halogen A-lamps; equivalent traditional incandescent/halogen A-lamps
2.
Market volume estimation. We will then determine the total 2015 sales in California for lamps
within each set of competing technologies.
3.
Net savings calculation. We will determine the difference in total consumption for each set of
competing technologies with and without the program in place. This difference is based on the
difference in market shares, the total market sales, and the unit energy consumption (UEC) for
each measure. We will calculate the UEC based on the gross parameters defined in Section 2.3.2,
DNV GL - Energy – www.dnvgl.com/energy
except that instead of applying delta Watts, we will use 2015-16 retail lamp stock inventory data
to calculate and apply the average wattage of each lamp technology on the 2015 market. We
will allocate the total net savings for each replacement lamp category to the efficient
technologies.
NTG calculation. Finally, we will determine the NTG ratios for each measure group as the ratio
4.
of net savings to gross savings at the measure group level.
We describe these steps in more detail below (including the basic methods and planned improvements).
2.3.3.1
Market share estimation
The following section describes the two perspectives we will use to estimate lighting market shares in the
presence and in the absence of the program.
Demand-side estimate: LCM
This method determines market share by estimating and applying a set of lamp choice models (described in
more detail in Section 2.2.4). For the 2015 impact evaluation, we will use the winter 2015-16 purchaser
intercept survey data to fit the models. We will use similar model forms in the 2015 impact evaluation as in
the 2013-14 evaluation.
Once we have estimated the LCM model coefficients, we will apply the model to a simulated population of
lighting purchases to determine the market shares of competing technologies with and without the program
in place. The relative change in market share for each program-supported technology is the basis for the net
savings estimate.
The 2013-14 evaluation encountered two challenges in applying the LCM:
1.
Prior to their 2013-14 programs, the IOUs offered CFL A-lamps ≤ 30 W and LED A-lamps ≤ 30 W in
limited quantities, with a larger share of upstream incentives directed toward basic spiral CFLs
≤ 30 W. In 2013-14, the programs offered all three of these competing technologies in varying
combinations. As a result, the effect of the program on one technology was dependent on which of
the other technologies were simultaneously present, with or without program discounts. We thus
had to segment our analyses based on the combination of technologies present which led to a
complex analysis and thin data for some channel-segment combinations. In addition, assessing the
effect of the program separately for each technology required interpretation of how to account for
the effect increasing one efficient technology at the expense of another. In the 2015 program,
however, there are few cases in which discounted technologies compete within the same
replacement lamp category (but there is still the potential for competition between discounted
energy-efficient lamps and energy-efficient lamps not discounted in the same stores).
2.
We simulated the universe of lighting purchase choices based on purchaser intercept respondents. If
we did not have enough respondents for a particular replacement lamp category in a particular
channel, the LCM was unable to estimate a NTG ratio for that channel and the relevant measure
group(s). In addition, we did not have a basis for weighting the simulated responses within a
channel and segment to the population level.
To address these challenges, the 2015 impact evaluation analyses include the following changes.
DNV GL - Energy – www.dnvgl.com/energy
1.
We will simulate the universe of lighting purchase choices based on consumer survey results rather
than on in-store purchaser intercept results. The consumer survey results represent a broader crosssection of lighting purchasers, and will not be restricted to customers present in particular stores on
particular days. Use of this population reduces concerns as to how representative the simulated
universe is, and also will make more cases available to represent particular choices.
2.
We will account for the effect of competing efficient technologies by considering the overall change
in market shares across all competing technologies.
Supply-side estimate: supplier interviews
As part of the supplier interviews, we will ask each respondent to estimate the market share of each
technology within a replacement lamp category, with and without the program in place (described in more
detail in Section 2.2.2 above). These market share estimates will not be channel-specific.
Consolidated market share estimates
For each replacement lamp category, we will obtain the proportion of lamp purchases made in each retail
channel from consumer survey responses. We will weight the channel-specific market shares by the channel
proportions to determine the overall market shares based on the demand-side approach.
To determine the ultimate with-program and without-program market share estimates, we will take a
weighted average of the supplier-reported market share estimates with and without the program and those
based on the LCM.
2.3.3.2
Market volume estimation
We will estimate total market volume for each replacement lamp category by two means:
1.
Stock turnover analysis of 2012 CLASS data
2.
Supplier survey responses
Stock turnover analysis
We will estimate the total volume of purchases for each replacement lamp category by assuming steady
state total stock. For each technology, we will have:
•
The total number of lamps in sockets during 2012 (from CLASS 22)
•
The rated lifetime hours of use (from lamp package information/manufacturers’ data)
•
Annual hours of use (from prior HOU analyses)
Average unit life in years is the ratio of lifetime hours of use to annual hours of use. We will explore
alternative sources of assumptions for lifetime hours of use, or for average unit life.
22
DNV GL, 2014a.
DNV GL - Energy – www.dnvgl.com/energy
We will combine these elements by dividing the number of lamps in sockets by the average lifetime to
estimate the number of lamp replacements occurring each year. While some of the replacements will come
from in-home storage, the steady-state assumption is that whatever is withdrawn from storage will be
replaced into storage, so that the number of replacements required is equal to the number of purchases.
This assumption is not strictly true, but is a reasonable high-level assumption. Summing the annual
replacements over technologies within a replacement lamp category gives the total market sales for that
category.
Supplier interview responses
As discussed in more detail earlier in the plan, we will also obtain information from lamp suppliers that will
allow us to generate a second estimate of total market size. The main approach will involve aggregating
each supplier’s estimates of his or her company’s sales (both program and non-program sales as well as
efficient and inefficient sales) to come up with an estimate of total market volume for lamps in each
replacement lamp category.
Consolidated estimate
We will assess the quality of supplier interview responses and the uncertainties associated with the stock
turnover analysis and develop a best estimate of overall market volume.
2.3.3.3
Net savings calculation
Combining markets shares, market volume, and UEC yields the total annual energy consumption of each
technology for the with-program or without-program scenarios. Summing over technologies yields the total
consumption for the replacement lamp category. Subtracting the with-program total from the withoutprogram total yields the total net savings. Table 17 illustrates the calculation for lamps in the A-lamp
replacement category assuming a total market volume of 100 million lamps.
Table 16. Net savings calculation example for A-lamp replacements (≤ 30 W)
Market Share
Technology
Adjusted
UEC
Energy Consumption
Without
Program
(A)
With
Program
(B)
Savings
(A-B)
Without
Program
With
Program
CFL A-lamp
25%
20%
62
1,547,095,680
1,237,676,544
309,419,136
CFL spiral
40%
36%
43
1,705,772,160
1,535,194,944
170,577,216
LED A-lamp
10%
12%
17
173,354,054
208,024,865
(34,670,811)
5%
9%
8
37,855,382
68,139,688
(30,284,306)
20%
23%
17
346,708,109
398,714,325
(52,006,216)
3,810,785,386
3,447,750,367
EISA inc/hal A-lamp
Traditional inc/hal A-lamp
Net Savings
This calculation provides overall savings, but does not explicitly allocate savings to each of the efficient
technologies. To obtain that allocation, we first re-calculate the net savings starting with the savings for
each technology relative to the least efficient technology (the traditional incandescent/halogen A-lamp).
DNV GL - Energy – www.dnvgl.com/energy
Table 18 illustrates this calculation. The UES for each technology is the difference between the incandescent
UEC and the technology’s UEC. The resulting net savings is the same as for the initial calculation.
DNV GL - Energy – www.dnvgl.com/energy
Table 17. Illustration of net savings calculation for A-lamp replacements (≤ 30 W) relative to
traditional incandescent/halogen lamps
Technology
Market Share
Without
Program
With
Program
Traditional inc/hal A-lamp
25%
EISA inc/hal A-lamp
Relative to Traditional
Incandescent/Halogen
UES
Total Savings
20%
-
-
40%
36%
19
(76,958,093)
CFL A-lamp
10%
12%
45
89,096,844
Basic spiral CFL
20%
23%
45
133,645,265
5%
9%
54
217,251,003
LED A-lamp
Net Savings
363,035,019
The calculation shown above still has negative savings for EISA-compliant lamps, as well as positive savings
for the efficient technologies. We will allocate the negative savings from EISA-compliant lamps to the other
technologies in proportion to the net incremental number of units of each technology due to the program, as
illustrated in Table 19.
DNV GL - Energy – www.dnvgl.com/energy
Table 18. Illustration of net savings calculation for A-lamp replacements (≤ 30 W) – all positive and negative savings
allocated to efficient technologies
Market Shares
Technology
Without
With
Program Program
Traditional inc/hal A-lamp
EISA inc/hal A-lamp
CFL A-lamp
Basic spiral CFL
LED A-lamp
Net Savings
DNV GL - Energy – www.dnvgl.com/energy
25%
40%
10%
20%
5%
20%
36%
12%
23%
9%
Relative to Traditional
Incandescent/Halogen
UES
19
45
45
54
Total Savings
(76,958,093)
89,096,844
133,645,265
217,251,003
363,035,019
Number of
Incremental
Lamps
Efficient
Tech % of
Net Shift
Negative
Savings from
EISA Lamps
Reallocated to
Efficient Lamps
Proportional to
Incremental
Lamps
(5,000,000)
(4,000,000)
2,000,000
3,000,000
4,000,000
22%
33%
44%
(17,101,798)
(25,652,698)
(34,203,597)
Efficient
Technology
Net Savings
(with EISACompliant
Reallocated)
71,995,045
107,992,568
183,047,406
363,035,019
These analyses provide net savings with estimates for each measure group calculated relative to traditional
incandescent/halogen lamps and EISA-compliant incandescent/halogen lamps only. Since LED lamps may
displace CFLs as well as both traditional and EISA-compliant incandescent/halogen lamps, we will calculate
LED lamp savings relative to a baseline that includes a proportion of CFLs. This calculation involves
multiplying the net incremental LED lamps by the UES determined with a mixed baseline of CFLs, traditional
incandescent/halogen lamps, and EISA-compliant incandescent/halogen lamps.
While this calculation yields a reasonable representation of net savings for LED lamps, the total net savings
across all efficient technologies is still the total indicated in Table 17, Table 18, and Table 19. Reducing the
net savings allocated to LED lamps means allocating more of the total net savings to CFLs. The savings
allocated to LED lamps in Table 19 is the number of incremental LED lamps due to the program times a UES
based on a baseline that is the weighted average of traditional and EISA-compliant incandescent/halogen
lamp UECs, with weights proportional to the total of each lamp displaced by the program. Implicitly, this
assumes that each LED lamp displaces traditional and EISA-compliant incandescent/halogen lamps in these
same proportions. Calculating LED lamp savings instead using a UES that includes some CFLs in the baseline
implicitly says that each LED lamp displaces some CFLs, and fewer traditional and EISA-compliant
incandescent/halogen lamps. If the LED lamps are displacing some CFLs, but the net incremental CFLs is as
indicated in Table 19, there must be additional CFLs attributable to the program that are balanced out by
those displaced by LED lamps, and these additional CFLs displace the traditional and EISA-compliant
incandescent/halogen lamps that were previously attributed to the LED lamps. This is the source of the
savings re-allocated to CFLs when the mixed baseline of CFLs, traditional incandescent/halogen lamps, and
EISA-compliant incandescent/halogen lamps is used for LED lamps.
Table 20 illustrates this reallocation. In this case, we assume that the mix of lamps displaced by LED lamps
includes 15% CFL A-lamps, 25% basic spiral CFLs, traditional/EISA-compliant incandescent and halogen
lamps in the same proportions as indicated in Table 19. The first two columns of Table 20 show the savings
relative to a blended traditional/EISA-compliant incandescent and halogen baseline. Relative to this baseline,
savings for traditional and EISA-compliant incandescent and halogen lamps are 0. Savings for the efficient
technologies are the same as those in Table 19. The next block of the table shows the LED lamp savings
recalculated relative to the mixed baseline of CFLs, traditional incandescent/halogen lamps, and EISAcompliant incandescent/halogen lamps with the remainder of the original LED lamp savings re-allocated to
the CFLs. The final block in Table 20 shows the total net savings for each technology with this re-allocation
and the equivalent incremental lamps implied.
DNV GL - Energy – www.dnvgl.com/energy
Table 19. Illustration of net savings calculation for A-lamp replacements (≤ 30 W) using different baseline technologies
Savings Relative to
Traditional/EISA Mix
Savings Relative to
Mixed LED Baseline
UES
UES
Technology
Total
Total
Savings with Different Baseline
Savings
Reallocated
from LED
Lamps to
CFLs
Baseline
UES
Savings Total
Equivalent
Incremental
Lamps
Traditional inc/hal A-lamp
(9)
42,754,496
Traditional inc/hal
EISA inc/hal
(9)
42,754,496
(5,000,000)
EISA inc/hal A-lamp
11
42,754,496)
Traditional inc/hal
EISA inc/hal
11
(42,754,496)
(4,000,000)
CFL A-lamp
36
71,995,045
21,598,514
Traditional inc/hal
EISA inc/hal
36
93,593,559
2,600,000
Basic spiral CFL
36
107,992,568
35,997,523
Traditional inc/hal
EISA inc/hal
36
143,990,090
4,000,000
LED A-lamp
46
183,047,406
(57,596,036)
Traditional inc/hal
EISA inc/hal
CFL
31.4
125,451,370
4,000,000
363,035,019
1,600,000
Net Savings
DNV GL - Energy – www.dnvgl.com/energy
363,035,019
31.4
125,451,370
-
The adjustment illustrated in Table 20 assumes that the baseline for CFLs is based on the net shift in nonefficient lamps, while the baseline for LED lamps is based on the replacement rates obtained from the
consumer surveys. We will perform a similar set of adjustments to provide net savings for each of the
technologies relative to baselines defined by the replacement rate mix, Adjustment all efficient technologies
in this way will require some iteration to reconcile to the total net savings.
2.3.4 NTG analyses
In prior evaluations, we used the LCM and the supplier interviews to estimate NTG ratios for each measure
group and multiplied the NTG ratio for each measure group by gross savings to estimate net savings. In this
evaluation, we start from market-level net savings using the approach described in Section 2.3.3., then
determine a NTG ratio for each measure group and IOU by dividing the net savings for each measure group
and IOU by the respective gross savings. This approach will not yield channel-specific NTG ratios; however,
the LCM will produce channel-specific program and non-program market shares, which will provide valuable
insights regarding the impacts of the program at the channel level.
DNV GL - Energy – www.dnvgl.com/energy
3
REFERENCES
CEC, 2014. Voluntary California Quality Light-Emitting Diode (LED) Lamp Specification: A Voluntary
Minimum Specification for “California Quality” LED Lamps. CEC-400-2015-001. December, 2014.
CPUC ED, 2016. 2013-2016 Energy Division & Program Administrator Energy Efficiency Evaluation,
Measurement and Verification Plan - Version 6. January 14, 2016.
DNV GL, 2016. Impact Evaluation of 2013-14 Upstream and Residential Downstream Lighting Programs.
Prepared for the CPUC ED. April 1, 2016.
_____, 2014a. Final Report - WO21: Residential On-site Study: California Lighting and Appliance Saturation
Study (CLASS 2012). Prepared for the CPUC ED. November 24, 2014.
_____, 2014b. California Upstream and Residential Lighting Impact Evaluation Work Order 28 (WO28) Final
Report. Prepared for the CPUC ED. August 4, 2014.
KEMA, Inc., 2010. Final Evaluation Report: Upstream Lighting Program, Volumes 1 and 2. Supported by The
Cadmus Group, Inc.; Itron, Inc.; PA Consulting Group; and Jai J. Mitchell Analytics. Prepared for the
CPUC ED. February 8, 2010.
Itron, Inc., 2014. California Commercial Market Share Tracking Study. Prepared for the CPUC ED. November
13, 2014.
DNV GL - Energy – www.dnvgl.com/energy
ABOUT DNV GL
Driven by our purpose of safeguarding life, property and the environment, DNV GL enables organizations to
advance the safety and sustainability of their business. We provide classification and technical assurance
along with software and independent expert advisory services to the maritime, oil and gas, and energy
industries. We also provide certification services to customers across a wide range of industries. Operating in
more than 100 countries, our 16,000 professionals are dedicated to helping our customers make the world
safer, smarter greener.
DNV GL - Energy – www.dnvgl.com/energy