Potential and Goals Study for 2015 and Beyond EM&V Update Meeting Notes
January 30, 2015
Slides
2015 PG Study Jan
30 EM#26V Update DAWG_FINAL v2.pptx
Summary of Action Items
 Navigant to review data for the instantaneous water heater efficiency ratings, comments from
stakeholders were that the efficiency level seemed high.
 Navigant to provide a more detailed description of each measure: data sources, assumptions,
etc.
o Can include this information in an appendix in the report.
o More detailed description already exists in the 2013 model and 2013 MICS, it’s just not
easily accessible.
 Navigant to review CFL hours of use data between Res. vs. Com (result of a discussion about
upstream CFL crossover into the commercial sector).
 Cynthia Mitchell (TURN) would like to understand the magnitude of cost updates for key
measures.
o Navigant can provide this data after the MICS measure updates have been finalized.
 Haley Goodson (TURN) would like to see a chart that indicates the data sources and mapping to
measures and parameters.
o Navigant can put this together for the March 17 draft timeline.
 Phil Toth (SCE) to provide IOU whole building measure bundles for Navigant’s review – will feed
in to Stage 2.
 Craig Tyler (SCG) asked about codes and standards layering
o Navigant points to the CPUC 10-12 EM&V study of IOU C&S programs. The topic of C&S
Layering is not the subject of this DAWG meeting.
 CPUC to circulate verbal comments captured from the meeting
o Stakeholders will have two weeks to submit comments after the document is released.
Summary of Discussions
 General questions arose concerning the timeline and timing of Tasks and Stages.
o What is the overall timeline of this update?
 Navigant: March 17 is the Stage 1 draft results. Stage 1 will be locked down at
the end of May so the results can feed into the regulatory cycle. Goals get
finalized in July and voted on in August.
o Will there be another workshop to discuss the results?
 CPUC: Yes, we will schedule a workshop to review the draft.

There are still questions around the use of workpapers in the 2016 Study from NRDC, SCE,
TURN.
o CPUC: An internal discussion needs to happen at the CPUC before further decision is
made on the use of work papers in the 2016 study.
1

Existing baseline
o CPUC: Will host a workshop in the next few months to discuss.
o Is there an opportunity to see everyone’s comments on the existing baseline memo?
 CPUC: Yes. Will follow up via email to connect those that commented with each
other.
o What will be the impact of the existing baseline on the potential and goals?
 CPUC: Navigant will prepare the technical analysis and present to stakeholders
before the CPUC makes a decision.
 Phil Toth: Certain customer types have a maintenance budget, but not a
replacement budget. There are savings here, but current rules don’t allow it.

James Fine (Environmental Defense Fund)
o What is layering of codes and standards?
 Navigant: A good resource is DNV GL and Cadmus' Impact Evaluation report. See
Codes and Standards Advocacy Programs.

Haley Goodson (TURN)
o Is there a chart that indicates the EM&V studies the team used and what it was used
for?
o Can we have a workshop for a discussion on this before the lockdown occurs?
 Navigant: Yes, there will be a series of workshops following submittal of the
March 17th draft report.

Chris Kavalec (CEC)
o When Navigant indicates realization rate, what does that entail?
 Navigant: We look at the realization rate based on each measure. In terms of
lighting, we had to look at what that really meant. Often times the results show
realization rate, but the data show it's an operational hours issue or some other
parameter issue. We base our measure updates on the parameters, not just the
realization rates.

Peter Miller (NRDC)
o Define the terms “research” and “calculated.” (with regards to slide 12)
 Navigant: With respect to unit energy savings, calculated means we back into it
by unit energy consumption. The ex-ante team provided model results. It's
calculated because it's not in a database. The term "research" refers to the
process we went through in 2013, when data was not as strong. We didn't relook at CEUS, etc. this time around so that term should be scratched for 2015.
o What does “High” mean?
 Navigant: This is a relative comparison of the quality and availability of certain
data. Our 2008 values were the best available at the time but we were clear
where data was still uncertain or lacking. Given the new EM&V studies and the
coverage is better now. “High” is really "best available." For example, the new
Itron Cost study provides more relevant data than DEER 2008 costs. In general,
we're conservative. If we weren't 100% sure of the mapping of new data to our
measures, we deferred to the 2013 study values.
 Navigant: With respect to the variables updated, for example, measure costs we used 2008 data for the 2013 study. Now we have the Itron measure cost
study, but it's not marked as "High" because it didn't always map one-to-one to
our measures.
2

Cynthia Mitchell (TURN)
o Why is low-bay HID to T5? Is there ever T8 to T5?
 Navigant: This measure is a specific application, usually in warehouses or similar
building type. The typical retrofit is HID to T5, not T8 to T5. The same applies for
high-bay applications. For linear fluorescents, we do include a T8 to low-wattage
T8, but not T8 to T5.
o Will there be a CPUC workshop to review EM&V results?
 CPUC: There is not a workshop planned at this time.
o How did Navigant deal with low lighting realization rates?
 Navigant: When we compared lighting results from the 06-08 evaluation and the
10-12 evaluation, both were low, but they were based on the same ex-ante
data, so it's consistent over time. Our 2013 study corrected for this, thus the
low realization rate was already built into the results in the last study. The 13-14
results will likely be closer to a realization of 100%. The key update we made
with lighting measure was updating hours of use in the savings calculation.
o How do EM&V results impact the market achievable?
 Navigant: The potential model reports gross savings, so while the CEC can look
at our net results, we report on gross. If savings come in lower than expected in
the EM&V results, all other things being equal, the market achievable would
decrease.
o Maybe some of this differential is just market changes and consumer preferences, but
some of it because of poor program design, are we building in sub-optimal program
design into projections?
 Navigant: This relates to model calibration and can be taken up in the next DAW
meeting. Also, the program design issue, our reference case is an assumption
about the reference case (and policies and funding levels) today. What-if
scenarios can address program design changes. We're using best available data,
it's not perfect.
o What are the Whole Building measure bundles?
 Navigant: Bundles are created from the customer perspective. We have Energy
Upgrade California data, so we don't have to assemble bundles for Residential
retrofits. The ARUP –study used building simulation model for ZNE in various
CZs so data was available there. In 2013, we made bundles, calculated energy
savings for commercial retrofits as no program exists from which data is
available. In Stage 1, we're refreshing the data in the existing bundles, not
changing which measures are in those bundles.

David Peck (ORA)
o Why does the MICS use unit energy consumption, not unit energy savings?
 Navigant: DEER provides UES, but we need to model consumption (not just
savings) for the purposes of modeling customer adoption.

Rob Ruben (SDG&E)
o Does “best available” mean “most current?”
 Navigant: “Best available” data was often the most recent data, but if we had
any question around the integrity of the most recent data, we would default to
previously used data. We understand that some EM&V data doesn’t have the
best sample size.
3

Phil Toth (SCE)
o How are Res/Com splits for upstream lighting programs handled?
o Does Task 2 include a scenario update?
 Navigant: We have yet to have that discussion with the CEC. Shortly after Task 1
is completed, we'll meet with the CEC to start discussing those.
o Will the results get down past IOU territory?
 Navigant: The level of disaggregation is under discussion for Phase 2 of the 2015
Study, but for Phase 1 it will not change from the 2013. Currently the results can
be disaggregated by climate zone and IOU service territory, though the results
are rolled up to IOU service territory in the goals and in the final report.
 CPUC: Can set up a workshop to discuss this topic.
o What is an “approved workpaper?”
 CPUC: Commission-reviewed workpapers or those with dispositions. Does not
include workpapers that have passed the 40-day review period without being
reviewed by the CPUC.
o How does Navigant account for code updates to measures?
 We updated measures to current codes to reflect above code savings.
 Many of our ET measures are improvements on existing technology, so the
assumptions for hours of use, etc. are the same, but the savings will be code to
LED rather than code to CFL.
o Are the commercial CFLs upstream?
 Navigant: The operating hours and EUL differ from residential CFLs. We don’t
model program delivery mechanism, just technologies in the model. We
understand a portion of upstream CFLs go to the commercial sector.
 Cynthia Mitchell: The distribution of where the bulbs go from the upstream
lighting program was res/com 90/10, now it's 94/6.
 Navigant: Note: Linear fluorescents dominate commercial lighting, so
commercial CFLs do not impact the results significantly. We will review our data
but are placing more focus on commercial linear fluorescents to make sure that
data is accurate.
o For LEDs, CA is unique in their costs and savings for LEDs because of the CRI the CEC
uses. Will Navigant’s updates be CA-specific?
 Navigant: We have national estimates that will be adjusted to fit CA. In Stage 1,
we are making that adjustment for CRI. In Stage 2, we'd like to get CA-specific
data rather than U.S. data that is adjusted for CA.
o Historically, when the goals are ruled upon they are issued as final. Is there a process to
update the 2016 goals with the updated results from Stage 2 (which may come out in
Q1 2016)?
 CPUC: We can look at the process when Stage 2 is complete.

Craig Tyler (SCG)
o What does Navigant use to set the baselines for measures?
 Navigant: We go to DEER 2015 as our source. DEER should have accounted for
codes and standards updates and CPUC policy regarding savings relative to
code.
o So Navigant assumes 100% compliance implicitly?
4

o
o


Navigant: In regards to UES values for rebated measures: Yes. When we
calculate savings attributable to IOU programs from C&S, we do have a
compliance rate built in.
Are administrative costs included in the model?
 Navigant: Yes, but not at the measure level. Admin costs feed into the TRC
calculation. TRC is a screen, it has little impact on the actual market potential.
The screen is useful in that we don't want to project an ET measure that's not
cost-effective.
 We looked at res and com separately for each IOU: Admin cost per kWh savings.
That figure includes anything that's not an IOU rebate - money that doesn't go
to a customer.
How is Navigant forecasting incremental cost changes over time?
 Navigant: Efficacy and cost studies - national level research - we have the UES
relative to code changing as a result over time. Sources will be cited in the
report.
Mark __ (SoCalREN)
o Is it possible to include a market transformation view on the updated potential study?
 CPUC: Something interesting to consider in Stage 2, haven’t thought about this
yet.
o What about capturing the “true market potential”? Including those measures beyond
MICS.
 Navigant: Our scope is to forecast the IOU potential in Stage 1. We need to align
with the existing policies around the goals and IOU programs. E.g. Code
baseline, ISP, etc.
 CPUC: The market potential assumes a certain level of codes, rebates, etc. We
need to forecast within the limits of what the CPUC agrees to fund.
Others
o Will the updated measures match the DEER 2016 update?
 Navigant: We had a coordination meeting with the Ex Ante team early on in this
process. We focused the discussion on high priority measures to learn what the
Ex Ante team planned to update and how Navigant can reflect those updates in
our model.
o Weren’t all the EM&V reports relevant to this task?
 Navigant: Not all EM&V reports were specific to residential and commercial
measures. We are looking at other EM&V reports, but for different modules in
the model. It is important to note that just because an EM&V study was
available, doesn’t mean we used data from it. But, we did critically review the
results.
5
6