Ex Ante Review Overview
Energy Division Staff and Contractors
Energy Efficiency Industrial/Agricultural Programs
and Portfolio Forecasting
California Public Utilities Commission
1
September 11, 2014
Ex Ante Review Overview
• Speaker:
– Jaclyn Marks
– Supervisor, Energy Efficiency
Industrial/Agricultural Programs and Portfolio
Forecasting
– California Public Utilities Commission, Energy
Division
2
Overview of Presentation and Topics
Purpose: Information Exchange, Q & A encouraged
1. Overview of CPUC oversight of its authorized EE
activities
2. Regulatory background on ex ante review process
(emphasis on Custom projects)
3. Commission staff’s expectations when performing
EAR
3
CPUC Oversight of IOU/CCA/REN
Energy Efficiency Programs
• Speaker:
– Jaclyn Marks
4
CPUC Organization
5
CPUC Organization
• President and four Commissioners
– Governor appoints and Legislature confirms, are independent
“Decision Makers”
• Administrative Law Judges
– Hired as state employees but operate independently as
“Decision Makers”
• Division Management
– Hired as state employees – Supervisors report to Program
Managers who report to Division Director who reports to
Executive Director
• Commission staff
6
– Hired as state employees and report to a supervisor for the area
they oversee
Energy Efficiency Organizational Chart
Energy Efficiency Branch
Manager: Pete Skala
Industrial and Agricultural Programs
And Portfolio Forecasting
Supervisor: Jaclyn Marks
Program Areas
• Industrial and
Agricultural
• Water Energy
Nexus
• Continuous Energy
Improvement
• Efficiency Savings
Performance
Incentive
• Emerging
Technologies
Program
Oversight Functions
• Cost Effectiveness
• Potential and Goals
• Data Base for
Energy Efficient
Resources (DEER)
• Non-DEER Work
Papers
• Custom Projects
• EE Policy Manual
• Strategic Plan
Residential Programs
And Portfolio Approval
Supervisor: Hazlyn Fortune
Program Areas
Oversight Functions
• Home
Upgrade/Advanced
Home Upgrade
• Portfolio Approval
• Regional Energy
Networks
• Programmatic
Stakeholder
Engagement
• Community Choice
Aggregators
• Government
Partnerships
• New Residential
Buildings
• HVAC
• Financing
• Behavior Programs
• ME&O
• Portfolio Analytics
• CARE Program
Enrollment
• Marketing,
Education, and
Outreach
Commercial Programs
And Portfolio Evaluation
Supervisor: Carmen Best
Program Areas
• New Commercial
Buildings
• Lighting
• Codes and
Standards
• Identification /
Tracking of EE
Safety Issues
• Workforce
Education and
Training
• Integrated Demand
Side Management
Program Oversight
• Institutional
Partnerships
• AB 758
7
• Prop 39/K-12
Schools
Oversight Functions
• Annual Report
• EM&V Plan/Budget
• Data management/
Reporting
• Non-Savings
Metrics
• EM&V Protocols and
Methods
• Audit Oversight
• Market Studies
CPUC Regulatory Process
• Rulemaking Proceedings
– The main regulatory process within which the CPUC authorizes
and oversees the EE activities – this is a “legal” process
whereby Decisions and Rulings are issued based on a legal and
policy analysis of documents placed into the record
– Commission Decisions and Resolutions plus Decision Maker
Rulings are the main vehicles for providing direction and making
or clarifying/adjusting policy
– Record is established by Decision Maker rulings and official
proceeding meetings and hearings
8
CPUC Regulatory Process
• Proceedings (continued)
– Commission creates a proceeding via a Decision
• President assigns proceeding to a Commissioner
• Chief ALJ assigns one or more ALJs to the Proceeding
– Party status requested via official request filing which can be
accepted or denied by Ruling or Decision
– Parties to the proceeding can have input invited into the
proceeding record
• The IOUs, CCAs, and RENs submit their proposed
activities via applications
9
CPUC Regulatory Process
• Commission Decisions for EE guidance and authorization
– D.09.09.047 authorized 2010-2012 programs and budgets
including creating the ex ante review process
• ALJ Ruling dated 11/18/2009 in A.07-08-021 created the non-DEER deemed
workpaper review process
– D.11-07-030 (the “ex ante” Decision) in response to IOU PfM
• Clarified many ex ante policy issues such as early retirement/dual baseline
• Created the custom measure and project review process (Attachment B)
– D.12-05-015 gave guidance on 2013-2014 EE applications
• Provided extensive clarifications and amplification of ex ante policy
– D.12-11-015 authorized 2013-2014 programs and budgets
extended the ex ante review process unchanged
– D.11-XX-XXX expected soon will authorize 2015 portfolios
10
CPUC Oversight of EE
• Questions?
11
Regulatory Background on Ex Ante
Review Process (emphasis on
Custom projects)
• Speaker:
– Jeff Hirsch
– James J. Hirsch & Associates
– Prime Contractor – Database for Energy
Efficiency Resources (DEER) and Ex Ante
Regulatory Support – CPUC 12PS5125
12
Ex Ante Contract Team
• Primary players
– JJ Hirsch & Assoc.
• Hirsch, Gandhi, Madison, Reeves, Addison, Hill,
Maddox, Gates, Sweek, and others
– Navigant for stakeholder interaction organization
support
– Energy Metrics (Rothenberg) for custom review
– Itron for custom review
– ERS for custom review
– Other for DEER, workpapers and regulatory support
13
Ex Ante Team Activities
• DEER
– Assumptions, methods and values
– Updated for Codes, Standard and Regulation changes as well as
to incorporation latest evaluation results and related research
• Non-DEER workpaper review and approval
– New measures and updates to existing measures
– Assumptions, methods, baselines, costs, EUL/RUL, NTG
• Custom measure and project review – main topic here
• Other regulatory support
14
Ex Ante Interrelations
Non-DEER
Workpapers
Measures not
in DEER
Custom
Projects
Site-specific;
Values
developed at
completion
Sub-selection
of submission
Ex Ante Values
Best available
information and
extrapolation
DEER
methodology
DEER
Common EE measures;
Savings estimates and
parameters
15
Why Ex Ante Review?
• Ratio of ex post to ex ante values needs improvement.
– Gross has fallen from ~.9 in 2000 to ~.6-.7 in 2008
– Free rider percent has remained and 40%-50% or increased
• Evaluations have indicated there are areas where policy
is not appropriately implemented and that assumptions,
methods and data utilized are not the most appropriate.
• Commission staff has an oversight responsibility to
ensure adopted policy is being followed .
16
Why review before agreement with
customer is signed not later?
• Cost effectiveness involves both the savings estimates
as well as the costs
– Improving savings estimates can result in more effective use of
incentive $
• History of reaction to ex post evaluation
– Review and oversight moved into the process to assist CPUC
and PA’s in finding issues and providing guidance designed to
improve results
– Provide real time oversight to accelerate improvement
17
Objectives of the ex ante review
process
• Improve the accuracy and reliability of the Energy
Efficiency portfolio overall energy savings and cost
effectiveness estimates
• Foster ongoing improvements to the quality and
consistency the portfolio implementers’ own internal due
diligence activities relating to ex ante values
18
Due Diligence
• Due Diligence is an investigation of a business or person
prior to signing a contract, or an act with a certain
standard of care, the process through which a potential
investor can evaluate a major planned investment for its
cost, benefits and risk.
• We are all involved in the due diligence effort.
19
CPUC DECISION 11-07-030
• “Ex Ante” Decision adopted by CPUC in July 2011 and
re-affirmed by two subsequent Decisions in 2012 for use
in 2013 and beyond
• Appendix B describes the custom project EAR process
• Sets minimum project documentation requirements
• Applies to Commission Staff (CS) selected and non
selected projects
• Allows review prior to customer agreement so as to
approve all values (savings, incentives, life, costs, etc.)
used for EE savings “claims” and cost-effectiveness
calculations
• Allows reviews of non-selected projects later (correct
20
errors and set prospective requirements)
CPUC DECISION 12-05-015
• Amplified and added clarifications and details to direction
and policy in D.11-07-030
• Clarification of project classifications (NC/NR/ER/etc.)
• Details of baseline definitions and selection including
requirements for use of early retirement classification
• Details use of EAR “free rider” reviews to provide
guidance and set rules to improve “net” performance
• Details of TRC cost calculation and how that values can
limit incentives
• Clarification on expectation for above code/ISP activities
and no “like” or “regressive” baselines
21
Process of EAR for custom projects
• CMPA-Custom Measure and Project Archive
• Summary list of custom projects uploaded to CMPA, biweekly basis (can be more often).
• Commission Staff reviews list, selects projects- selected
projects may be pre-application stage through claims
stage.
• Utility uploads project documents to the CMPA, email
notification sent to various parties.
• Commission Staff dispositions are posted on the CMPA,
email notification sent to various parties
22
Ex Ante Review Reality
• Commission Staff review only a small percentage of
custom projects.
• PAs generally appear to put more effort into projects
Commission Staff pick for review than non-selected
projects.
• Because PAs appear to put additional emphasis on
“picked projects”, Commission Staff reviewed projects
may not be representative of the full project population.
23
Future direction of the custom EAR
process
• Improving the accuracy and reliability of the Energy
Efficiency portfolio overall energy savings and cost
effectiveness rests with the PAs reviewers.
• Commission Staff will be increasing efforts to work with
PA reviewers and monitor the progress of changes in the
PA review/due diligence process.
• Reviewed projects may be pre-application stage through
claims stage.
• Commission Staff will focus more reviewers activities to
assess the area where due diligence is inadequate.
24
Questions?
25
Commission staff’s expectations
when performing EAR
• Speaker:
– Keith Rothenberg
– Energy Metrics
– Subcontractor to JJH&A for custom EAR
26
Overview
•
•
•
•
Role of the Commission staff representative reviewer
Documentation Expectations
What we expect and what we receive
Different project phases and purposes of Commission
staff review
• Questions
27
Role of the Commission staff
reviewer
• Not to supplement the Utility reviewer.
• Reviewing the Utility reviewers’ due diligence efforts.
• We rely on the Utility reviewers to accomplish the
Commission’s goals:
– Improving the reliability of the savings estimates
– Ensure project compliance with CPUC policy
28
What complicates documentation
requirements?
• Q. For retrofit project types, why don’t we simply
calculate the energy savings directly associated with
replacing the existing system/equipment and use that
value for the savings impacts?
• A. In some cases we do, however CPUC Policy
attempts to balance the ratepayer’s interest and the
customer’s interest with the realities of the
circumstances driving the project.
• Result: Policy has created considerations for free
ridership, rules associated with code compliance, ISP
assessments, TRC considerations, fuel switching
considerations, dual baseline rules, EUL, RUL, etc.
29
Project Types, Baselines, Life
Project
Type
Measure (RUL)/First Period
Life
Energy Savings
Basis
Baseline
EUL
Code or Industry
NC (NEW)
Standard Baseline
EUL
Code or Industry
ROB
Standard Baseline1
EUL
Code or Industry
NR
Standard Baseline1
RUL/
Customer Existing
Baseline
EUL-RUL
ER (RET)
30
REA
RUL or
EUL
Customer Existing
Baseline
(EUL – RUL)/Second
Period Energy
Savings Baseline
N/A
N/A
N/A
Code or Industry
Standard Baseline1
N/A
Early Retirement (AKA “ProgramInduced Early Retirement”)
• Requirements described in guidance document “Early
Retirement Using Preponderance of Evidence”
• Preponderance of evidence supporting that the program
induced the replacement.
• Approved dual baseline energy savings calculation
approach.
• Full measure cost, incremental measure cost for the
second baseline.
• Proposed RUL of the existing equipment supported by
evidence.
• Measure EUL with justification/source.
31
Examples of the expected
documentation
• Project type clearly described, not inferred.
• Documentation to support baseline assignment
• Facility description
– Is there co-generation on site?
• Project description, including relevant associated
systems.
• Existing system controls and operating status.
• Existing system output capacities – current output and
maximum/design capacity.
• Proposed modifications with schematic as applicable.
Preliminary savings calculations and supporting data
with documentation to ensure replicability.
32
Expected documentation (Cont’d)Calculation Method
• Written calculation methodology.
• Precise step-by-step equations.
• Associate the proposed methodology with specific
equipment and systems.
• Energy savings principle for each measure should be
discussed.
• Comprehensive and complete leaving only the final
verified variables and data to be determined after project
completion.
• Generic methodology is not acceptable.
• Documents that require a reviewer to hunt through
spreadsheets cell by cell to investigate the calculation
methodology are unacceptable.
33
Expected documentation (Cont’d)M&V Plans
• M&V plans, reports when required.
• M&V required when impacts are difficult to estimate by
calculation alone.
• System diagrams with measurement points.
• Concise descriptions including measurement period,
measurement interval, measurement equipment.
• Measurement equipment accuracy, result uncertainty.
• May be necessary to collect additional baseline data
before the project commences.
• Raw data archives.
34
Expected documentation (Cont’d)
• Manufacturer’s cut sheets when used to estimate ex
ante savings or when needed to ensure replicability.
• Fuel switching considerations and any required analysis.
• HVAC or process interactive effects values and methods
used to develop ex ante savings values.
• Interactions between multiple measures that act to
increase or decrease savings-parametric analysis.
• Dual baseline analysis for ER projects.
• EUL/RUL value, analysis or source.
35
Expected documentation (Cont’d)
• Pre/post production output data when used in savings
calculations and the source of such records.
• Pre-installation inspection report.
• Post-installation inspection report.
• Other Utility review documents.
• Installation report where M&V performed.
• Utility or implementer program manual (a single archive
of these documents should be referenced).
36
Pre-Installation Review
• Perform a “parallel” review with the Utility
– What is parallel review supposed to be?
• Provide input to the Utility on the estimated custom
measure or project ex ante savings
• Set precedents for future projects
37
Post-Installation Review
• Continued opportunity for CS to review and provide input
on the accuracy of ex ante values.
• Review Utility compliance with earlier project specific
guidance.
• “True-up” estimates with site specific M&V data/results.
• Set precedents for future projects.
38
Utility Claim Review
• Conduct a review of energy savings and costeffectiveness parameters for custom projects
included into the Utility Quarterly Claim.
• Review project files to ensure the CPUC directed
archiving of documentation is being followed.
• Appropriate default realization rates were applied to
ex ante gross savings estimates for projects that
were not reviewed.
39
Utility Claim Review (Cont’d)
• Recommendations made by Commission staff for
reviewed projects were accurately reflected in the claim.
• Guidance provided in CS reviewed projects being
followed in projects that were not previously reviewed by
CS.
40
Early Opinions
• Utilities may submit to Commission staff a request for an
early Commission staff review or opinion on a specific
issue.
– Appropriate for policy grey areas.
– Eligibility issues.
– Other areas where there can be uncertainty regarding
policy or Program compliance issues.
• This should be used when either a utility or an
implementer has concerns or is uncertain how a
particular project might be handled by CS review
41
Summary of CPUC Reviewers’ Role
• Not to supplement the administrator reviewer but to
ensure proper administrator due diligence is being
followed.
• Ensure program rules are being followed.
• Ensure CPUC policy is being followed.
• Verify that utility staff and contract reviewers perform
reviews in an appropriate manner balancing ratepayer
and customer interests.
• Ensure that all required documentation is in the project
file whether it is selected or not for ex ante review by
CPUC staff reviewers.
42
Wrap Up
• Our work plays an important role in the implementation
of the CPUC authorized ratepayer funded programs.
• As reviewers, we are all engaged in a due diligence
effort on behalf of the rate payers as well as the Utility
customers.
• The CPUC has authorized ex ante review to improve
the accuracy and reliability of the Energy Efficiency
portfolio overall energy savings and cost effectiveness
estimates.
43
Wrap Up
• Commission staff are relying on IOU reviewers to
achieve this goal. We are interested in working with you.
• There are changes to the EAR process that will allow CS
to assess the progress in this area: categorization of
comments, tracking of comments with project phases,
project types, reviewers, review firms, etc.
44
Questions??
45
Your comments on the EAR
process?
• How can CS improve the process?
• What works and does not work?
• What issues do Utility reviewers have?
46
Standardization of required
documentation?
• Would a minimum required documentation template that
is required to be included for every project be helpful?
• What is included in the application review packet that
comes to your desk?
47
What additional information would
be useful for Utility reviewers?
48
Documents Available Online
1. Industry Standard Practice Guidance Document
2. Project basis as Early Retirement/Replace-on-burnout
/Normal Replacement/New Construction/Add-on Retrofit
and Remaining/Effective Useful Life, and Preponderance
of evidence Guidance Document
http://www.cpuc.ca.gov/PUC/energy/Energy+Efficiency/Ex
+Ante+Review+Custom+Process+Guidance+Documents.
htm
3. Commission Decisions and Rulings establishing policy
and processes.
49