Independent Audit of Vermont Energy Efficiency Utility
Energy and Capacity Savings for 2005 Through 2007.
Martin Cummings
January 5, 2009
1
Table of Contents
Executive Summary
Pg. 3
I.
Introduction
Pg. 5
II.
Methods Used to Calculate Energy and Capacity Saving
Pg. 6
III. EVT Data Collection and Analysis Procedures
Pg. 10
IV. Verification of EVT Annual Report
Pg. 14
V. Sources of Engineering Values and Adjustment Factors
In the Technical Reference Manual.
Pg. 15
VI. Discussion of Individual Program Year Verification
Issues and Final Savings Estimates.
Pg. 17
VII. Findings
Pg. 20
Appendix – Qualifications of the Auditor
2
Executive Summary
This audit of Efficiency Vermont (EVT) was undertaken to comply with 30 V.S.A. §209(e)(12),
which requires that an independent audit of the energy and capacity savings claims and costeffectiveness of the Energy Efficiency Utility be completed by January 1, 2003 and every three years
thereafter.
The audit focused on a review of the methods used by EVT to collect data and estimate program
energy savings and the methods used by the Department of Public Service to verify EVT’s claimed
savings. Because the 2008 program year was still underway at the time of the audit, the audit is
limited to a review of program year 2005, 2006 and 2007 data.
THE FINDINGS OF THE AUDIT ARE:

The EVT estimates of Annual Energy and Capacity Savings, as verified and adjusted
by the Department of Public Service, are reliable and unbiased estimates of program
savings.

During the period 2005 through 2007, the Efficiency Vermont program was directly
responsible for the installation of energy efficiency measures that save 202,739
megawatt hours of electricity each year. Over the 10.7 year average lifetime of these
measures, they will save 2,171,739 megawatt hours of electricity.

Efficiency Vermont’s programs are highly cost-effective. During the period 2005
through 2007, the EVT programs were cost-effective from the perspective of the
Utility Cost Test, the Total Resource Cost-Benefit Test and the Vermont Societal
Cost-Benefit Test.

The Utility Cost Test is an analytic tool that is used to compare the cost to the utility
of supplying electricity with the cost of providing energy efficiency services. Based
on a utility cost test analysis, EVT delivered energy efficiency to Vermonters at a cost
of 3.055 cents per kWh. In comparison, the marginal price of electricity purchased by
Vermont’s utilities averaged 10.23 kWh over the period 2005-2007. Since EVT’s per
kWh cost is less than the Vermont electric utilities’ per kWh cost of generation,
energy efficiency is a cost effective way to meet system needs.

The Total Resource Cost Benefit Test compares all the costs of conserving a kWh of
electricity, including costs to program participants and third parties, to the costs of
supplying a kWh of electricity. This test shows that Vermonters receive $1.57 in total
resource benefits for every dollar spent on the program. The program returns $1.61 for
each dollar spent in the Business sector and $1.54 for each dollar spent in the
Residential sector.

The Vermont Societal Cost-Benefit Test adds the value of environmental benefits to
the Total Resource Test. Using this Cost-Benefit Test, Vermont receives $1.77 in
benefits for every dollar spent on the EVT program. The program returns $1.80 for
3
each dollar spent in the Business sector and $1.73 for each dollar spent in the
Residential sector.

The cost-benefit test results are very robust. Based on the high cost-benefit ratios
shown in this analysis, the EVT program would remain cost effective, from the
perspective of the Societal Test even if actual savings were only 57% of estimated
savings.

VT data collection and analysis procedures are appropriate and include effective
quality assurance checks. The Department of Public Service has developed effective
procedures for verifying EVT savings estimates.
Table 1 summarizes EVT electric energy and capacity savings for 2005, 2006 and 2007. The auditor,
based upon his analysis of program cost-effectiveness, has verified these results.
Table 1: Cumulative 2005-2007
EVT Program Results
Business
Energy Services
Residential
Energy Services
Total
Programs
87,486
1,243,184
115,252
928,555
202,739
2,171,739
12,452
14,030
18,868
16,379
31,319
30,408
$73,254,274
$69,272,084
$142,526,361
$0.0299
$0.0316
$0.0306
1.61
1.54
1.57
1.73
1.77
Electric Energy Savings
Annualized MWh
Lifetime MWh
Capacity Savings
Winter Peak KW
Summer Peak KW
Total Resource Benefits
Utility Cost Test
Cost per kWh Saved
Total Resource Cost Test
Benefit to Cost Ratio

Societal Cost Test
Benefit to Cost Ratio
1.80
Annualized megawatt hour (MWh) savings are those savings that occur in the first year the energy
efficiency measures are installed. Lifetime MWh savings are the energy savings expected to accrue
over the lifetime of the measures.
4
Capacity is analogous to horsepower. It is the maximum amount of electric power (kW) that can be
supplied to Vermont customers at any given instant. If customer demand exceeds system capacity,
brownouts or blackouts occur. Capacity savings are reductions in customer demand that improve
system reliability and reduce the need for new power plants, purchase power contracts or
transmission lines. Capacity savings are particularly important during times of peak demand and in
geographic areas that do not have adequate transmission links to the rest of the grid.
The cost per kWh of energy efficiency can be directly compared to an electric utility’s cost of
generation. The Utility Cost Test is a commonly used benefit/cost analysis tool, which estimates the
annualized cost to the sponsoring utility (in this case EVT) per kilowatt-hour (kWh) of electricity
saved by its programs and compares it to the cost of generating electricity. Since EVT is directly
funded with ratepayer money, this test reflects the benefits received and costs incurred by Vermont
ratepayers.
The Total Resource Cost-Benefit Test is a broader measure of program cost-effectiveness. It seeks to
summarize and compare all the economic costs and benefits of the program, regardless of who pays
the costs or receives the benefits. Costs included in the test are: EVT’s program costs (including any
rebates or incentives paid to participants), performance incentives paid to EVT, participant costs, and
third party costs (including Department of Public Service program evaluation costs). Program
benefits included in the test are: avoided electric generation costs, avoided electric capacity costs, and
other incidental resource savings (such as reduced fossil fuel consumption and reduced water usage).
The Vermont Societal Cost Benefit Test includes all the factors in the Total Resource Test plus a
$/kWh value for the environmental benefits of energy conservation and a risk reduction adjustment.
The benefit to cost ratio is calculated as the ratio of total program benefits to total program costs
under the Societal Cost-Benefit Test. A benefit-cost ratio of more than one indicates that a program
is cost-effective.
I.
Introduction
Sections 209(d) and (e) of Title 30, as amended in 1999, give the Public Service Board (Board
or PSB) the authority to establish an Energy Efficiency Utility (EEU). The EEU is intended
to improve the delivery of energy efficiency services by replacing the variety of programs
offered by the State’s municipal, cooperative and investor owned utilities with a single
uniform statewide energy efficiency program. In 1999 the Board awarded a three year
contract to the Vermont Energy Investment Corporation (VEIC) to implement the EEU
program under the trade name of Efficiency Vermont (EVT). This contract was renewed in
2002 and rebid in 2005. In both years VEIC has won renewal of the contract.
Since the inception of the program, EVT has offered information and financial incentives to
encourage both residential and commercial customers to improve the energy efficiency of
homes and places of business. The methods used by EVT have ranged from providing rebates
or other incentives to encourage the use of compact florescent bulbs, to complex analyses of
the ways that manufacturing processes can be made more energy efficient.
5
Each spring Efficiency Vermont is required to submit a report, the Annual Savings Claim, to
the Department of Public Service (DPS, the Department) describing its program
accomplishments for the prior year. The DPS, with the assistance of outside consultants,
verifies the data and procedures used by EVT to calculate the previous year’s savings
estimates, and usually recommend modification of savings estimates.
Under the administrative structure established by the PSB, an independent Contract
Administrator (CA) reviews the DPS’s recommendations and EVT’s responses to those
recommendations, and makes his recommendation to the Board regarding the savings
estimates. Then the Board makes a final decision and adopts the EVT savings estimates. In
late summer or early fall, EVT submits an Annual Report that incorporates the revised savings
estimates for the prior year.
To ensure accountability, the Legislature, in 30 V.S.A. § 209(e)(12), required an independent
audit of the energy and capacity savings and cost effectiveness of EEU energy efficiency
programs by January 1, 2003, and every three years thereafter. This is the third of the
legislatively required independent triennial evaluations of the cost effectiveness of the EEU.
It was undertaken by Martin Cummings during the last quarter of 2008 under contract to the
Vermont Public Service Board. Because EVT’s final Annual Report on 2008 will not be
completed until the second half of 2009, this audit covers 2005, 2006 and 2007.
Mr. Cummings has over thirty years of regulatory experience with the New York State
Department of Public Service (NYDPS). He is the former Project Director for Program
Evaluation for NYDPS Demand Side Management programs and has extensive experience
overseeing field audits and program evaluations of electric utility energy efficiency programs.
See Appendix A for additional information regarding Mr. Cummings’ qualifications.
The audit examines the procedures used by EVT to estimate program savings, and by the
Department to verify those savings estimates. The objective of the audit is to determine if
these procedures, as designed and implemented, yield reliable and accurate estimates of
program savings and cost benefit test results. This audit will largely focus on the specific end
use measures that resulted in the most savings claimed by EVT. While EVT offers financial
incentives for a variety of energy efficient technologies, energy efficient lighting measures
(both residential and business sector) accounted for over 55% of 2005 through 2007
electricity savings. Improvements in the efficiency of industrial processes and energy
efficient motors accounted for almost 15% of savings.
II.
Methods Used to Calculate Energy and Capacity Savings
It is helpful to think of Efficiency Vermont as providing two different levels of service to its
customers. This requires two different levels of sophistication in the estimation of energy and
capacity savings.

At the simpler level, EVT offers residential and business customers rebates for standard
off-the-shelf energy efficiency measures – compact florescent light bulbs or energy
6
efficient refrigerators for example. EVT refers to these as “prescriptive measures”.
Savings for these measures are estimated using modified engineering estimates.

EVT also offers project-specific assistance and incentives for improving the energy
efficiency of unique industrial processes, and for complex commercial and industrial new
construction projects. EVT refers to these as “custom measures”. Estimating the energy
and capacity savings from these projects requires complex project-specific analysis.
A. Modified Engineering Estimates
EVT’s energy and capacity savings estimates for standard energy efficiency measures are
prepared on a measure-by-measure, program-by-program, basis using a method known as the
modified engineering estimate. This is a standard method used by energy efficiency programs
across the nation. Savings estimates are ordinarily calculated in two forms: gross savings and
net savings.

Gross savings can be thought of as the energy and capacity savings resulting from
installing an energy efficiency measure that an individual customer can expect to see at
the meter.

Net savings are the adjusted energy and capacity savings, directly resulting from customer
participation in an energy efficiency program, which a power company can expect to see
at the generator.
While gross savings are important to individual customers, net savings are critical for electric
system planning and evaluating program cost-effectiveness. In addition, net savings benefit
all of a utility’s customers since they allow a utility to purchase less power, the cost of which
would have been paid for by all the utility’s ratepayers. The savings reported by EVT and
verified by this audit are net savings.
Sections 1 and 2 below briefly review the concepts behind the calculation of gross and net
savings from individual measures. Energy efficient compact florescent bulbs (CFLs) are used
as an example.
1. Gross Savings Calculations
a) Demand Reduction/Capacity Savings (kilowatt or kW) – The calculation of gross
savings begins by comparing the electric demand (wattage) of the energy efficient device
with the wattage of the less efficient device it will replace. For example, a 29-watt
compact florescent bulb produces as much light as a 100-watt incandescent bulb, so
replacing the inefficient bulb reduces the customer’s electric demand by 71 watts
whenever the lighting fixture is on.
From the point of view of an electric system planner, it matters a great deal when the
CFLs are on since demand reductions during periods of peak usage can save system
capacity, reduce costs and delay the need for constructing new power plants or
transmission lines or negotiating new power contracts. Load shape data is used to
estimate how much of the demand reduction from the CFLs will fall during on-peak and
off-peak time periods. Since the lights in a house are typically turned on and off at
7
different times than the lights in a supermarket or office or school, different load shape
estimates are used for different building types.
b) Energy Savings (kilowatt hour or kWh) - Annual electric energy savings from a CFL
are calculated by multiplying demand savings by an estimate of the number of hours an
average bulb is turned on during the course of a year. Estimates of a measure’s “hours of
use” are based upon independent market studies performed by the DPS and other regional
and national sources. Like load shapes, estimates of a CFL’s annual hours of use are
different for different building types. Thus, a CFL which reduces demand by 71 watts
would save about 244 kilowatt hours a year if installed in an office building where the
lights are left on for an estimated 3,435 hours a year, but would produce different annual
savings in buildings used for other purposes.
Lifetime electric energy savings are calculated by multiplying demand savings by the
number of hours the CFLs are expected to operate before they burn out – a much longer
period than the expected lifetime of an incandescent bulb. Since CFLs that are turned on
and off frequently tend to fail somewhat earlier than those that operate more continually,
lifetime energy savings also vary by building type.
Compact florescent lights also have indirect effects on energy savings. Like all energy
efficient measures, CFLs release less waste heat than the inefficient devices they replace.
A large building lit solely by florescent lights will require significantly more energy to
heat in the winter and less energy to cool in the summer than an identical building lit by
incandescent bulbs.
2. Net Savings Adjustments
The calculation of net savings incorporates all the factors included in the calculation of
gross energy and capacity savings and adds four additional adjustments.
a) Free Rider Adjustment Factor – It is likely that at least a few of the people who
participate in energy efficiency programs would have installed efficiency measures even if
the programs did not exist. Using our CFL example, assume a homeowner:
 Learns about the advantages of CFLs in a magazine or from a cable TV show,
 Makes up his or her mind to try out CFLs based on that information,
 Walks into a hardware store intending to buy several CFLs,
 Discovers at the checkout counter that he or she can get a rebate from EVT, and
 Applies for and receives a rebate.
That customer would be considered both a program participant (because he received a
rebate) and a free rider (because the rebate had no influence on his purchase decision.)
Gross energy and capacity savings estimates are adjusted downward by a free ridership
factor that is determined individually for each measure in each program to account for
such circumstances.
b) Spillover Adjustment Factor– The spillover factor is the converse of the free
ridership factor. It accounts for the likelihood that an energy efficiency program is
responsible for some purchases of energy efficiency measures that never show up in the
8
program’s tracking system. EVT’s CFL instant rebate program limits rebates to the first
six CFLs purchased by a customer each year. Assume a customer:
 Is motivated by EVT program literature and rebates to buy CFLs,
 Decides to install CFLs in eight fixtures in his or her house,
 Purchases eight CFLs at the hardware store, and
 Receives rebates on six of the eight CFLs he or she purchases.
The purchase of the additional two CFLs is an effect of program spillover. The purchases,
and the consequent energy and capacity savings, are a direct result of the program, but the
unrebated CFLs never show up on the tracking system. Another example of spillover is a
customer who intends to participate in a mail-in rebate program, makes the purchase, and
then loses the sales receipt or rebate application before he or she can send it in.
Gross energy and capacity savings estimates are adjusted upward by a spillover factor that
is determined individually for each measure in each program.
c) Persistence Adjustment Factor – The persistence factor adjusts for the fact that in
some circumstances, anticipated savings may not persist for the normal life of the
measure. For example, energy efficient lighting measures have been found to have shorter
lifespan in the hostile operating environments of dairy farms. Gross energy and capacity
savings estimates are adjusted downward by a persistence factor that is determined
individually for each measure in each program to account for such circumstances.
d) Line Loss Factor – The application of free rider and spillover adjustment factors to
gross savings estimates produces an estimate of net savings at the customer’s electric
meter. However, because of transmission losses, the actual effect of the energy efficiency
measures on the electric system is significantly greater. According to data supplied by the
DPS, and depending on weather and system load, between 10.68% and 13.70% of the
electricity generated at a Vermont power plant, or purchased from an out-of-state supplier,
is lost before it reaches Vermont electric utility customers. Gross savings are adjusted
upward by a line loss factor to correct for this effect.
3. The Technical Reference Manual
The formulas and values used to calculate gross and net savings using the modified
engineering estimate approach are reviewed by the Technical Advisory Group (TAG).
This group consists of EVT, DPS, the Contract Administrator and the Burlington Electric
Department (BED)1. The formulas and values are recorded in a lengthy and
comprehensive document called the Technical Reference Manual (TRM). Section V of
this report discusses the sources of the engineering values and gross-to-net adjustment
factors in the Technical Reference Manual and the procedures used by EVT and DPS to
update the Manual.
B. Project Specific Engineering Analysis
1
The Burlington Electric Department is a municipal utility which runs its own energy efficiency program.
Although not a part of EVT, BED uses the TRM to calculate the results of its own program.
9
The predetermined engineering formulas and values in the Technical Reference Manual
may not be adequate to estimate savings from complex commercial and industrial retrofit
and new construction projects. These projects may involve:
 Unusual technologies (energy efficient snow making guns, for example);
 Project-specific applications of relatively common technologies (retrofitting high
Efficiency motors with adjustable speed drives to an existing industrial process); or
 The integration of energy efficient technologies and design principals in new buildings.
In these cases, energy and capacity savings estimates are based on project specific
engineering analyses. These analyses are similar in principle to the modified engineering
estimates used to calculate savings from rebated measures, but are more complex and
utilize project-specific information provided by customers or collected on-site. This
information may include electrical metering of specific processes before and after efficient
equipment is installed.
III.
EVT Data Collection and Analysis Procedures
The following section discusses how EVT collects and analyzes energy savings data.
EVT has adopted a formal Quality Assurance (QA) program with specific performance
targets and milestones. Many aspects of the QA plan are directed towards assuring the
integrity of the data and analyses that feed into estimates of energy and capacity savings.
Applicable aspects of the QA program are addressed in this section.
A. Promoting Energy Efficiency in the Residential Sector
Efficiency Vermont has several programs targeted towards promoting energy savings in the
residential sector. For example, it offers rebates for the purchase of energy efficient washers
and dryers and provides assistance and incentives to design more energy efficient new homes
or retrofit existing homes. However, the most important source of energy savings in the
residential sector is the replacement of incandescent lighting with more efficient compact
florescent (CFL), halogen and LED lights. In 2007 89% of all the energy savings in the
residential sector came from efficient lighting products.
Efficiency Vermont has undertaken several initiatives to promote efficient lighting. These
include training for architects and builders designing and constructing new homes, incentives
to get stores to stock CFL’s and other efficient lighting measures and community wide energy
conservation programs. The most important of the techniques used by EVT to promote
energy efficient lighting are direct rebates to customers and cooperative agreements with
retailers.
Since the gross and net energy and capacity savings estimates per energy efficiency measure
rebated have been estimated in advance, the only data required to estimate total annual
savings from the rebate program is the number of valid rebates issued. The primary task is
verifying that the rebates have been issued to eligible Vermont customers, and that the limit
on number of rebates per customer has not been exceeded. Rebate forms also allow program
10
evaluators to contact a sample of customers and ask about the number of hours the lights are
on in a typical day.
In recent years Efficiency Vermont has increasingly relied on cooperative agreements with
lighting equipment manufacturers and retailers to promote CFL’s2. Under these agreements
EVT agrees to make payments directly to major stores (supermarkets, hardware stores and
chain retailers like Home Depot) to encourage them to stock efficient lighting measures.
These are then sold at reduced prices that are negotiated in advance. The customer receives
the cost savings of the rebates without having to go through the inconvenience of filling out
and mailing a rebate form. EVT then receives information from retailers about how many of
the subsidized lighting units were sold at each location in a given month.
This method is both more convenient for the customer, and more economical for EVT.
However, it does pose some measurement issues. Without a rebate form filled out with the
customers address it is impossible to determine how many of the subsidized CFL’s are being
purchased in Vermont stores by out-of-state residents. It is also impossible to determine how
many subsidized CFL’s are purchased by each customer. This is significant because
evaluation data suggests that customers who purchase large numbers of CFL’s may stockpile
some against future needs or install them in locations where they are used for only a few
hours a day. Either action would reduce the energy and cost savings from the bulbs in any
given year.
Efficiency Vermont uses survey and evaluation data collected during the years when rebates
were more important to estimate hours of use and where customers shopping at a given store
live. These estimates are reviewed and modified, as appropriate, by the Technical Advisory
Group. The DPS is currently conducting an evaluation study to determine the number of
CFL’s sold in Vermont through the EVT program and the total number sold outside of the
program. This information should resolve some of the measurement issues discussed above.
B. Helping Commercial and Industrial Customers Improve the Energy Efficiency of
Existing Buildings and Processes
Efficiency Vermont attempts to identify commercial and industrial customers at critical
moments (for example: when they must replace failing or obsolete equipment, or when they
plan to remodel or add additional equipment) and intervenes to ensure that they install the
most cost-effective energy efficient equipment available for that application.
The service is primarily marketed through trade allies - the contractors, engineers and
equipment vendors likely to be contacted by a commercial or industrial customer when they
need to replace or add to equipment. Efficiency Vermont has established working
relationships with most of these trade allies and has provided training or information on
efficient products. Other leads may come from “cold calls” by EVT staff, referrals from
Several years ago about 80% of Vermonters who purchased CFL’s through the EVT program filled out and
submitted rebate forms. Currently only about 10% submit rebates. The remaining CFL’s are promoted through
cooperative agreements.
2
11
utilities, advertising in trade publications, or direct contacts by customers who have
participated in the program in the past.
In recent years EVT has begun assigning “Efficiency Vermont Account Managers” to develop
ongoing relationships with the 65 Vermont business customers which use over 1MW of
electricity at a single location. The goal of this program is to develop collaborative
relationships that allow the account manager to participate in corporate-level discussions of
energy related issues.
After a lead is received, it is screened by EVT to determine if the customer’s project is a
likely candidate for the installation of cost-effective energy efficiency measures. A qualifying
lead is assigned to a project manager. In some cases, a trade ally may have already identified
opportunities to improve the energy efficiency of the customer’s project; in other cases, EVT
technical staff suggests ways to make the project more energy efficient.
Some energy efficiency projects are limited to the straightforward installation of measures,
such as florescent lighting fixtures with high efficiency electronic ballasts. These are covered
by the predetermined engineering formulas and adjustment factors in the Technical Reference
Manual. In EVT’s jargon, these are “prescriptive” projects.
“Custom” projects, those that involve more complex or unusual measures or applications,
require project specific engineering analyses. The initial purpose of these engineering
analyses is to determine if energy efficiency improvements are cost-effective. It is frequently
the case that energy efficiency improvements that could save money over the long run are not
installed by business customers because they substantially increase initial costs. To overcome
this barrier, EVT may offer the customer a project-specific financial incentive based on the
engineering estimate of project savings.
All energy and capacity savings estimates are based on a comparison of the energy efficient
version of the project to a base case representing the project, as it would have been
implemented in the absence of the EVT program. When customers have already prepared
initial project plans before they are contacted by EVT, the base case is straightforward. In
other cases, equipment vendors are able to provide information on the types of equipment
ordinarily included in a “standard bid.” Where these sources of information are not available,
the base case is usually assumed to be the minimum code standard.
In most cases, the EVT project manager, or technical staff under his/her direction, prepares
the engineering estimate of savings. In other cases, the trade ally who referred the customer
staff prepares initial savings estimates that are subsequently reviewed and approved by EVT
technical staff. EVT uses industry standard engineering analysis software for thermal
modeling, heating and air conditioning sizing, lighting design and some other aspects of the
engineering estimate of project savings.
The project managers and technical staff appear to have adequate professional training and
experience to perform these analyses. Many of the project managers are electrical or
mechanical engineers who are members of professional societies. Many are certified as
Energy Managers by the Association of Energy Engineers. All technical staff working on
12
commercial and industrial projects have been trained in the Vermont Commercial
Construction Energy Guidelines and engineering analysis software.
EVT uses data input forms for the engineering analysis software. These forms reduce data
entry errors by refusing to accept inputs that fall outside of the feasible range for each item.
The engineering analysis software is integrated with EVT’s tracking system to permit
automatic transfer of information to the database that generates annual savings estimates.
This integration reduces the opportunity for data entry errors.
EVT inspects nearly all “custom” projects and all “prescriptive” projects that receive
incentives greater than $2500. Once EVT has verified that the project has been completed
and meets agreed upon standards, the energy and capacity savings estimates developed by the
project-specific engineering analysis are included in that year’s estimate of program savings.
The Quality Assurance plan calls for a sample of the initial project screening analyses and
project-specific engineering analyses to be cross-checked and verified by a “third party” - a
qualified EVT staff member who has not been directly involved in that project. A sample of
completed projects is also field-inspected a second time by a “third party” EVT staff member.
C. Helping Business Customers Construct More Efficient New Buildings
Efficiency Vermont attempts to intervene between the decision to construct a new commercial
and industrial building, and the final project design by providing technical advice and
engineering analysis for thermal envelope measures (insulation, efficient windows, etc.) and
building design features (day lighting and passive solar heating, etc.). Depending on the size
of the project, its potential for energy savings and the stage the project has reached in the
design process, EVT may pay for the additional architectural or engineering design work
needed to improve building efficiency and/or, provide incentives for the installation of
specific energy efficiency measures.
Over the years that this program has been running, EVT has developed close links with
Vermont “trade allies” - the architects, engineers, equipment suppliers and contractors
involved in designing and constructing commercial buildings. Most of the information that
EVT receives about new projects comes through these links. EVT also develops leads by
monitoring the Dodge Report (a trade publication that reports on planned construction
projects), other general, and trade publications, and Act 250 records for project
announcements. In general the base case for calculating energy savings is derived from
architectural plans developed before EVT becomes involved in the project, from building
codes, or by determining the “standard industry practice” for constructing buildings of a
particular type.
Projects subject to Act 250 are treated somewhat differently. Act 250 requires that projects be
built to higher standards of energy efficiency than non-Act 250 projects, but does not define
those standards in great detail. Based on its experience administering Act 250 energy
efficiency standards, the Department of Public Service has developed a baseline construction
standard that represents the level of energy efficiency improvements (above industry
standards) commonly required of projects approved under Act 250. EVT is allowed to claim
13
energy and capacity savings on only those measures or design features that exceed this
baseline.
D. Preparation of Initial Annual Savings Claims
The EVT program year ends on December 31. Early in the following year, EVT: (1)
completes the entry of data from rebated measures installed during the prior year, and the
calculation of savings and data entry from custom measures and new construction projects
completed during the prior year, (2) calculates aggregate program savings, and (3) prepares a
savings claim for submission to the Department of Public Service, the PSB, the CA and
Vermont Electric Utilities.
IV.
Verification of EVT Annual Report
A. Department of Public Service Verification Activities
The Department’s technical review and verification activities are conducted by experienced
professional staff supplemented by West Hill Energy and Computing Inc. (West Hill) an
outside engineering contractor. The DPS review takes several months.
1. Review of EVT Database. West Hill reviews the EVT database files that generate
savings estimates for rebated measures to insure that the formulas, engineering values and
gross-to-net adjustment factors were consistent with the Technical Review Manual.
2. Identification of Savings Estimate Issues Affecting Residential Projects - Based on the
database review West Hill identifies measurement issues which could affect EVT’s
estimates of savings from residential sector programs.
3. Identification of a Sample of Business Projects for Review – West Hill also identifies
two groups of specific Commercial and Industrial projects for detailed review.
a. It selects for detailed review all of the largest business projects completed during
the years. Based on its analysis it may recommend modifications of savings
estimates for specific larger projects.
b. West Hill also selects a statistically valid sample of the smaller business projects.
It reviews VEIC’s savings estimates for this sample group and may recommend
modification to the savings estimates for projects in the sample. The ratio of
VEIC’s estimated savings to West Hill’s revised savings is used to calculate a
“realization rate”. It is assumed that this rate reflects systematic errors in VEIC’s
savings estimates. The “realization rate” is applied to the entire population of
smaller business projects completed during that year.
4. Issue Resolution – In spring, the DPS provides EVT with an extensive list of issues to be
resolved. Many of the issues concern the calculation of EVT savings for specific large
business sector projects, or the calculation of savings for complicated or unusual energy
efficiency measures in the commercial and industrial sector; others concerned savings
from residential programs. In most cases the DPS proposes specific reductions in energy
14
and capacity savings for the projects or measures under review. Over the following month
the Department conducts further analysis and meets with EVT staff and the Contract
Administrator in an attempt to resolve verification issues. In some cases, EVT agrees that
the Department’s concerns are valid and voluntarily accepts reductions in claimed energy
and capacity savings. In other cases EVT is able to satisfy DPS concerns.
B. Role of the Contract Administrator
As required by the PSB, the Department reports to the Contract Administrator in spring or
early summer on its review of EVT’s savings estimates for the prior year. The report
describes the steps the DPS took to verify savings, the issues raised during the verification
process, and the Department’s proposed adjustments to EVT’s claimed savings. The
Contract Administrator resolves any remaining disputes between the Department and EVT at
the conclusion of the Department’s verification process, and recommends final savings
numbers to the Board.
C. Efficiency Vermont Annual Report
In late summer or fall, EVT issues a revised annual report for the previous year which
includes the savings adjustments recommended by the Department and reviewed by the
Contract Administrator, and approved by the Public Service Board and estimates of program
cost effectiveness.
D. EVT Incentive
The legislation setting up the Energy Efficiency Utility includes a provision for awarding
incentive payments to EVT if it meets negotiated performance goals over a three year period.
Currently, these goals include targets for:
1. the total MWh, TRB and capacity savings and the savings that occur in geotargeted
areas,
2. the number of small business served,
3. the number of large grocery stores selling CFL’s, and
4. the energy savings achieved in the Towns of Hardwick and Northfield.
The current incentive will be based on EVT’s accomplishments over the period 2006 through
2008. Since 2008 results have not yet been formalized it is not possible to determine the
incentive that EVT will receive for this period.
V. Sources of Engineering Values and Adjustment Factors
In the Technical Reference Manual.
The accuracy of the EVT savings estimates is, in the end, dependent on the validity of the
hundreds of engineering values and adjustment factors contained in the Technical Reference
Manual (TRM). It is therefore important to review the original source of this information and
the procedures used by EVT, the DPS, and the Contract Administrator to update and refine it.
15
A. Initial Source of Technical Reference Manual Data
Most of the engineering values and adjustment factors in the initial (2000) version of the
Technical Reference Manual (TRM), were derived from “The Power to Save: A Plan to
Transform Vermont’s Energy-Efficiency Markets”, a feasibility study for the statewide
energy efficiency utility. This report, issued by the Vermont Department of Public Service in
May of 1997, included analyses of the potential energy and capacity savings from a variety of
energy efficiency programs and technologies.
The data and assumptions underlying these analyses, including engineering values and
adjustment factors, are contained in appendix 3 of “The Power to Save” and associated work
papers. Optimal Energy, the consulting firm that prepared this section of the report,
specializes in energy efficiency programs and has conducted similar studies in several states
in the Northeast. The engineering values and adjustment factors underlying its analysis were
primarily derived from manufacturer specifications and from studies conducted by Vermont
utilities and energy efficiency programs in other states. Annotations and footnotes in the
TRM cite the sources for many, but not all, of the engineering values and adjustment factors.
B. Updating the Technical Reference Manual
The Technical Reference Manual has been frequently updated to add data on measures not
covered in “The Power to Save” and to revise existing data as programs change or new data
becomes available. Efficiency Vermont maintains a master copy of the TRM that records all
changes in formulas, engineering values, and adjustment factors. The reasons for many, but
not all, of these changes are briefly noted in the TRM.
Either EVT or DPS staff may initiate proposed revisions to the TRM. The Technical
Advisory Group (TAG), comprised of staff from both EVT, BED and DPS, reviews all
proposed changes. The Contract Administrator chairs the TAG. If the group is unable to
resolve an issue, EVT modifies the TRM to reflect its current practice. The DPS may propose
modifications of those savings estimates in the annual verification process.
C. Program Evaluations
Many of the engineering values (hours of use, load shapes, etc.) and adjustment factors (free
ridership, spillover, etc.) in the TRM were initially based on impact evaluations undertaken by
utilities or government agencies in other states. This is understandable. Program evaluations
can be very expensive, and the cost of evaluations is not closely linked to the number of
participants in a program. Consequently, extensive evaluation studies may be prohibitively
expensive for small states like Vermont.
While methodologies vary, it is typical for program evaluations to be based on: mail or
telephone surveys of a statistically valid sample of participants and non-participants;
16
statistical analysis of participant and non-participant bills; or, end use metering. Because
climate, program design and market characteristics vary from state to state, extrapolation of
impact evaluation data collected outside of Vermont requires careful analysis and professional
judgment.
The Department of Public Service has undertaken some impact evaluations studies during the
period covered by this review. Early in 2006 RLW Analytics and KEMA Inc. reported on an
evaluation of the EVT Business programs. This study focused on surveys of commercial
customers eligible for EVT programs, customers who actually participated in the programs
and “trade allies” such as architects, contractors and engineers. The evaluation concluded that
EVT was doing a good job of contacting and informing eligible customers and trade allies and
was having a positive impact on their decision to implement energy efficiency measures. In
2005, KEMA conducted an evaluation of residential programs, finding that EVT was doing a
good job with the marketing and delivery of each of the three main residential sector services
offered at that time. This evaluation included a survey intended to produce reliable and
accurate estimates of appliance saturation and improved information on the energy related
characteristics of Vermont’s residential market. The DPS also developed an evaluation plan
for the 2006-2008 contract period which will focus on on-site market assessments of single
family homes, and existing, remodeled and new business buildings.
V.
Discussion of Individual Program Year Verification Issues and Final Savings
Estimates.
This section of the report briefly summarizes the major savings verification issues that arose
each year during the Department of Public Service and Contract Administrator review and
verification of EVT’s claimed savings for the year explains how these issues were resolved
and presents the final, verified, program results for the year.
A.
Program Year 2005
In the executive summary to its 2005 annual report, VEIC characterized 2005 as “a year of
building depth and maturity in service and initiative offerings”. The program offerings
remained essentially the same as in prior years and program budgets grew modestly.
The Department of Public Service recommended that Efficiency Vermont claimed MWh
savings for 2005 should be reduced by 9.9%. VEIC and the contract administrator agreed
with the proposed reductions. All of the reductions in 2005 savings estimates were made in
the business sector and were based on detailed reviews of the 23 largest projects undertaken in
2005 and a statistical sample of 99 smaller commercial and industrial projects. Several of the
largest projects reviewed involved attempts to improve the energy efficiency of snowmaking
operations at major Vermont Ski resorts. Estimating the energy efficiency impacts of
installing efficient snow guns, water pumps and air compressors is particularly complex
because actual impacts depend not only on the performance of the equipment but also on
weather, the ski areas snowmaking strategy, the anticipated number of skiers in a given period
and the number of trails opened.
17
The Department prepared a comprehensive report on the reasons for these modifications. It
is, however, difficult to summarize the reasons for the reductions in savings estimates because
of the diversity and complexity in the types of businesses and projects reviewed.
Efficiency Vermont’s final verified results for 2005 are shown in table 2.
Table 2: Program Year 2005
Business Energy Residential Energy Total
Services
Services
Programs
Electric Energy Savings
Annualized MWh
27,394
28,465
55,859
Lifetime MWh
410,643
230,681
641,324
Capacity Savings
Winter Peak kW
4,179
4,498
8,678
Summer Peak kW
3,972
4,697
8,669
$23,164,903
$13,013,013
$36,177,916
Total Resource Benefits
B. Program Year 2006
Some fundamental changes in EVT programs, objectives and budgets were initiated by the
Public Service Board in late 2006. These changes were a consequence of Public Law 61,
which removed a statutory cap on the EVT budget and established criteria for the Board to
review in determining the appropriate annual budget. In reviewing the new criteria established
by Public Law 61 the Board determined that it was appropriate for EVT to use part of the
increased budget to implement “geographic targeting” in an attempt to defer or negate the
need for transmission and distribution upgrades and the attendant financial costs and
environmental impacts.
In an order dated September 25, 2006, the Board directed EVT to use a portion of its budget
to develop special “geographically targeted” energy efficiency and demand reduction
programs to slow or reverse peak load growth in four defined areas with serious transmission
constraints. The rest of 2006 and much of 2007 were devoted to defining geographic
targeting in more detail and planning and initiating the programs that would accomplish the
Board’s objectives. The “geographically targeted” programs had no impact on 2006 results
and only modest impacts on 2007 results.
In 2006 West Hill, the Department’s consultant selected a stratified sample of 86 out of 831
business and multifamily residential projects completed during that year. After detailed
engineering analysis of claimed savings from 27 different projects, mainly customized
projects installed by large business sector customers, the DPS recommended that the EVT
18
claimed savings be reduced by a total 6.2%. Because of the great diversity in the customers
and projects reviewed by DPS it is not possible to make meaningful generalizations about the
reasons for the recommended adjustments.
Efficiency Vermont’s final verified results for 2006 are shown in Table 3.
Table 3: Program Year 2006 Business Energy
Services
Residential Energy Total
Services
Programs
Electric Energy Savings
Annualized MWh
23,314
29,633
52,947
Lifetime MWh
318,135
268,813
586,948
Winter Peak kW
3,440
4,738
8,178
Summer Peak kW
4,490
4,320
8,809
$18,495,505
$23,435,542
$41,931,042
Capacity Savings
Total Resource Benefits
C. Program Year 2007
EVT’s budget was increased by the PSB by nearly $4,000,000 (about 27%) in 2007. The
Board also adopted new, and substantially increased, values for avoided costs. As a
consequence of these changes, total resource benefits increased by nearly 54%.
Much of EVT’s focus in 2007 was on developing and rolling out geotargeted programs in the
four areas of the state where transmission constraints raised a risk of brownouts during peak
load events. Separate strategies were developed for three classes of customers.
 150 large business accounts in the targeted areas with annual consumption of more
than 500,000 kWh were identified. Each of these accounts was assigned an account
manager to work closely with the business to identify and implement customized
energy efficiency measures.
 EVT selected and trained RISE Engineering, a general contractor, to conduct free
energy analyses and perform direct installation of efficient lighting measures in
medium sized businesses in the targeted areas.
 EVT developed a more aggressive targeted efficient lighting program, which includes
direct mailings of rebate coupons, to reach residences and small businesses in the
targeted areas.
Since the geotargeted programs did not get off the ground until part way through 2007 the
results for the first year were limited. EVT estimates that these programs reduced winter peak
demand by 1,952 kW and summer peak demand by 1,624 kW.
19
As in prior years, Staff of the Department of Public Service conducted a detailed review of the
energy and capacity savings claims submitted by Efficiency Vermont. Staff recommended
that the EVT energy savings claims be reduced by 2.8% and that winter and summer peak
demand savings estimates be reduced by 4.2% and 5.2% respectively.
Efficiency Vermont’s final verified results for 2007 are shown in table 4.
Table 4: Program Year 2007
Business Energy Residential Energy Total
Services
Services
Programs
Electric Energy Savings
Annualized MWh
36,778
57,154
93,933
Lifetime MWh
514,406
429,061
943,467
Winter Peak kW
4,833
9,631
14,463
Summer Peak kW
5,568
7,362
12,930
Capacity Savings
Total Resource Benefits
VI.
$31,593,866
$32,823,534
$64,417,403
Findings
A. The Use of Modified Engineering Estimates is an Appropriate Method for
Calculating Program Energy and Capacity Savings
The use of modified engineering estimates to calculate gross and net program energy and
capacity savings is a standard technique commonly used by energy efficiency programs
nationwide. The basic formulas and adjustment factors (free riders, spillover) used by EVT in
the Technical Reference Manual are consistent with industry practice.
B. Efficiency Vermont’s Data Collection and Analysis Procedures are Appropriate and
Include Adequate Quality Assurance Checks
Based on the auditor’s review of EVT documents and interviews with EVT staff, the EVT
staff that collect and analyze energy efficiency data have appropriate training and experience.
EVT data input forms, engineering analysis programs, and tracking systems are designed and
integrated to minimize data entry errors. EVT has also developed and implemented a quality
assurance program, much of which is directed towards ensuring the accuracy of engineering
estimates of savings.
20
C. Department of Public Service Staff and the Contract Administrator Have Developed
Appropriate Procedures for Verifying EVT Savings Estimates and Have Implemented
These Procedures Diligently.
The DPS has developed detailed and effective procedures for verifying and, if necessary,
adjusting EVT’s savings estimates and has obtained the specialized consulting services
necessary to assist it in the implementation of the verification process. The Contract
Administrator has been effective in managing the verification process and resolving disputes
between EVT and the DPS.
D. Although the Source of the Engineering Values and Adjustment Factors in the TRM
Is Not Completely Documented, the Values That Are Documented Appear to be Reliable
and Unbiased.
The sources of many, but not all, of the hundreds of engineering values and adjustment factors
in the TRM are documented by annotations or footnotes in the TRM. Those that are
documented appear to have been derived from appropriate sources. The contractor that
developed the original values for “The Power to Save” had considerable experience in the
field of energy efficiency and there is no reason to believe that the undocumented values are
inaccurate or come from inappropriate sources. It is likely that EVT or the Department, in
the course of preparing and verifying savings estimates, would have picked up any significant
errors in major measures.
E. The EVT Estimates of Annual Energy and Capacity Savings, as Verified and
Adjusted by the Department of Public Service, are Reliable and Unbiased Estimates of
Program Savings.
Because the calculation of program savings from the EVT energy efficiency program is
necessarily dependent on a large number of engineering values and adjustment factors, some
of which cannot be known with certainty, savings estimates must be recognized as just that –
estimates. The audit has shown that EVT and the Department have put into place appropriate
procedures and controls to minimize sources of error and bias in the estimation process.
Accordingly, the savings estimates in the adjusted EVT annual reports should be considered
reliable and unbiased estimates of actual program savings.
F. During the period 2005 through 2007, the EVT Program was Cost-Effective From the
Perspective of the Utility Cost Test.
Using these savings estimates and EVT program expenditure data, the auditor calculated
Utility Cost Test results for the 2005 – 2007 EVT programs. The Utility Cost Test is a
commonly used benefit/cost analysis tool, which estimates the annualized cost to the
sponsoring utility per kWh of electricity saved by its programs. The inputs to the test are the
annualized MWh savings from EVT programs, EVT’s actual program expenditures, and the
average lifetime of the energy efficiency measures installed. Because the program is funded
by ratepayers through a System Benefit Charge, the utility test quantifies state ratepayer
revenue requirements rather than utility investment.
21
Table 5 shows Utility Cost Test results for EVT’s program as a whole, and separately for the
Commercial and Industrial and Residential market segments.
Table 5: Utility Cost Test Results
Annualized Cost per kWh Saved
Business Programs
$0.0299
Residential
Programs
$0.0316
Total Program
$0.306
G. During the period 2005 through 2007, the EVT Program was Cost-Effective From the
Perspective of the Total Resource Cost-Benefit Test.
The Total Resource Cost-Benefit Test is a broad measure of program cost-effectiveness. It
seeks to summarize and compare all the monetary costs and benefits of the program,
regardless of who pays the costs or receives the benefits. The following costs are included in
the test:
 EVT’s program costs - including any rebates or incentives paid to participants and
performance incentives paid to EVT;
 Participant and Third Party Costs – mainly the measure costs above EVT rebates
that were paid by participants.
 Administrative Costs– including DPS’s verification costs and the cost of the Contract
Administrator.
The following program benefits are included in the test:
 Resource Benefits - The great majority of the resource benefits from EVT programs
come from electric energy and capacity savings. Resource benefits also include the
net effects on oil and other fossil fuel consumption of improving building insulation
and of switching customers from electric space and water heating to other fuels.
Decreases in water consumption from energy efficient washing machines, etc. are also
included in resource benefits.
22
Table 6:
2005-2007 Total Resource Cost Test Inputs and
Results
Business
Sector
kWh savings
EVT Costs (including incentive payments)
Participant and Third Party Costs
Administrative costs (DPS,CA,FA)
Total Costs
Resource Benefits (Lifetime)
Resource Benefit $/kWh
Benefit /Cost Ratio
Residential Sector
All Programs
87,486,000
115,252,000
202,739,000
$24,995,100
$ 19,721,129
$889,004
$23,505,258
$20,659,400
$889,004
$48,500,358
$40,380,529
$ 1,778,088
$45,605,233
$45,053,662
$90,658,975
$73,254,274
$69,272,084
$142,526,361
$0.0330
$0.0243
$0.0327
1.61
1.54
1.57
Test results are shown in two formats.

Resource benefits $/kWh expresses the test results as net resource benefits per kWh of
electricity conserved as a result of EVT programs.

The benefit/cost ratio compares the costs of the program to Vermonters to the benefits
they receive. Overall, Vermont receives $1.57 in benefits for every dollar spent on the
EVT program. Most of those benefits are in the form of reduced energy costs for
Vermont businesses. This test does not attempt to compute the increased employment
or economic activity that results from reducing the energy costs of Vermont
businesses
H. During the period 2005 through 2007, the EVT Program was Cost-Effective From the
Perspective of the Societal Cost-Benefit Test.
The Societal Cost-Benefit Test is a broader measure of program cost-effectiveness than the
Total Resource Cost Test. In addition to the costs and benefits included in the Total Resource
Cost Test it includes the per kWh environmental benefits of reduced generation. It also
includes a 10% risk adjustment factor that takes into account that experts consider energy
efficiency improvements to be a more reliable way to meet changing electric capacity needs,
than investments in new electric supply.
The reasons for the superior reliability of energy efficiency programs are detailed in the
Board’s 4/16/90 order in case 5270. In brief, efficiency programs have a much shorter lead
time than the construction of new power plants or transmission lines. The scale and budgets
of efficiency programs can be quickly increased or decreased to meet unexpected changes in
the amount, or location, of load.
Table 7 shows Societal Cost Test results for EVT’s program as a whole, and separately for the
Business and Residential market segments.
23
Table 7:
2005-2007 Societal Cost Test Inputs and Results
Business
Sector
kWh savings
EVT Costs (including incentive payments)
Participant and Third Party Costs
Administrative costs (DPS,CA,FA)
Total Costs (adjusted for risk)
Resource Benefits (Lifetime)
Environmental Benefits ($0.007 kWh)
Total Benefits
Societal Benefit $/kWh
Benefit /Cost Ratio
Residential Sector
All Programs
87,486,000
115,252,000
202,739,000
$24,995,100
$ 19,721,129
$889,004
$23,505,258
$20,659,400
$889,004
$48,500,358
$40,380,529
$ 1,778,088
$41,044,710
$40,548,296
$81,593,078
$73,254,274
$726,134
$69,272,084
$956,592
$142,526,361
$1,682,734
$73,980,408
$70,228,676
$144,209,095
$0.0393
$0.0298
$0.0395
1.80
1.73
1.77
These are very robust cost-benefit test results. Finding E noted that program savings are
based on engineering estimates that, no matter how carefully applied, may differ slightly from
actual savings. Based on the high cost-benefit ratios shown in this analysis, the EVT
programs would be cost effective even if actual savings were only 57% of estimated savings.
Clearly, the EVT engineering estimates are far more accurate than that. There can be no
doubt that, even allowing for a reasonable level of uncertainty; the Efficiency Vermont
program is highly cost effective.
24