Custom Impact WO033
Non-Residential New Construction
Addendum
to the
2010-2012 Impact Evaluation
Research Plan
Submitted to:
Energy Division
California Public Utilities Commission
505 Van Ness Ave.
San Francisco, CA 94102
Submitted by:
Itron, Inc.
1111 Broadway, Suite 1800
Oakland, CA 94607
(510) 844-2800
and
KEMA, Inc.
155 Grand Avenue, Suite 500
Oakland, CA 94612
May 10, 2012
Non-Residential New Construction
The Savings By Design programs (SBD) offered by California’s Investor-Owned Utilities (the
IOUs) include design assistance and financial incentives to improve the energy efficiency of
commercial & industrial new construction and load expansion projects. The incentive program
has two principal approaches to program participation:
 Systems Approach
 Whole-Building Approach
Systems Approach
The Systems Approach is applicable to projects where specific measures or systems have been
identified for improvement in a new or expanded facility. Energy savings are calculated for the
individual measures using standard algorithms or simulation software where the baseline
equipment efficiency is determined by minimum code requirements or industry standard
practices. The efficiency measures incentivized through the program are broadly available in the
marketplace though not currently typical practice. Systems covered under this approach include:
 Daylighting Systems
 Interior Lighting Systems
 HVAC Systems
 Service Hot Water Systems
 Refrigeration Systems
 Other Systems
Whole-Building Approach
The Whole-Building Approach is not limited to any particular measures, but provides incentives
based on reduced energy consumption relative to Title 24. This program component provides
design assistance and building energy simulation modeling to help provide an optimized “wholebuilding” or integrated design. In addition to informing the design process, the simulation
models are used to calculate the estimated total annual energy savings for the building compared
to the Title 24 minimum requirements. The analysis can be prepared by the design team, or by
an energy consultant provided by the utility, using an approved computer tool. DOE-2, and
eQUEST are examples of computer tools approved for use by the program.
The programs offer incentives to owners and design teams of new commercial and industrial
projects that are more efficient than projects meeting minimum code requirements and/or
improving upon standard industry practice. Participants in the SBD program systems element
will be evaluated under the overall non-residential Custom work order (WO033). The whole
building projects subset of these programs will be evaluated via the process described below
while standard custom engineering analysis will be applied to other new construction projects.
Non-Residential New Construction Whole Building Evaluation
The Team has developed a comprehensive and efficient process for estimating the Whole
Building impact of California Investor-Owned Utilities’ Statewide SBD program. The
1
methodology builds upon the approach used for evaluating the net and gross impacts of the
1999-2001, 2002, 2003, 2004-05, and 2006-2008 Savings By Design Program as part of the
Building Efficiency Assessment. These same methodologies were also applied on the California
Board for Energy Efficiency (CBEE) Nonresidential New Construction Statewide Baseline
Study. The evaluation methodology is based on engineering models of participant projects. The
evaluation will include the following steps:
 Conduct detailed on-site surveys and build/modify computer energy simulation models
for each building in the sample using DOE-2, Energy Pro or eQuest.
 Conduct focused short-term metering, with an emphasis on capturing peak load operating
conditions for measures with large demand savings. The metered data will also be used to
calibrate analysis simulation models.
 Calculate gross and net energy savings and DEER-defined demand reduction by end use
and for whole buildings as the difference between as-built and baseline energy
efficiencies.
 Conduct Decision-Maker and Design Team surveys to determine the net to gross ratios
for each sampled building. Develop quantifiable information on the changes in building
efficiency relative to the new Title 24 requirements.
 Identify and record specific building and equipment characteristics (e.g. types of glazing,
types of lamps, ballasts and light fixtures, HVAC system types, etc.)
 Calculate the “whole-building” impacts to both gas and electric consumption (positive
and negative).
Gross Energy (kWh and Therm) Assessment
A series of parametric simulation runs will be used to estimate gross savings for participants on a
whole-building and measure-class basis by subtracting the as-built energy consumption and
demand from the baseline energy consumption and demand. The parametric runs proposed for
this study are listed below:
As-Built Parametric Run
Once the models are completed, calibrated to all available billing data, interval meter data,
and/or end-use metered data and QC checked, the as-built parametric run will be performed.
Monthly schedule variations resulting from partial occupancy, capacity expansion and building
startup will be minimized, and the models will be run using the Climate Thermal Zone (CTZ)
long-term average weather data from the California Energy Commission and the 1991 calendar
year.
Baseline Parametric Run
Key building performance parameters will be reset to a baseline conditions to calculate gross
energy savings for participants. One difficulty each project will need to overcome will be the
determination of the appropriate baseline. The version of the Title 24 Standards in effect when
the project applied for a building permit will be the primary reference for establishing baseline
performance parameters. When available, surveyors will be able to determine code version
using the building plans. In cases where building plans are not available the Team will contact
2
the consultant responsible for preparing the Title 24 documents. For each project entered into
the database the surveyor will be required to select the appropriate baseline condition.
Gross Peak Demand (kW) Assessment
To estimate the Gross Peak Demand (kW) impact the evaluation will use an approach very
similar to the approach used to estimate the Gross Energy impact. A series of parametric
simulation runs will be used to simulate gross demand savings for participants on a wholebuilding and measure-class basis by subtracting the as-built demand from the baseline demand.
The same set of parametrics will be run as used in the Gross Energy Impact estimation but with
the DEER default peak demand weather file and the corresponding calendar year instead of
actual weather files and matching current calendar year. Following the protocol, only the impact
on the DEER-defined peak days will be reported.
Net Energy and Demand Impacts
The Nonresidential New Construction (NRNC or Building) component of the Custom sample
will use a custom version of the Net-To-Gross Survey developed by the Nonresidential Net-toGross Working Group. This will be a modification to the common self-report approach, to
perform the net-to-gross (net of free riders) assessment for all large nonresidential programs.
The self-report approach involves asking one or more key participant decision-makers a series of
structured and open-ended questions about whether they would have installed the same energy
efficient equipment in the absence of the program. This method for estimating the net to gross
ratios (NTGRs) is consistent with the CPUC’s Guidelines for Estimating Net-To-Gross Ratios
Using the Self-Report Approaches. The battery of questions developed by the Nonresidential
NTGR working group will be modified to reflect the special needs of whole building projects.
Building on lessons learned in past evaluations and the experience of evaluation contractors, the
group has developed both a common battery of questions and associated scoring algorithms for
calculating NTGRs. A method to develop Net-To-Gross ratios at the measure and/or program
level will be coordinated with the early EM&V tasks and the crosscutting EM&V study. The
general approach taken will be similar to previous cycles, which involved a stratified sample by
size and rigor categories.
The NTG team will conduct telephone surveys, interviewing account representatives (when
necessary), participating customers and members of the design teams. Professional staff will be
used to conduct interviews for the largest projects, with CATI interviews for basic and low cost
low rigor projects. A second, follow up interview will occur for pre-post design (Early EM&V)
sites to determine the baseline and where Title-24 codes do not apply for these projects, an
industry typical practice will need to be established. Information on typical practices in specific
industries or applications will be gathered from secondary sources such as manufacturers and
end users outside the program.
Overall Measurement and Verification (M&V) Plan
This section describes the overall methodologies for all data collection to be performed for the
2010-2012 NRNC Whole Building sample points in the Custom evaluation. Data collection
3
pertinent to the study includes recruiting, survey instrument design, on-site surveys and
modeling, and model calibrations.
Recruiting
Project evaluation engineers will use contact information found in the program application files
to identify key decision-makers for the program participant surveys and scheduling on-site
surveys. These contacts will be used as the initial contact. The engineers will follow up with
additional contacts identified by the initial contact. New construction projects differ from retrofit
projects in that the list of contacts usually includes members of the design team, commissioning
agents, and development team. As in past studies, we expect that several different respondents
may need to be contacted for approval to conduct site visits. To expedite the on-site survey
process, the interviewer will also ask the customer to have building plans available for the
surveyor when they arrive at the scheduled date and time. When possible, the surveyor may
obtain copies of the plans to expedite the review process. Contacts from previous NRNC studies
will also be used whenever possible.
Survey Instruments
The on-site survey instrument will be an updated version of the instruments used for the previous
evaluation. Revisions to the instrument will be made to reflect changes to the Savings By
Design program, specifically new measures, and to reflect updates to Title-24.
Previous evaluations have had a considerable number of refrigerated warehouses in the sample.
Due to the unique nature of refrigerated warehouses in comparison to other commercial
buildings a separate survey form has been developed for such sites. The survey instrument used
will also be essentially the same as that used in the previous evaluation with updates for the 2008
version of Title-24, if necessary.
On-site Surveys and Modeling
The on-site evaluation staff will consist of skilled engineering personnel from the Team. The onsite visits will require from four hours to a full day, by one or more engineers, depending on the
size and complexity of the project. Sites which have short term metering will require a second
visit to retrieve the metering equipment.
Data Collection for Commercial Buildings
The on-site data collection process involves the following:
 Survey with building personnel to determine hours of operation and schedules for energy
consuming loads.
 Building orientation, general building configuration and architectural features.
 Envelope construction, construction assemblies, including fenestration types and
configurations based on orientation.
 Lighting, plug, and miscellaneous electrical power densities.
 HVAC configuration, including layout, zoning and control.
4



Schedules, setpoints, and monitoring data from Building Energy Management systems
and/or setup of additional data collection from existing control points.
Pro-forma budgets, occupancy rates, and records of occupancy during the initial start-up
phase.
Other energy-consuming equipment inventories.
The step-by-step methodology for on-site data collection and modeling methods for commercial
sites follows:
1. Site recruitment - the site will be qualified and recruited into the study. Once recruited,
engineers will review the program file for project difficulty and assign the site to
appropriate audit staff member.
2. Engineering file review prior to on-site data collection – the evaluation engineer
assigned to the site will review program records for measure identification including
measures with large demand reductions. Project files will be thoroughly reviewed for
identification of specific measures and measure parameters to be metered.
3. Development of site plan – a site-specific evaluation plan will be developed. These plans
will include a metering plan, when applicable.
4. Detailed on-site data collection – the evaluation engineer responsible for site analysis
will conduct and/or oversee all data collection and be directly responsible for its integrity.
5. Installation of metering equipment (only for sites identified as a metering site) – if
applicable the site specific end-use metering plan will be implemented by the evaluation
engineer. The engineer may be assisted by other Team staff or an electrician. Efforts to
utilize and/or establish new control points for existing building energy management
systems will be explored prior to the installation of stand-alone metering equipment unless
there is a reason to be skeptical of the quality of the data from the energy management
system.
6. Simulation data entry - the on-site surveyor will enter data directly into the eQuest or
Energy Pro building energy simulation tool. Where possible, we plan to utilize the
simulation modeling utilized in the program filing, with modification based on our
evaluation findings. All data problems and data inconsistencies will be identified and
corrected within a few days of the on-site.
7. End-use metering equipment retrieval (only for sites identified as a metering site) after the planned metering duration has passed the metering equipment will be retrieved
from the site by the senior engineer who installed the equipment. If new control points
were setup for data collection purposes, we will attempt to gather the data from the
building energy management system via electronic transfer of the data from our on-site
contacts.
8. Metered data analysis (only for sites identified as a metering site) - the metered data
will be analyzed according to the site-specific metering plan. The results will be
integrated into the QC’d eQuest or Energy Pro model if the metered data indicates
equipment run-times or equipment efficiencies that vary from the original inputs.
5
9. DOE-2 simulation modeling - An hourly simulation will be performed on the eQuest or
Energy Pro models to calculate the gross savings. The commercial grocery store with
refrigeration component and refrigerated warehouse whole building sites will use DOE2.2R simulation models to calculate the gross savings. The models will be checked for
reasonableness first by the modeler, and last by senior staff.
10. Calibration – Any data available that may useful for calibrating the model to the as-built
conditions will be explored even if not directly related to the calculation of measure
savings.
11. Savings and building characteristics quality control (QC) - The full set of end-use
parametrics are run for each participant building during the QC process to identify and
investigate under-performing measures.
12. Site report - A site-specific savings report will be produced, summarizing savings and
noting any discrepancies between the evaluation model and program estimates.
End-Use Metering
End-use metering will be performed with emphasis on capturing the peak load operating
conditions for end uses associated with measures with demand reductions. The Team will install
metering equipment and/or identify control points on the building energy management system
for a sample of commercial sites concentrating on projects with large demand reductions and
demand reductions with uncertainty. The final number of end-use metered/EMS sites is
dependent on the variability of the measures in the population and budget available.
Model Calibration
Engineering models will be calibrated where end-use metered data are available. Calibration will
be completed by the engineer that conducts the on-site visits and energy simulation modeling,
the one who is most familiar with the project and measures. Calibration is accomplished by
reading in the DOE-2 end-use load profile, along with the end-use metered data and visually
inspecting the load profiles. The end-use metered data will be compared with modeled output
and considered “calibrated” when the mean bias error and RMSE are within reasonable limits for
the equipment under consideration. If the initial as-built model output does not constitute a
reasonable match, the model inputs will be adjusted until the model output is representative of
the metered data. In cases where end-use metered data is not available or doesn’t cover a
sufficient metering period, models will be calibrated to whole-building billing data, and any
available end-use metering information will be compared to modeled usage as a reasonableness
check. Calibration activities will be overseen by experienced engineers who will “sign off” on
the calibrations to assure the most accurate models feasible.
Reliability, Bias and Uncertainty
Measurement bias
All instrumentation has some associated error, but through careful application of site
methodology and calibration to any available billing or interval meter data, SBD projects are
unlikely to have significant uncertainty stemming for instrument based measurement error. In
cases where measurement error has a bearing on site level estimates, such as an industrial
measure evaluated with logger data, instrument error will be documented and reported for
propagation of error analyses and/or site level uncertainty.
6
Experienced engineering staff will test and calibrate metering equipment prior to installing to
minimize error and quantify any identifiable bias in measurement. Further, all on-site surveyors
will have training, both in the office and on-site, to ensure that on-site building characteristics are
being collected appropriately.
A more significant contribution of uncertainty for SBD projects is simulation model data entry
errors. Given, the complexity of the projects, some instances of random errors in the data entry
process are inevitable. The nature of these types of errors are less likely to introduce a
systematic bias because the modeling error exists in both the baseline and as-built model and
such errors tend to “cancel out” of the final results. Estimates of energy savings and demand
reduction will be higher or lower in an unpredictable manner. Nevertheless, the potential for
these types of errors are minimized by a series of quality control checks performed throughout
the modeling and calibration process.
First, the surveyor must complete a checklist of items that may identify random errors in the
model inputs. The checklist, developed by experienced modelers, includes checks of lighting
power density at the space level and comparison of the sum of the modeled floor areas versus the
reported floor area for the site. Next, the site savings, whole building and measure savings must
conform to reasonable ranges or have an explanation why any values are out of range for the site.
A new QC step added for the 2006-2008 evaluation and continued for 2010-2012 will be a “load
shape QC” whereby a senior engineer will view the model output with data visualization
software to identify any errors not detected by the first two steps. The next QC is the analysis
QC, during which end-use realization rates for individual sites that appear to be much larger or
smaller than expected will be evaluated. If any site fails any of these analyses the model is
“kicked back” to the surveyor for revisions or justification of why any end-use realization is out
of range.
Baseline Issues
Title 24 standards will be used as a baseline for this study wherever applicable. In cases where
Title 24 is not applicable, such as industrial facilities, refrigerated warehouses, and
unconditioned spaces, the evaluation will use the best “typical practice” available for that space
type or application. Where typical practices are not easily available baseline issues will be
resolved in discussions with ED and DMQC and the NTG working group.
7