Custom Impact WO033
Research Plan Addendum IV –
Revisions to Sampling
Addendum and Scope
Adjustment
2010-2012 Impact Evaluation
Submitted to:
Energy Division
California Public Utilities Commission
505 Van Ness Ave.
San Francisco, CA 94102
Submitted by:
Itron, Inc.
1111 Broadway, Suite 1800
Oakland, CA 94607
and
KEMA, Inc.
155 Grand Avenue, Suite 500
Oakland, CA 94612
April 3, 2012
Table of Contents
1 Research Plan Addendum IV ............................................................................ 1-1
1.1 AD Period Scope and Budget Adjustments ................................................ 1-1
1.2 Sampling Plan Update ................................................................................ 1-2
1.2.1 Changes to the Overall Sampling Plan ...................................................................... 1-2
1.2.2 Changes to Periodic Sampling from the AD Population ............................................ 1-3
1.2.3 Expected M&V Precision by Sampling Domain ......................................................... 1-4
List of Tables
Table 1-1: Evaluation Activity Reduction and Associated Reduction in AD Period
Budget .............................................................................................................. 1-1
Table 1-2: Current Research Plan-Based Sample Sizes for WO033 ..................... 1-2
Table 1-3: Adjusted Sample Sizes for WO033 ....................................................... 1-3
Table 1-4: Forecasted AD Period Sampling for M&V ............................................. 1-3
Table 1-5: WO033 M&V Sampling Precision by IOU-Electric Domain ................... 1-4
Table 1-6: WO033 M&V Sampling Precision by IOU-Gas Domain ......................... 1-4
Itron, Inc.
i
Table of Contents
1
Research Plan Addendum IV
This document is an addendum to the Custom Impact Research Plan that addresses scope
adjustments and related budget changes, and an update to project sampling plans.
Scope adjustments involve a reduction to planned measurement and verification (M&V) and netto-gross (NTG) efforts in the AD period. The approximate related budget reduction is also
addressed in this addendum. Scope-level reductions are designed to ensure that planned project
activities are completed before the end of 2013. Adjustments address the after-decision (AD)
period, as defined in the Custom Impact Work Order 33 (WO033) Evaluation Plan dated
December 27, 2011.
A specific sample design and sample selection is not included in this addendum. Three periodic
sampling updates are needed for the AD period. Each sampling update will be submitted as an
addendum to the research plan or as a standalone supporting document.
1.1 AD Period Scope and Budget Adjustments
Table 1-1: identifies evaluation activity reduction and associated likely reduction in overall
budget required to complete the evaluation.
Table 1-1: Evaluation Activity Reduction and Associated Reduction in AD Period
Budget
Impact Evaluation Activity
Budget
Reduction ($)
M&V Points (including lower rigor activities)
$ 814,001
NTG Points
121,038
Total
Scope Reduction
150 M&V points
300 NTG points @ $403 each
$ 403,288
The 150 point reduction in the number of M&V points is estimated. This consists of 75 points
that are stand-alone WO033 points that do not overlap with WO002 efforts (275 M&V points
reduced to 200 M&V points), plus an estimated reduction of 75 points in overlapping
WO002/WO033 points (125 points reduced to 50 points). The overlapping points represent a
census, but it is now anticipated that the number of completed WO033 projects (by the end of
2012) that overlap with WO002 may be much lower than originally planned.
Itron, Inc.
1-1
Research Plan Addendum IV
2010-2012 Custom Impact Evaluation Research Plan – Final Draft
The change in budget includes not only a reduction in the number of M&V points but also an
increase in anticipated costs per point. The net difference represents anticipated savings of
$841,001; some additional cost savings may occur due to quicker than anticipated site processing
and/or reduced M&V costs per point. The assumption for the remaining 250 M&V points is that
they will cost $10,000 per point for each assigned engineer, covering labor and other direct costs
such as travel.
The 300 point reduction in the number of NTG points is also estimated. Cost savings associated
with that volume of work reduction is estimated to be $121,038.
1.2 Sampling Plan Update
This section of the addendum reflects sampling updates in response to the above scope changes.
More specifically, this section covers the following topics:

Changes to the overall sampling plan;

Changes to periodic sampling from the AD population; and

Expected M&V precision by sampling domain.
As described earlier, detailed sampling addenda will be prepared for each of three AD period
sample pulls.
1.2.1 Changes to the Overall Sampling Plan
The current research plan-based sample sizes for the BD and AD periods, segmented by each
type of sample point, are shown in Table 1-2Table 1-2: below.
Table 1-2: Current Research Plan-Based Sample Sizes for WO033
BeforeDecision
After-Decision**
Total
M&V Points (Gross Realization-Rates + NTG)
200
400
600
Lower Rigor Points (Qualitative + NTG)
100
0
100
Incremental NTG-Only Points
480
680
1,160
Total*
780
1,080
1,860
Impact Evaluation Component
* All points incorporate NTG evaluation in addition to gross impact evaluation efforts.
** Some after-decision M&V points will include pre-installation data collection performed under WO002.
The adjusted sample sizes for the BD and AD periods, addressing the scope changes presented in
Section 1.1 above, are shown in Table 1-3 below.
Itron, Inc.
1-2
Research Plan Addendum IV
2010-2012 Custom Impact Evaluation Research Plan – Final Draft
Table 1-3: Adjusted Sample Sizes for WO033
BeforeDecision
After-Decision**
Total
M&V Points (Gross Realization-Rates + NTG)
200
250
450
Lower Rigor Points (Qualitative + NTG)
100
0
100
Incremental NTG-Only Points
480
530
1,010
Total*
780
780
1,560
Impact Evaluation Component
*All points incorporate NTG evaluation in addition to gross impact evaluation efforts.
**Some after-decision M&V points will include pre-installation data collection performed under WO002.
The sample sizes shown in Table 1-3 are targets that the evaluation team considers to be
attainable in the timeframe required for timely reporting of evaluation results. However, the
evaluation may or may not achieve these targets. Limited time remains in the evaluation
timeline to complete all remaining evaluation activities, and IOU data quality issues, lack of IOU
timeliness and thoroughness in data deliverables and lack of cooperation by IOU representatives
and customers are all considerations that will factor into evaluation success in meeting targets.
1.2.2 Changes to Periodic Sampling from the AD Population
Periodic M&V sampling from each of three AD periods will generally be performed as described
in previous research plan documents. However, an attempt will be made to complete the
majority of non-overlapping WO033 points in the first two AD periods. Additionally, while an
attempt will be made to oversample in the first two AD periods, less effort (compared with the
BD period) will be made to ensure completion of each point and fewer point substitutions will be
made for difficult to obtain/refusal points. Instead, it is anticipated that make-up points will be
obtained from the third AD period as needed. This is characterized with the forecasted sampling
plan shown in Error! Reference source not found..
Table 1-4: Forecasted AD Period Sampling for M&V
WO002 EAR
Overlap Points
NonOverlapping
Attempts
Non-Overlapping
Achieved
Total
Q3/4 2011
1
92
80
81
Q1/2 2012
9
92
80
89
Q3/4 2012
40
46
40
80
Total
50
230
200
250
Participation Period
As specified in the original research plan, NTG data collection will also be completed for all 250
M&V points. Sampling for NTG-only points will generally be completed in accordance with the
sampling strategy outlined in the BD Period Sampling Addendum.
Itron, Inc.
1-3
Research Plan Addendum IV
2010-2012 Custom Impact Evaluation Research Plan – Final Draft
1.2.3 Expected M&V Precision by Sampling Domain
Table 1-5 and Table 1-6 present the expected precision for the WO033 non-overlapping M&V
sample by sampling domain. Shown is expected precision for the combined BD plus AD period
(based on the full adjusted M&V sample size). The design generally seeks relative precision of
10-15 percent at the 90 percent confidence interval for the combined BD and AD periods.
Table 1-5: features sampling precision by IOU-electric domain, while Table 1-6 presents
sampling precision by IOU-gas domain.
Table 1-5: WO033 M&V Sampling Precision by IOU-Electric Domain
Utility
Positive Electric
Sampling Units
(N)*
M&V
Allocation
(n)
Assumed
Error
Ratio**
Expected
Precision at 90%
C.I.
Expected
Precision at
80% C.I.
PG&E
6,320
100
0.80
13%
10%
SCE
2,298
100
0.80
13%
10%
SCG
0
0
-
-
-
996
60
0.80
17%
13%
9,614
260
-
8%
6%
SDG&E
Total Electric
* Forecasted figure.
** Error ratio obtained from 2006-2008 PG&E Fabrication and Manufacturing Contract Group results.
Table 1-6: WO033 M&V Sampling Precision by IOU-Gas Domain
Utility
Positive Gas
Sampling Units
(N)*
M&V
Allocation
(n)
Assumed
Error
Ratio**
Expected
Precision at 90%
C.I.
Expected
Precision at
80% C.I.
PG&E
532
80
0.80
14%
11%
SCE
12
0
-
-
-
SCG / SDG&E
374
60
0.80
16%
12%
Total Gas
918
140
0.80
10%
8%
*Forecasted figure.
**Error ratio obtained from 2006-2008 PG&E Fabrication and Manufacturing Contract Group results.
Itron, Inc.
1-4
Research Plan Addendum IV