STATE OF CALIFORNIA
Edmund G. Brown Jr., Governor
PUBLIC UTILITIES COMMISSION
505 VAN NESS AVENUE
SAN FRANCISCO, CA 94102-3298
California Public Utilities Commission
Energy Efficiency Evaluation Measurement and Verification
CPUC Contract Number:
09PS5863A
Prime Contractor:
Itron, Inc.
Work Order Number:
ITRON012 – Amendment #1
Work Order Project Title:
Overarching Market and Process Evaluation, Nonresidential Programs
ED Work Order Project Manager:
Jean Lamming, 415-703-2142, (please direct any comments on this work order to
JL2@cpuc.ca.gov)
ED Contract Manager:
Tim Drew
Contractor Project Manager:
Christina Torok
Work Order Budget:
$1,166,292 (Includes $153,650 under KEMA W.O.)
Amendment:
This Amendment adds scope and budget to WO012. The new total contract amount is
increased from $441,515 to $1,012,640 for Itron and its subcontractors.
Term:
Effective Date through June 30, 2012
Planned Budget:
$2.3M (through 6/30/2012) Includes XREF 17, XREF 56, and half of XREFs
36, 37 and 38
Purpose: This amendment work order governs the scope of work for EM&V Work Plan Study 17, 56, and the nonresidential portions of XREF 36, 37 and 38. The
project name is “Nonresidential Overarching Process Evaluation and Market Research”. The work scope included in this Work Order Itron012 supports Contract Tasks
4, 5, and 6 of Contract 09PS5863A and its Amendment One.
This Work Order Amendment does not authorize work that is beyond the scope of the work authorized by the underlying prime contract, may not expand the budget
of the underlying prime contract, nor can this Work Order change, amend or modify any of the substantive terms and/or conditions of the underlying prime contract
nor add any new substantive terms and/or conditions to the underlying prime contract.
The actual costs to complete an approved Work Order shall not exceed the amount authorized in this Work Order. If, in the performance of the work, the Contractor
determines that the actual costs are likely to exceed the estimated costs, the Contractor shall immediately notify the Energy Division Contract Manager and Project
Manager. Upon such notification, the Energy Division Contract Manager will:
(1)
(2)
(3)
Amend the Work Order scope to accomplish the work within estimated costs; or
Amend the Work Order budget to accomplish the existing scope; or
Terminate the Work Order.
Any expenses incurred by the Contractor that have not been authorized shall be borne by the Contractor. No amendments to this Work Order shall be made for work
undertaken without the specific approval of the Energy Division Contract Manager.
Approvals:
07/05/11
_________________________________________________
ED Contract Manager
Date
1
________________________________________________
Contractor Project Manager
Date
Work Order 12 Amendment 1
Section 1. Scope and Rationale for Amendment
This amendment affects Tasks SOW. Objectives and required approach to this overarching evaluation have
evolved to keep pace with the nonresidential process and market evaluation environment.
Project Management
Task 00 – Project Management
This Task is being added to support ongoing management and coordination of the activities and objectives of
this work order. It also provides regular status reports and meetings between WO12 project management and
ED project management. This task also provides funding for the WO12 team to provide necessary
management and quality control of work conducted by subcontractors retained for tasks contained in this
work order.
Budget: Itron $45,000:
Study Development Framework
Task 1 – Nonresidential Program and Portfolio Characterization
The program characterization database was uploaded in late April. The overall effort took moderately more
resources than anticipated, and was completed with a larger share of Itron resources than KEMA resources.
Budget: Itron $70,000
(Old budget: Itron $60,800)
Task 2 - Develop IOU-ED Collaborative Study Development Framework
A meeting was held to gather input to build the framework. However, the degree to which collaboration in
study scoping is practical or relevant no longer supports formalizing such a framework. This task is no longer
needed and is terminated.
Budget: Itron $0,
(Old budget: Itron $7,000;)
Nonresidential Market and Process Study Development
Task 3 - Collaborate with IOU staff in Developing Study Scope
This task funded the WO12 team efforts to contribute to scope development for IOU-led studies and to
identify tasks and studies that should be undertaken by ED staff or its contractors. WO12 team met with IOU
EM&V staff, met with IOU program managers, met with ED and ED consultants, reviewed monthly reports,
previous evaluation studies, current tracking data, the California strategic plan and other sources to assist in
scoping. The team worked with ED staff to prioritize studies to identify the most critical arenas for mid-course
corrections, support for bridge funding decisions, and portfolio planning and strategy.
Through these efforts the On-Bill Financing and Benchmarking studies were identified, and these studies are
underway. Recently a need was identified for a study of Third Party programs that focus on the nonresidential
sector, and this Work Order Amendment funds a research plan for that effort.
2
Document1
While most of the scoping work is complete, there is interest among some Energy Division and Itron/KEMA
staff in exploring possible hybrid process/impact work. Over the coming weeks, WO12 team will meet with
the impact team to discuss such research needs. If an additional study should arise from these discussions, it
will be added as a new sub-task to Task 5.
For IOU managed studies, ED staff and the Itron/KEMA team will continue to be involved in all aspects of study
implementation, providing technical input and oversight, as well as facilitating coordination. The oversight of
IOU managed projects will be funded through category 4 tasks, as discussed in the following text.
The original budget for Task 3 covered collaborative study ideation and the designation of an entity to manage
the study through planning and implementation. As described in the original work order, a small Itron/KEMA
team was dedicated to this task, but the degree of effort required was difficult to predict. The original budget
provided for Itron to spend approximately 215 hours. The actual hours needed exceeded these early
expectations. However, the scoping work included the development of research needs in support of OBF and
benchmarking, as well as the selection of contractors and writing of RFPs, the completion of a supporting data
request for OBF, and refinement of selected subcontractor proposed SOW.
Revised Budget: Itron $100,000;
(Old Budget: Itron $40,425)
Task 4 Oversight of IOU managed studies
Two new IOU led process evaluation is added to the list of IOU managed studies with oversight provided by
WO12.
Task 4.4 Workforce Education and Training Process Evaluation
Study Budget $500,000
Budget: Itron $8,000
Task 4.5 Oversight of IOU Nonresidential Process Evaluation
Study Budget $1,750,000
Budget: Itron $36,750:
Task 5 Plan and Implement ED-Led Market and Process Evaluation Studies
Per the process described in work order 12, new sub-tasks to task 5 have been identified as follows:
Task 5.2.1 Overarching Study on Integration Effectiveness, Study Implementation and Reporting
Budget: Itron $107,500 :
Task 5.4 On-Bill Financing Process Evaluation Study
Budget: Itron $21,600: Cadmus $175,000
This majority of this evaluation study will be subcontracted to Cadmus. Specifics surrounding Cadmus tasks,
deliverables, timeline, and personnel are provided in WO12 Task Order 5.4 subcontract to Cadmus.
Task 5.5 Benchmarking Process Evaluation
Budget: Itron $15,600: NMR $150,000
3
Document1
This majority of this evaluation study will be subcontracted to NMR. Specifics surrounding NMR tasks,
deliverables, timeline, and personnel are provided in WO12 Task Order 5.5 subcontract to NMR.
Task 5.6 Benchmarking Process Evaluation- Technical Report
Budget: Itron $6,000 Optimal Energy $50,000
This majority of this Task will be subcontracted to NMR. Specifics surrounding NMR tasks, deliverables,
timeline, and personnel are provided in WO12 Task Order 5.5 subcontract to NMR.
Task 5.7.0 Third Party Assessment, Research Plan
Budget: Itron: $25,000
This research plan will address nonresidential third party programs as a whole, investigating and documenting
the unique successes and challenges of these program models and the associated administration.
Task 5.8.0 Nonresidential Best Practices, Research Plan
Budget: Itron: $25,000,
Overarching Management and Quality Control
Task 6 –Oversight of PPM and MTI Construction
At this time the scope of this task is unclear. For now we reduce the budget, and will hold off activity until
further notice.
Budget: Itron $40,000
(Old budget: Itron $51,600)
Task 7 - Data Collection Guidelines
The scope for the data collection guidelines effort remains the same, but the Itron budget reduced.
Budget: Itron $35,000
(Old budget: Itron $56,100)
Task 8 – Coordinate Nonresidential Sector-Wide Primary Data Collection
The Work Order 12 team will provide overarching coordination of data collection efforts. This coordination
effort will span all data collection across the nonresidential sector, including CSS, process studies and impact
work. Tools have been developed (under other work orders) that keep track of primary data collection efforts,
including on-site surveys, logger/meter installs, and CATI surveys. These tools also house accomplishments
versus quotas by strata for each effort. the WO12 Team will expand upon these tools to include data
collection of IOU led studies, as well as creating a repository of IOU research documents.
The Team will seek to match the opportunities present in the planned data collection activities to the needs of
the overarching research. The data collection plans will be examined for opportunities to collect data in
support of the overarching research, and those opportunities pursued in coordination with topical research
study teams.
The original budget for this task allowed for Itron to dedicate approximately 8 weeks of labor to this effort.
However, there are somewhat fewer than anticipated IOU led process evaluations, and the team is able to
4
Document1
leverage existing data collection management tools (developed under other work orders) such that the
expected cost for this activity is reduced.
Budget: Itron $25,000
(Old Budget: Itron $51,800)
Reporting
Tasks 9 and 10
It is unclear at this time whether there is a need for overarching reporting above and beyond what is called for
within the scopes of Tasks 1 through 8. Therefore, this additional layer of reporting is suspended until such a
need is clear. No budget has been spent to date on these tasks.
Budget: Itron: $0
Quarterly/Monthly Reports: Old budget: Itron $51,800;
Annual Reports: Old Budget: Itron $44,800;
5
Document1
Section 2. Summary of Task Budgets and Work Order 12 Changes
The overall Work Order Task Budget changes for Itron and its subcontractors retained for studies on
Benchmarking and On-Bill Financing are summarized in Table 2. The Itron personnel rates are summarized in
Table 3 and in the staff matrix workbook in Attachment B.
Table 1. Work Order Budget Summary
Task
Tasks 1 - 3, Study Framework
00
Work Order Management/QC
1
Program Characterization DB
Develop IOU-ED Collaborative Study Development
2
Framework
3
Study Ideation/RFP/SOW
Task 4, Oversight of IOU Studies
4.1
Oversight - Sempra OA NR
4.2
Oversight ACEEE "Big Savers" Best Practices Study
4.3
Energy Savings Calculation Tools Development for EBCx
4.4
Oversight of WE&T
4.5
Oversight, IOU NR Process (XREF 19)
Task 5, ED Studies
5.1.0 Selected NR Market Assessments, Research Plan
5.2.0 Integration Effectiveness Research Plan
5.2.1 Integration Effectiveness, Implementation and Reporting
5.3.0 Adoption Effectiveness Research Plan
5.4
On-Bill Financing
5.5
Benchmarking Study
5.6
Benchmarking Study - Technical
5.7.0 Third Party Assessment, Research Plan
5.8.1 NR Best Practices, Research Plan
Tasks 6 - 10, Overarching Activities
6
PPM/MTI
7
Data Collection Guidelines
8
Coordinate Sector Wide Data Collection
9
Monthly Quarterly Rpting
10
Annual Reporting
Total
6
Original Budget
Itron
Total
New Budget
Itron
Sub
Budget Change
Itron
Sub
60,800
76,000
45,000
70,000
-
45,000
9,200
-
7,000
40,425
9,333
56,595
100,000
-
(7,000)
59,575
-
33,600
1,600
2,240
-
48,000
1,600
2,240
-
33,600
1,600
2,240
8,000
36,750
-
8,000
36,750
-
21,000
9,375
9,375
-
30,000
12,500
12,500
-
21,000
9,375
107,500
9,375
21,600
15,600
6,000
25,000
25,000
175,000
150,000
50,000
-
107,500
21,600
15,600
6,000
25,000
25,000
175,000
150,000
50,000
-
51,600
56,100
51,800
51,800
44,800
441,515
68,800
74,800
74,000
69,066
59,733
595,167
40,000
35,000
25,000
637,640
375,000
(11,600)
(21,100)
(26,800)
(51,800)
(44,800)
196,125
375,000
Document1
Table 2. Staff Information Matrix
<Removed for Public Version>
Section 3. Deliverables
The preliminary list of deliverables for the Nonresidential Overarching Market and Process Evaluation includes
the following items:







7
Task 1 – Program and Portfolio Characterization
1. Database of program information
(delivered)
Task 3 – Collaborate with IOUs Staff in Developing Study Scope
1. Initiation of work orders for ED-led Studies
Task 4 – Oversight of IOU Managed Studies
1. Reviews of research plans, memorandums, data collection instruments and reports
Task 5 – Plan and Implement ED Led Market and Process Evaluations
1. Selected Nonresidential Sector Market Assessments, Research Plan
2. Overarching Study on Integration Effectiveness, Research Plan (submitted)
3. Overarching Adoption Effectiveness Assessment, Research Plan (on hold)
4. CEI Research Plan
5. Benchmarking Process Evaluation
6. On-Bill Financing Process Evaluation
Task 6 – Oversight of PPM and MTI Construction
1. Memorandum documenting construction, consistency of PPMs
Task 7 – Data Collection Guidelines
1. Guidelines governing primary data collection in the areas of:
1. Firmographic characteristics
2. Satisfaction scales and questions
3. Integration effectiveness
4. Other topics, to be determined
Task 8 – Coordinate Nonresidential Sector-Wide Primary Data Collection
1. Data collection coordination and reporting tool, maintenance and administration
2. Mapping of WO12 data collection needs to planned data collection efforts
Document1