Statement of Work
Phase II HVAC Maintenance Study Behavioral Research
This study is a follow-up to the Phase I study, “HVAC Maintenance Energy Efficiency Study”.
That study examined the uncertainties and unknowns associated with HVAC maintenance and proposed research elements to mitigate the uncertainties and address the unknowns. Phase II research will be conducted via several studies and will consist of four primary categories of activities: field research, lab research, behavior research, and data analytics. This study comprises behavioral research focused on HVAC maintenance contractors and technicians, including both program-participating and non-participating contractors who provide service to residential and commercial customers throughout California Investor Owned Utility (IOU) territory. Maintenance includes refrigerant charge adjustment, duct testing and sealing, coil cleaning, and comprehensive approaches to improving system operation and efficiency of HVAC systems.
This study will involve gathering data about activity in the market to construct a sample frame, surveys and/or focus groups with technicians and contractors, and field observations of technicians conducting maintenance work. The specific research elements in Task 4 are recommended by the Phase I study
1
. The page citations are in parentheses.
This document describes the behavioral research planned for the second phase of HVAC maintenance research conducted by California IOUs and the California Public Utilities
Commission Energy Division.
Task 1. Kickoff Meeting and Check-in Meetings
The first task will entail a kickoff meeting to be held at a site in California to be determined by the Project Manager. The meeting will be attended by the Contractor’s key project staff, IOU program and evaluation staff, and CPUC staff, consultants and advisors (Project Team). The purpose of the meeting will be to clarify project objectives, establish points of contact, discuss research goals and methodologies, and make any necessary refinements in the schedule and work scope. The Contractor will be expected to have draft research plans and objectives, which will be discussed along with the project goals and research questions. The Contractor will be responsible for developing a meeting agenda under the supervision of the Project Manager. At the kickoff meeting, a follow-up meeting will be scheduled to discuss the draft research plan developed as part of Task 2. The Contractor will also be required to prepare a summary of the meeting, highlighting decisions made with respect to the project. The Contractor will participate in twice-monthly check-in meetings with members of the project team.
Task 1 Deliverable: Project Manager-approved meeting summary due within 10 days of the kickoff meeting.
Task 2. Develop Research Plan
1 Davis Energy Group, Inc. “HVAC Energy Efficiency Maintenance Study.” December 29, 2010. CALMAC ID
SCE0293.01.

Based on the discussions at the kickoff meeting, the Contractor will develop a draft research plan. This will include specific activities and methodologies for fulfilling the tasks in this
Statement of Work, timelines including key milestones and deliverables, and budget breakdowns. A meeting will be held with the Contractor’s team and ED and IOU team members to discuss and comment upon the draft. Based on this feedback, the Contractor will develop a final research plan.
Task 2 Deliverable 1: Draft research plan, timelines and budget within approximately two weeks of the kickoff meeting, due at least 48 hours before the meeting scheduled at the kickoff meeting to discuss it.
Task 2 Deliverable 2: Final research plan based on the draft research plan and the feedback at
the discussion meeting.
Task 3. Sampling Frame and Sampling Plan
The true population of HVAC maintenance contractors must be defined. More contractors provide HVAC maintenance services than participate in IOU energy efficiency programs, but this group is smaller than the total number of contractors with C-20 licenses, and may include companies without C-20 licenses. This intermediate population should be defined in terms of its size and characteristics. Based on the definition of the population, the Contractor should develop a sampling plan for the sub-tasks in Task 4, including stratification and allocation.
Task 3 Researchable Questions

What are the characteristics of the HVAC maintenance contractor population, including firm size and union membership, and what distinguishes members of this group from the broader population of C-20 contractors?

How large is the population?

What are the characteristics of the technician population that forms the labor force for the contractors?

What is the optimal stratification of the HVAC maintenance population for sampling?

What are the desirable properties of the target sample (e.g. precision, confidence, power to detect differences and trends, non-statistical properties); what is the target sample and how will that be allocated to the optimal stratification?

How should the samples for sub-tasks 4a and 4b be related, if at all, to ensure the validity of the findings while also being able to compare results from surveys, focus groups and field work?

How stable is the population frame? To what extent do firms and individuals enter or leave this population? How long should we assume the current assessment of the population frame is valid?
Task 3 Deliverable 1: Memo defining the population for the sampling frame and providing a sampling plan. This memo will also address sample recruitment challenges and how they will be overcome.
Task 4. Behavior Research
The behavior research will address how technicians in the residential and non-residential markets conduct diagnostics and remediation, how contractors’ structure their businesses, and how market actors make decisions about maintenance. Bidders must suggest in their proposals an appropriate mix of surveys and focus groups (which may include no focus groups), how

participants will be recruited for a representative sample, and how tasks 4a and 4b should be coordinated.

Task 4a . Technician and contractor surveys and focus groups (E-2, 66, 70)
The goal of this research will be to understand the HVAC contractor business model and how energy efficiency programs can effectively interface with that business model to improve provision of high-quality maintenance. Beyond the goals and constraints of contractors and technicians, it will investigate how these goals and constraints affect the ways in which technicians understand and communicate the value proposition to the customer regarding maintenance activities, and what are the stated or implicit policies of how technicians are to conduct maintenance work.
Based on the results, the research plan may need to be revised for task 4b.

Task 4b . Study of field maintenance activities (66)
This element will look at how diagnostics and remediation are actually accomplished by field technicians, including variation between contractors and differences between stated or implicit guidelines and actual activity.
Task 4 Researchable Questions

What practices are implemented in the field to conduct and sell HVAC maintenance?

How do those practices differ from stated or implicit policies and guidelines?

What differences exist between program participating contractors and non-participant contractors?

What differences exist between certified technicians and uncertified technicians in terms of skills, practices, and prices/fees?

What are the differences between contractors whose focus is non-residential versus those whose focus is residential?

What level of interest/involvement do contractors have in training/certification on fault detection and remediation?

Why do non-participant contractors, including participants in previous programs, not participate in the programs? Why do participants participate?

What are the predominant (general) business models of contractors and what are the differences between groups of contractors?

How do contractors structure their prices for maintenance services?

What are the primary barriers to contractors to implementing high quality HVAC maintenance services?
 What is a technician’s standard practice for bidding various maintenance approaches such as low cost, added value, or guaranteed performance contracts? Do technicians have certain questions they ask customers to determine how to bid these various maintenance contracts?

How do technicians communicate and bid a value proposition that addresses air quality, comfort, and energy savings to customers?

What are the primary barriers to technicians to implementing high quality HVAC maintenance services?
Task 4a Deliverable 1: Survey instruments and focus group plans prior to implementing the surveys and focus groups.
Task 4a Deliverable 2: Memo describing the results of the surveys and focus groups.

Task 4a Deliverable 3 (if needed): Revised research plan.
Task 4b Deliverable 1: Memo describing the results of the field study.
Task 5. Prepare Reports
After the completion of the Task 4 research, the Contractor will develop a draft report on the research methodologies and findings, which may incorporate the summaries delivered in Task 4.
This draft will be reviewed by members of the Project Team and edited based on input. The
Contractor will provide a public presentation of the draft findings and will allow external stakeholders to review and comment further. Based on these comments, a final draft will be developed.
Task 5 Deliverable 1: First draft report for internal review including methodologies and findings for all research and analysis tasks.
Task 5 Deliverable 2: Second draft report for external review. Public presentation of draft findings.
Task 5 Deliverable 3: Final report.
Proposed Budget
$200,000 drawn from the budget set aside for the Phase II HVAC Maintenance Study.
Rough Timeline

Bidding and procurement process August/September 2011

Kick-off and research plan September/October 2011

Sampling frame development November/December 2011

Surveys/Focus Groups January/February 2012

Field observations March/April 2012

Draft Report April 2012

Final Report May 2012