Chapter 3 Strengths and weaknesses
3.2 Consumer Surveys
Advantages
1. Comprehensive information on all fuels used in private households
2. Best achievable data quality if they are well prepared and combined with a
comprehensive data validation process
3. Can be used directly and as input for model calculations
Disadvantages
1. Resource intensive
2. Expensive
3. Time consuming
4. High respondent burden
In opposition to supplier surveys – mostly a census - consumer surveys normally are conducted as sample surveys. Depending on the maximum statistical error allowed the range
of the sample size normally starts with a minimum 0.1% and ends with a maximum of 1% of
the overall population. It should be determined by the methodology department of the National Statistical Institutions to meet the statistical requirements for the appropriate statistical error.
Different methods like stratified clustering or establishing a panel can help to reduce sample size and to save resources.
We think that this should be organized with the collaboration/assistance of organisms specialized in the field of statistics, like INE (Institute National of Statistics) in Spain.
Panel surveys are a very good instrument to observe developments continuously and
with relatively low costs if enough volunteers join the action
It is true what was said previously on panel surveys, but we should mark that it can also generate biased information. Usually the lower classes of society do not participate in these
panels.
Mostly the observation units are households, but some restrictions like households on
their main residences or completely different units like dwellings may be chosen. The
respondents are members of the respective household or inhabitants of the dwelling.
In any case, we believe that necessary to define in detail what is meant by households and
dwelings.
General household energy consumption surveys should include at least the frequently used
energy carriers in households broken down by the main consumption purposes that are
space heating and cooling, water heating and cooking. Detailed consumption pattern of
electricity for lightning and electric appliances are difficult to obtain from such a survey. If
this information is needed it makes more sense to conduct a separated electricity consumption survey.
A way to get the consumed electrical detailed is through measurements of consumption "in
situ" which will be discussed in section 3.5. I think that we should make here that they are
costly operations and that aims to provide data to the modeling of the electric consumption.
The survey can be conducted as simple postal survey, but due to the fact that energy consumption is complex to ask and the often missing knowledge of the respondents an interview either personally or by telephone is the more adequate survey form. Computer assistance in any case improves data quality. Therefore it should be applied if possible.
In the consumer panel that we have used in the project SECH has used another method
based on an electronic instrument that sends the answers in a questionnaire previously
loaded into the instrument by telephone. If you want we can send you the method documentation.
In case the survey is conducted on a voluntary base it makes sense to do it together with an
obligatory survey (e.g. the labour force survey). This increases the response rate significantly.
The questionnaire should be as simple and as short as possible. An appropriate questionnaire design influences the response rate positively. If using a questionnaire for the first
time, or important changes are introduced to the existing questionnaire, it is useful to conduct
a test survey on a small sample (100-200 households), a few months before the main survey. The test survey points out the questions which cause major problems to the respondents and allows adjusting these questions, thus improving the questionnaire for the main
survey.
It is recommended to send a letter of approval to the chosen households before the survey
starts. With this letter the respondents are informed about the purpose and the content of the
survey, as well as the data they will be asked for. The respondents can also be asked to
prepare the data on energy consumption (quantities and expenses), electrical appliances
(energy classes), light bulbs and cars (fuel consumption, distance travelled). The notification
letter could be accompanied by examples of energy bills, to assist the households providing the requested data.
In case of interviews the interviewers should have a good knowledge on the different fuels.
Furthermore, the interviewers should be well trained on consumption patterns of different
types of fuel burning appliances and household purposes (e.g. space heating, water heating
and cooking). Practical leaflets with region-specific examples about electricity, natural gas
and district heat bills for private households, and training in reading the bills could enable the
interviewers to assist the respondents.
In the case of the SECH project to Spain this point has been basic, the questionnaire provided information on how to read invoices from different suppliers, for knowledge of the respondent and telephone support to help the respondent.
If it is manageable quantities and expenditures should be asked to calculate prices. These
prices are valuable information for data validation by comparing them with average prices. If
not known from other sources the floor area of the dwelling and the number and age of
inhabitants or household members should be asked in any case.
Additional information like insulation measures or the use of thermal setting points can
improve the understanding of consumption pattern better. Therefore it is highly recommended to include such questions into the survey.
We believe that information about insulation does not usually known by the families. This
information should be determined from the year of construction of housing and the existing
building at that time code.
The thermal setting points can be used to determine the level of the indoor temperature
which is linked to the energy consumption in heating, but this should be accompanied by
information on geographical area and heating degree days of the location of the households
If the results of the survey are planned to be used for reporting obligations of the RESdirective, a special focus has to be given on renewables. Detailed information how reliable
data in this difficult field can be achieved are given in chapter 6.1.
Our experience in the Spanish study is that the penetration of these energy sources is low
and is therefore best to perform statistical operations specialized on them. For this reason,
we in the SECH project carry out data matching between operations specific to RES statistics and surveys of the SECH project.
Although fuel consumption of privately owned cars is not attributed to households but to
road transport, additional questions on fuel type, average consumption, annually driven km
and age of the cars make sense. Normally households know this much better than dwelling
bound energy consumption and this information is very useful for compiling energy accounts
end energy efficiency indicators for transport.
Although the information on cars is very interesting, it may happen that the interviewees can
be oversaturated with too many questions. It’s necessary to have in mind that the maximum
time to answer the questionnaire should not exceed 20 minutes.
The survey frequency should not exceed 5 years to provide information on developments
in time. The extrapolation of the results until the next surveys and the follow up revision of
the period between the last two survey cycles can be done with heating and cooling degree
days and trend analyses. Details how doing this will be given in chapter 3.4.
Because of the high complexity of energy consumption survey, normally the data collected cannot be used directly, but have to be checked and validated very carefully. One
very important aspect is the fact, that in case of storable fuels like fuel oil, gasoil, coal or
fuel wood normally the purchased and not the consumed quantities are known and reported. Detailed description of possible data validation procedures which are mainly model
based crosschecks one can find in chapter 3.4.
We would add that checking and validation were done with different sources, if possible official.
Another aspect that can be improved with additional information is the grossing up procedure. Here e.g. the number of gas and electricity meters attributed to households - information that should be available from the suppliers – can be used to achieve more realistic
figures.
Conclusions and summery
The 4 main elements to achieve good results are a careful preparation, a simple questionnaire, well trained interviewers and comprehensive data validation.
Grossing up procedures can be improved by using supplier information, e.g. gas meters
attributed to households.
Model based extrapolation e.g. heating degree days help decrease survey frequency.
Other alternatives can be applied such as an adjustment on the trends according to the evolution of representative variables of the energy consumption and purchasing power of the
families and households.