Examples of success criteria, indicators and baselines for different types of projects
Contents
1.
2.
3.
4.
5.
6.
Energy audits
Energy advice
Financial support and services
Information and education campaigns
Metering and feedback
Voluntary programmes and commitments
Notes concerning the tables
Your project may have other success criteria than those exemplified here. Examples of other success criteria can include particular social goals like reduction
of fuel poverty or stimulation of employment in a certain region. Please take these examples as indicative only. For further reading and more indicators, we
recommend the following references and sites:
 IEA DSM Evaluation Guidebook, Volume 1 (http://www.ieadsm.org/)
 Changing Energy Behaviour: Guidelines for Behavioural Change Programmes (http://www.energy-behave.net/pdf/Guidelines_Changing_Energy_Behaviour.pdf)
 AID-EEE Guidelines for the monitoring, evaluation and design of energy efficiency policies (http:// http://www.aid-ee.org/)
It may be difficult to define baselines for many of the indicators, and all projects cannot afford to conduct baseline studies. Depending on your resources and
the expectations of your funding bodies, you may consider using ‘proxy data’ for baselines like:
• national and regional statistics
• existing general knowledge, attitude and behaviour surveys in your country
• expert interviews on the current status at the start of the project
• survey with small, random sample of target group
• available default data from existing CO2 tools or existing data on the specific consumption of various appliances
1. Energy audits: examples of success criteria, indicators, baselines and ways to measure
Possible success criteria
Possible indicators
Possible baselines
How to measure
Increased customer awareness
of energy use
Number of audits conducted (or share of audited
facilities in a sector)
Depends on target group: maybe
‘0’, maybe people already had one:
find out via survey?
Counts and records of audits
Increased customer awareness
of energy use
Attention to and use of audits/ Number of investments Depends on target group: maybe
made
‘0’, maybe people already had one:
find out by means of survey? (to test
for additionality, find out plans of
investments/changes before the
project)
Number of changes
implemented, Number /
amount of investments made
Savings potential – what measures have been
identified and the savings in energy and
corresponding financial costs and savings
Theoretical savings potentials of implemented
measures – what measures have been undertaken
and estimate of energy savings thus achieved
Estimation of savings during
planning phase of project and
evaluation of savings after
implementation of measures
Reduced energy consumption
Energy consumption before audit
(Default data from statistics)
Amount of energy use
Measured savings at the site level – shows whether
implemented measures actually reduce the energy
consumption as expected
Verified results – usually a statistical study of results
of implemented measures
Customers & stakeholders:
overall satisfaction with the
programme
Number of new energy management practices
adopted (self-reported behaviour)
Existing level of investments,
practices or energy consumption
Survey, measurements of
energy consumption
Customer & stakeholder survey: overall satisfaction
(project, process, outcomes)
Customer & stakeholder
expectations
Survey including customer
ratings (and qualitative data)
2. Energy advice: examples of success criteria, indicators, baselines and ways to measure
Possible success criteria
Possible indicators
Possible baselines
How to measure
Reaching a large number of
people
Number of advice contacts, number of people
attending events
Available advice at the start of the
project
Keeping count and records of
customer contacts
Mapping problems and offering
suitable solutions
Relevance to recipients/participants
Status of know-how /information of
participants
(Satisfaction) survey using
quantitative and qualitative
data
Promotion of energy efficiency
investments and improved
energy management practices
Share of new investments generated as a result of
the advice1 among those receiving advice
Level of f energy investments at the
start of the project
Records of number and
amount of investments
Share of advice recipients reporting new practices
Status of recipients’ practices at the
start of the project
Survey, self-reported
behaviour
Increased customer awareness
and competency, capacity to
advise others
Knowledge tests
Self-reports of new practices (also changes in
physical environment that allow new practices)
Baseline customer survey:
Customer surveys/interviews
knowledge and practices at the start comparing knowledge and
of the project
practices before/after project
Customers: lower energy bills,
improved comfort, convenience
Customer satisfaction with the outcomes of the
project
Customer expectations
Customer survey
Customers & stakeholders:
overall satisfaction with the
programme
Overall satisfaction (project, process, outcomes)
Customer & stakeholder
expectations
Survey
1
Especially for larger investments, it is not likely that all investments made within the target group are directly due to the advice. People considering an investment are more
likely to seek advice. Thus, it is necessary to find out how many of the customers would have made the investment even without the advice.
3. Financial support and services: examples of success criteria, indicators, baselines and ways to measure
Possible success criteria
Possible indicators
Possible baselines
How to measure
Promotion of energy efficiency
investments
Number or share of customers making Number at start of initiative
use of support/service
Level of investments at the start of the
initiative
Number/value of investments made
Customer contacts and
records
Energy savings from the measures
applied
Energy consumption compared to
baseline
Figures on the total consumption of electricity
or heat / figures on specific consumption of
appliances
kWh consumption, gas/oil
consumption
Audit/analysis
Promotion of low-carbon solutions:
e.g. appliances, technology, systems
of provisions, behavior options
Increase in the adoption/market share
of low-carbon solutions
Adoption/market share of low-carbon solutions Sales statistics, market share
at the start of the project
statistics?
CO2 profile
Carbon footprint of
households or enterprises
Cost-effectiveness
Money and other resources
used/energy or CO2 emissions saved
Comparison to other projects/programmes
with the same goal
Net Present Value of
programme impacts i.e., a
sum of the benefits of the
programme during its effective
period divided by the costs of
the programme plus the cost
of capital (interest rate)
Customers: improved comfort,
convenience
Experience of improved comfort,
convenience
Baseline assessment (pre-project survey) of
current feeling of comfort and convenience
Survey /questionnaire on
qualitative data
Customers: lower energy bills,
kWh used per month/per year,
Monthly energy (or yearly average) of energy
used before start of project, statistics/old bills
before measures taken
Check bills, but take into
account: free rider or
‘deadweight’ effects, rebound
effect
Customers & stakeholders: overall
satisfaction with the project
Experience overall of satisfaction
(project, process, outcomes)
Customer & stakeholder expectations
Customer & stakeholder
survey
4. Information and educations campaigns: examples of success criteria, indicators, baselines and ways to measure
Possible success criteria
Possible indicators
Possible baselines
How to measure
Reaching a particular audience with
the intended message
Products delivered by the campaign,
media & partners involved, number of
people exposed to the message
Amount and quality of information existing
Number of newspaper/online
items, number of readers,
clicks on webpage
Pre-project survey on the level and quality of
consumer’s knowledge and skills
Survey, self-report on
awareness, attitude;
information
Pre-project sales statistics on energy efficient
products, interviews with salespersons
Sales statistics/market share
information/ information from
appliance shops
Pre-project survey
Survey, self-reports e.g. via
energy diary
Changes in awareness, knowledge,
Changes in awareness, knowledge
attitudes and internalised norms (i.e., attitudes
what people feel they should do)
Increased information search on
energy-efficient products
Awareness of energy efficient products
and changes in future decision-making
processes
Changes in behaviour
Self-reported behaviour
5. Metering and feedback: examples of success criteria, indicators, baselines and ways to measure
Possible success criteria
Possible indicators
Possible baselines
How to measure
Increased customer awareness of
energy use
Attention to and use of feedback –
self-report from survey
Pre-project survey: check of current
awareness and use of feedback (if any)
Comparison of awareness
before/after project on the
basis of surveys
Reduced energy use
Changes in energy use after feedback
Energy use before provision of feedback
kWh/gas/oil consumption
Customer expectations
Customer/stakeholder survey
Overall satisfaction with the project
Customer survey: overall satisfaction
(e.g. the extent to which it is
(project, process, outcomes)
perceived of as useful and interesting
by the recipients)
6. Voluntary programmes and commitments: examples of success criteria, indicators, baselines and ways to measure
Possible success criteria
Possible indicators
Possible baselines
How to measure
Getting energy users ‘on board’
Number of participants
Number before project
Participant count
Supporting change
Number of participants receiving
training etc.
Number before project
Participant count
Improvements made
Number of improvements made
Number before project
Counting and keeping records
of improvements made
Energy consumption (CO2 emissions) when
joining the project
kWh/gas /oil consumption –
resulting emissions
Default data form statistics/calculations
Carbon footprint of household
/enterprise
Customer & stakeholder expectations
Customer & stakeholder
survey on willingness to
participate again / to realize
the recommendations
Energy conservation, CO2 reductions Changes in energy consumption,
in target group
calculated CO2 emissions
Customers & stakeholders: overall
satisfaction with the programme
Overall satisfaction (project, process,
outcomes)