Use of KPIs in an Integrated Decision Support
System (IDSS) for Energy Efficient District
Retrofitting
ECODISTR-ICT H2020 (FP7)Project
Esra Bektas and Bart Luiten (TNO)
Structure
Section 1:
Background –the necessity for the IDSS and use of KPIs
Section 2:
Modelling the Decision Process that the IDSS supports
Section 3:
Inclusion of KPIs in the IDSS supported decision process
1. BACKGROUND
Background: Complex Decision Process in
Energy Efficient Urban Retrofitting
Multi-stakeholder, with a wide range of stakes and priorities
Is boosted due to incorporation of both selecting and using different
sustainability measures dealing with each stakeholder’s vision,
financial capacity, and different performance expectations.
Multiple decision levels
Multiple time horizons
Multiple KPIs
++++++
Not only rational decisions but intuitive
Dynamic process
ECODISTR-ICT project
ECODISTR-ICT aims to enable local authorities, architects and
urban planners planning in making decisions on retrofitting of
districts through a new open source software tool.
Called the Integrated Decision Support System (IDSS)
• The IDDS refers to a ‘software backbone’ that integrates
existing
design
and
calculation
modules
rather
than
developing a new ones.
• The IDSS facilitates decision making through credible input
for selection and incorporation of measures in design phase.
The IDDS Components
a) Graphical User Interface (GUI), b) the Dashboard, and c) The
Framework.
Critical Observations
When such complex, dynamic, iterative –sometimes intuitive process
is mediated by computers. There is a probability
to end up with a rigid and prescriptive decision model, which does
not correspond to the stakeholders’ practices.
If/when included, to focus on KPIs scores rather than how can be
instrumented for the stakeholders in making choices.
--or– a clear decision model but the way missing the way that KPIs are
used.
The Need regarding the KPIs in the IDDS
There is a need to define decision model that the IDSS can support
yet to be dynamic, iterative, fit-to-the purpose.
There is a need to identify the use of KPIs that stakeholders require
and the way to support it through the IDSS.
There is a need to define the support via the integrated environment,
which synthesizes the calculation modules and assessment modules
for selected KPIs.
2. The Decision Model for
the IDDS
Decision Process: Why to model and plan?
Having higher chance to establish clear goals and ambition,
made a conscious and directed series of choices.
Providing a standard of measurement,
to direct whether decision maker go towards or further away from
the ambition.
Enabling to convert values to action iteratively
to advance the plan best
Clarifying the boundaries
To manage the commitments
Carpenter et al. (2009) Steinitz (2013) Kepner et al. (1965)
Decision Theories
Decision steps defined by Carpenter et al. Design steps in geo design context of
Steinitz (2013)
(2009)
The Decision Process for the IDSS
ECODISTR-ICT Decision Process
Phase A: Defining Ambition
Example:
High level ambition : Flourished cities…
District to be flourished!!
Analyse
problem
Collect data
• Energy consumption is too high.
Data collection
modules
• Buildings are deteriorated.
• Housing typology is not suitable anymore to the families.
• Maintenance is (poor and) expensive.
• Vacant buildings are too many.
• There is insufficient greenery.
• There is insufficient facilities for inhabitants.
• Crime rate is too high.
<District to be flourished>
Identify
stakeholders
Problem statement
Selected KPIs
Analyse
problem
Identify &
Select KPIs
Collect data
• Energy consumption is too high. [City]
• Buildings are deteriorated.
[Investors]
Data collection
modules
• Housing typology is not suitable anymore to the
families.
[Investors/Citizens]
• Maintenance is (poor and) expensive.
[Housing Associations]
• Vacant buildings are too many. [City]
• There is insufficient greenery.
[Citizens]
• There is insufficient facilities for inhabitants.
[Citizens, Housing Associations]
• Crime rate is too high.[Citizens]
AS-IS data to
enable
assessment
Assess ASIS
 Energy-efficiency neutrality
 Quality of buildings
 Comfort
 Flexibility
 Easy-Maintenance
 Maintenance cost
 Attractiveness
 Greenery
 Sufficiency of facilities
 Safety
Assess AS-IS
Districts
scored XX in
aspect
Assessment
modules
KPI’s + AS-IS score
Define
TO-BE
ambition
Gap between TOBE and AS-IS KPI’s
+ values
District flourishing; [Input to Program]
- Create small-businesses and cultivating economic activities
- Create attractiveness to the vacant office stock there
- Improve housing stock’s quality (%)
- …
- …
- …
ECODISTR-ICT Decision Process
Phase B: Selecting Best Alternative
3. Inclusion of KPIs in the
IDSS supported decision
process
user manually do elsewhere
user via the IDSS
ICT-ICT automatically(dahboard-framework &framework-module connections
The legend to read the IDSS supported actions in the decision process
KPI Inclusion in Defining Ambition Phase
Basic Step
A1
Analyse problem
Define the
problem & the
users
A2
Collect data
A3
Assess AS-IS
Visualization and KPI’s ASIS
Calculation module
Select &
Manage KPIs
Data colection
prior to assesment
2. Define and
select KPIs by
a)-selecting
certified full KPI
sets,
b) choosing KPIs
from sets,
c) adding new
KPIs
1. Create contexts
Run calculation module
for assesing AS-IS scores
2. Inform which
data (in which
level of detail) is
needed for
calculation
Dashboard connects
automatically to the
calculation modules
registered to the IDSS
3. Enable
uploading the data
for calculation
-or-Run calculation
module outside of the
IDSS and add values for
acquiring scores of
KPIs, when the
calculation modules
are not coupled to the
IDSS.
A4
Define TO-BE
Ambition
Define the gap between
AS-IS
and TO-BE Ambition
Actions
1. Capture the
users /the
stakeholders
present;
define
geographical
area; define
the context of
the problem
3. Management
of the content
of KPIs
Definition of
scores,
prioritization
of KPIs,
selection of
calculation
modules.
++ Check the
consistency/
sufficiency of the
data uploaded
Show alert about the
missing data
through modules
1.Visualize AS-IS scores as
reference point prior to KPI
target values
2. Define TO-BE ambition
KPI Inclusion in Defining Ambition Phase
Basic Step
Actions
B1
Develop
alternatives
B2
Asses alternatives
B3
Select best
alternative
Designing modules in
fixed context
Calculation modules
with different context
Best alternative and
the chosen context
1. Inform the user about
the list of measures in
relation with the selected
KPIs
1. Run calculation
Modules for each
selected variant
Providing a
‘scoreboard’ as a
summary that the
final decision can be
made.
2. Administer alternatives;
choose /modify contexts
(if necessary), and make
variants
3. Upload data/design
models
2. Visualize the
scores of AS-IS,
TO-BE Ambition
and the scores,
the context and
the alternative
used in each
variant.
--When possible-4. Access to design
Modules
Sense making
between KPI scores
and target values
and decide
Conclusions & Discussions on
Does the modelled and ‘rationalized’ decision process seem
recognizable?
How can we ensure the balance should be in proposing rationalized
decision process and ‘irrationality’ of the decision makers which is
natural and can play great role in making choices?
How can we improve our decision model and make applicable for
other contexts?
THANK YOU.
Example:
High level ambition : Flourished cities…
District to be flourished!!
Analyse
problem
Collect data
• Energy consumption is too high.
• Buildings are deteriorated.
• Housing typology is not suitable anymore to the families.
• Maintenance is (poor and) expensive.
• Vacant buildings are too many.
• There is insufficient greenery.
• There is insufficient facilities for inhabitants.
• Crime rate is too high.
<District to be flourished>
Problem statement
Selected KPIs
Analyse
problem
Collect data
• Energy consumption is too high. [City]
• Buildings are deteriorated.
[Investors]
• Housing typology is not suitable anymore to the
families.
[Investors/Citizens]
• Maintenance is (poor and) expensive.
[Housing Associations]
• Vacant buildings are too many. [City]
• There is insufficient greenery.
[Citizens]
• There is insufficient facilities for inhabitants.
[Citizens, Housing Associations]
• Crime rate is too high.[Citizens]
Identify
stakeholders
Identify &
Select KPIs
Assess ASIS
 Energy-efficiency neutrality
 Quality of buildings
 Comfort
 Flexibility
 Easy-Maintenance
 Maintenance cost
 Attractiveness
 Greenery
 Sufficiency of facilities
 Safety
Inclusion of KPIs in Analyzing the Problem
1. Capture the users of the IDSS, the stakeholders present; define
geographical area; define the context of the problem,
User makes a sub-set of KPIs
depending on the problem &
high-level ambition.
2. Define and select KPIs by a)-selecting certified full KPI sets, b)
choosing KPIs from sets, c) adding new KPIs.
3. Management of the content of KPIs, definition of scores,
prioritization of KPIs, selection of calculation Modules.
a) Choose a certified full KPI set (i.e., from
BREEAM, Open House, and Super
Buildings);
b) Select KPIs from existing sets;
c) Adding new KPIs (when existing KPIs do not
satisfy the specific needs of the stakeholders
for the specific case).
Inclusion of KPIs in Collecting Data
1. Inform the user about which data (in which level of detail) is needed for
the assessing AS-IS,
User explores what data
is needed for the
selected KPIs
2. Create contexts
3. Enable the user to upload the data that will be used for the calculation
and check the consistency/sufficiency of the data uploaded.
Inclusion of KPIs in Assessing AS-IS situation
1. Run calculation Module via Dashboard for assessing AS-IS,
2. Run calculate elsewhere (i.e. separate Module) and manually add
User runs calculation
Modules and gathers
scores for each KPI
Figure 1
values to KPIs which are created/defined in the IDSS but not either
automatically or directly calculated via the IDSS.
The inclusion of KPIs in Assessing AS-IS (step A3)
Inclusion of KPIs in Defining TO-BE Ambition
1. Visualize AS-IS scores as reference point prior to KPI target values
2.(Re) Prioritize and define target values of KPI's based on AS-IS scores and
boundary conditions of the stakeholders.
User re-evaluates
priorities, and, if
necessary, re-do Phase A
of the decision process.
Figure 1
--When necessary-Re-select KPIs/ KPI Modules, run calculation and get new results, then
(re)define TO-BE ambition .
Prioritizing and define target values for KPIs (step A4)
Inclusion of KPIs in Developing Alternatives
1. Inform the user about the list of measures in relation with the
selected KPIs
User develops alternative
2. Administer alternatives; choose /modify contexts (if necessary),
solutions to meet the TO-BE
and make variants
ambition
3. Upload data/design models
--When possible-4. Access to design Modules
Figure 1
Developing alternatives (step B1)
Inclusion of KPIs in Assessing Alternatives
1. Run calculation Modules for each selected variant
User asses design
alternatives and make
2. Visualize the scores of AS-IS, TO-BE Ambition and the scores, the
context and the alternative used in each variant.
adjustment.
Figure 1
The alternatives are assessed against the AS-IS and TO-BE situation (step B2)
Inclusion of KPIs in Selecting the Best Alternative
1. Providing a ‘scoreboard’ as a summary that the final
User evaluates the alternatives
decision can be made.
by the score of the KPIs.
Figure 1
The best alternative is selected (step B3)
Decision Levels