http://mobi.vub.ac.be
NISTO: A new evaluation framework
for mobility projects
NISTO
Jeroen Bulckaen
PhD Research Associate
NEW INTEGRATED SMART
TRANSPORT OPTIONS
Source: http://simonkneebone.com/2013/02/01/evaluation-tool/
Source: Evaluation matters. A practitioner’s guide to sound evaluation for urban mobility measures. Waxmann Verlag, Munster 2013.
4 main input channels
State of the art
analysis
Stakeholder
survey
Stakeholder
workshops
NISTO
evaluation
framework
Pilot and
Demonstration
projects
2 main elements of the framework
MCA
Evaluation
tools
MAMCA
NISTO
Target monitoring
Indicator 1 (Core)
Core criteria
Indicator 2
(Optional)
Criteria and
indicator
framework
Indicator 3 (Core)
Local criteria
Indicator 4
(Optional)
Assessment of project sustainability
MCA
Evaluation
tools
MAMCA
NISTO
Target monitoring
Indicator 1 (Core)
Core criteria
Indicator 2
(Optional)
Criteria and
indicator
framework
Indicator 3 (Core)
Local criteria
Indicator 4
(Optional)
Assessment of project sustainabilility
Define evaluation
criteria
Identify
indicators
Weight
the
criteria
Collect &
measure
data
Score the
criteria
Calculate
overall
scores
Define evaluation
criteria
Identify
indicators
Weight
the
criteria
Collect &
measure
data
Score the
criteria
Calculate
overall
scores
16 NISTO core criteria
SUSTAINABILITY
ECONOMY
ENVIRONMENT
SOCIETY
Economic activity
Land consumption
Cost efficiency
Greenhouse gas emissions Security
Business productivity
Air quality
Public funding of transport Resource use
Noise
Safety
Health
Liveability
Equity
Socio-political acceptance
Accessibility for people with
special needs
Define evaluation
criteria
Identify
indicators
Weight
the
criteria
Collect &
measure
data
Score the
criteria
Calculate
overall
scores
20 core & 15 optional indicators
11
Define evaluation
criteria
Identify
indicators
Weight
the
criteria
Collect &
measure
data
Score the
criteria
Calculate
overall
scores
20 core & 15 optional indicators
ECONOMY
ENVIRONMENT
SOCIETY
TRANSPORT
Cost-efficiency
- Investment costs
- Operating costs
- Revenues
Noise
- Perception
- Produced noise
- Exposure
Safety
- No. of accidents
- Perception
Modal split:
Passenger kms
12
Define evaluation
criteria
Identify
indicators
Weight
the
criteria
Collect &
measure
data
Score the
criteria
Calculate
overall
scores
Distribution of 100 points
for the criteria under each pillar
SUSTAINABILITY
ECONOMY
ENVIRONMENT
SOCIETY
Cost efficiency (0,23)
Noise (0,16)
Safety (0,24)
Define evaluation
criteria
Identify
indicators
Weight
the
criteria
Collect &
measure
data
Score the
criteria
Aid is provided by the set-up of a
Data Collection Plan
Calculate
overall
scores
Define evaluation
criteria
Identify
indicators
Weight
the
criteria
Collect &
measure
data
Score the
criteria
online TOOLKIT
November 2015
http://www.nisto-project.eu
Calculate
overall
scores
16
17
18
Source: Evaluation matters. A practitioner’s guide to sound evaluation for urban mobility measures. Waxmann Verlag, Munster 2013.
http://mobi.vub.ac.be
Jeroen Bulckaen
Vrije Universiteit Brussel
Jeroen.Bulckaen@vub.ac.be
Dr. Imre Keseru
Vrije Universiteit Brussel
Imre.Keseru@vub.ac.be
Prof. dr. Cathy Macharis
www.nisto-project.eu
THANK YOU FOR YOUR
ATTENTION!
Vrije Universiteit Brussel
Cathy.Macharis@vub.ac.b
Links
NISTO Project
• http://www.nisto-project.eu
MOBI Research Group, Vrije Universiteit Brussel
• http://mobi.vub.ac.be
Department BUTO, Vrije Universiteit Brussel
• http://research.vub.ac.be/buto
20
Criteria weights
MOSI - Transport & Logistics
21
The NISTO indicator framework
22
MOSI - Transport & Logistics
23
1
Project alternatives
2
3
4
Input data
Criteria
Stakeholders
Project
type
Indicators
Targets
NISTO Evaluation tools
MCA
MAMCA
Target Monitoring
Output
Stakeholders’
performance scores
Target achievement
MOSI - Transport & Logistics
Sustainability scores
24
Multi-Actor Multi-Criteria Analysis
Stakeholder
Stakeholder analysis
analysis
2
2
1
1
StakeStakeholder
holder11
StakeStakeholder
holderm
m
C
C
11
C
11
C
n1
n1
C
Cnm
C
C
nm
C
C11
11
C
C
11
W
11
W
n1
n1
C
Cnm
W
W
nm
C
Cnm
nm
Alternatives
Alternatives
3
3
5
5
Overall analyses
(MCA)
Ref.
Ref.
Altern
Altern
W
W11
11
C
C11
11
result
result
4
4
W
Wnm
nm
C
Cnm
nm
6
6
Indicators
Indicators
Results
+/0/+/0/result
result
result
result
ImplemenImplementation
tation
7
7
Mitigation
Mitigation
strategies
strategies
result
result
scenarios
scenarios
Measurement
Measurement
methods
methods
Deployment
Deployment
scenarios
scenarios
Macharis, 2000 & 2004
http://mobi.vub.ac.be