Statistical evaluation of model
uncertainties in Copert III,
by
I. Kioutsioukis & S. Tarantola
(JRC, I)
Objectives of the statistical analyses
Sensitivity analysis tests performed using COPERT III
Interpretation of the results
Objectives
Precision of emission estimates depends on the assumptions made in
the definition of the various model input parameters.
Need to check robustness of emission estimates to
poorly known parameters and model assumptions.
Reflect our poor knowledge on input parameters by means
probability distributions and apply Monte Carlo analysis
to estimate probability distributions of emissions.
Representation of a
Monte Carlo simulation
Objectives
Uncertainty should always accompany an estimate,
as it is a measure of the quality of the estimate.
Representation of the Monte Carlo simulation
Objectives
Estimates (and related uncertainties) can then be used
1. to adopt traffic policy measures
2. for inventory systems
3. as input to air quality models
Objective is to apply up-to-date sensitivity analysis to identify the
parameters mainly responsible for uncertainty in the emissions
Help us improving the quality of emission estimates
if we direct efforts to improve our knowledge of
the important parameters
Statistical analyses
• Description of sources of uncertainty (input):
- Uncertainty in traffic parameters (how to model them)
- Uncertainty in average speed
- Uncertainty in emission factors
• Description of the set up of the analyses
• Results (Figures and Tables)
Model uncertainty in traffic parameters
All the categories of vehicles considered
Country-specific mileage data taken from MEET deliverable #22
FBM–INFRAS used for decomposition of fleet into sub-categories
Uncertainty in traffic parameters
τ: steers the technology stage percentages;
τ = –1, 0 and +1 represent fleet with 'low/medium/high'
amount of new technology vehicles, respectively.
δ: steers the diesel share of PC and LDV;
δ = –1, 0 and +1 represent fleet with 'low/medium/high'
amount of diesel vehicles, respectively.
σ : steers the size (weight class) distribution of HDV;
σ = –1, 0 and +1 represent fleet with 'low/medium/high'
amount of heavy-weight HDV's, respectively.
FBM (expensive) is only executed at selected points
σ
δ
τ
τ
We feed COPERT with a representative configuration of
fleet breakdown at each Monte Carlo run i.
We sample a point τi, δi, σi over the square and
interpolating the FBM runs we obtain the
configuration of fleet breakdown f (τi, δi, σi )
Uncertainty in average speed
Currently described with rather rough statistical distributions
Exploratory analyses have shown that average speed
is rather an important parameter.
Perform more refined analyses…
f(x)
More reliable pdf’s using Goodness of fit tests
based on driving cycles
0,140
0,120
0,100
0,080
0,060
0,040
0,020
0,000
40
60
80
100
velocità
media [km/h]
■ Average speed in rural road
● average
speed in motorway
120
Uncertainty in emission factors
Very low
regression
coefficients
ef reg
Not sufficient
EF  ef reg  S j N (0,1)
EF  ef
reg

1  si
s
i

1

S j  

Ni
Ni 1 
2
* R  S j N (0,1) * (1  R )
2
2
Uncertainty in load factors
Pdf=Normal; mean=50%, std = 10%
(questionnaire - expert opinion)
Uncertainty in meteo conditions
(statistical model - INFRAS)
Uncertainty in average trip length
Pdf=Log-Normal; mean=12Km, std=3Km
(questionnaire - expert opinion)
Variable Description
Units
Distribution
(,)
PPC
PLDV
PHDV
PUB
P2W
MPC
MLDV
MHDV
MUB
MPW
UPC
HPC
ULDV
HLDV
UHDV
HHDV
UUB
HUB
UPW
HPW
VU
VRPC
VHPC
VRLDV
VHLDV
VRHDV
VHHDV
VRUB
VHUB
Km
Km
Km
Km
Km
%
%
%
%
%
%
%
%
%
%
Km/h
Km/h
Km/h
Km/h
Km/h
Km/h
Km/h
Km/h
Km/h
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
(34799160, 347992)
(2142083, 21421)
(1293357, 12934)
(83851, 838)
(2000000, 20000)
(10059, 1006)
(17706, 1771)
(38741, 3874)
(41800, 4180)
(5000, 1000)
(35, 10)
(15, 2.5)
(40, 12)
(30, 4.5)
(30, 9)
(50, 7.5)
(75, 10)
(15, 2.25)
(30, 9)
(40, 6)
(20, 3)
(65, 9.75)
(100, 15)
(60, 9)
(90, 13.5)
(50, 7.5)
(80, 12)
(50, 7.5)
(85, 12.75)
population of PC
population of LDV
population of HDV
population of UB
population of 2-wheel vehicles
Annual mileage of PC
Annual mileage of LDV
Annual mileage of HDV
Annual mileage of UB
Annual mileage of PW
driving share (urban) of PC
driving share (highway) of PC
driving share (urban) of LDV
driving share (highway) of LDV
driving share (urban) of HDV
driving share (highway) of HDV
driving share (urban) of UB
driving share (highway) of UB
driving share (urban) of PW
driving share (highway) of PW
velocity profile (urban)
velocity profile (rural) of PC
velocity profile (highway) of PC
velocity profile (rural) of LDV
velocity profile (highway) of LDV
velocity profile (rural) of HDV
velocity profile (highway) of HDV
velocity profile (rural) of UB
velocity profile (highway) of UB
UHDV
HHDV
UUB
HUB
UPW
HPW
VU
VRPC
VHPC
VRLDV
VHLDV
VRHDV
VHHDV
VRUB
VHUB
VRPW
VHPW
Ltrip
LP
slope
A
H
D



eEF
driving share (urban) of HDV
driving share (highway) of HDV
driving share (urban) of UB
driving share (highway) of UB
driving share (urban) of PW
driving share (highway) of PW
velocity profile (urban)
velocity profile (rural) of PC
velocity profile (highway) of PC
velocity profile (rural) of LDV
velocity profile (highway) of LDV
velocity profile (rural) of HDV
velocity profile (highway) of HDV
velocity profile (rural) of UB
velocity profile (highway) of UB
velocity profile (rural) of PW
velocity profile (highway) of PW
Average trip length
load factor
slope category
lowest minimum temperature
highest minimum - lowest minimum temperature
highest maximum - (A+H) temperature
traffic parameter
traffic parameter
traffic parameter
amplitude Emission Factor
%
%
%
%
%
%
Km/h
Km/h
Km/h
Km/h
Km/h
Km/h
Km/h
Km/h
Km/h
Km/h
Km/h
Km
%
C
C
C
-
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Normal
Log-Normal
Normal
Normal
Normal
Normal
Normal
Uniform
Uniform
Uniform
Normal
(30, 9)
(50, 7.5)
(75, 10)
(15, 2.25)
(30, 9)
(40, 6)
(20, 3)
(65, 9.75)
(100, 15)
(60, 9)
(90, 13.5)
(50, 7.5)
(80, 12)
(50, 7.5)
(85, 12.75)
(65, 9.75)
(100, 15)
(12, 3)
(50, 10)
(0, 1)
(3.4, 0.35)
(14.9, 0.51)
(13, 0.48)
(-1, 1)
(-1, 1)
(-1, 1)
(0, 1)
Results: total emissions in Italy for years 2000 and 2010
first stage: screening analyses (Morris and Standardised Regression
Coefficients (SRC)) to identify the non-influential input parameters.
40 parameters  15 parameters
Identified 25 parameters that do not influence
the variability of the emission estimates (eg meteo variables)
Results of the screening technique – yr 2000
5
2
VOC
x 10
delta
1.8
1.6
1.4
SIGMA
1.2
Region of the
noninfluential
parameters
1
tau
0.8
0.6
VU
eEF
MPC
ltrip
0.4
0.2
0
UPC
VR126
UHDV
PC
VH126
MHDV
HPC
HHDV
ULDV
MLDV
VR34
VH34
H
UUB
dA
MPW
sigma
LP
UB
HLDV
HUB
HPW
LDV
UPW
PW
HDV
MUB
0
0.5
1
1.5
MU
2
2.5
3
5
x 10
Results of the screening technique – yr 2010
VOC
16000
eEF
14000
12000
VU
delta
SIGMA
10000
Region of the
noninfluential
parameters
8000
ltrip
6000
tau
4000
2000
0
VR126
UHDV
HHDV
VR34
VH126
VH34
A
MLDV
sigma
PC
H
ULDV
MPW
d
UPW
UUB
MUB
HPC
HDV
HLDV
HUB
LDV
LP
HPW
UB
PW
0
MPC
UPC
MHDV
1
2
3
MU
4
5
6
4
x 10
LAT data
Uncertainty analysis on 15 parameters
VOC Emissions - Italy
700,000
600,000
500,000
[t]
400,000
300,000
200,000
100,000
Passenger Cars
Light Duty Vehicles
Heavy Duty Vehicles
Buses
2020
2015
2010
2005
2000
1998
1997
1996
1995
1994
1993
1992
1991
1990
1989
1988
1987
1986
1985
1984
1983
1982
1981
0
Uncertainty analysis - 2010
180
VOC
160
120
100
80
LAT value
over-estimation of VOC:
probably l-trip is
overestimated
60
40
20
0
8.
1E
1. +04
0E
1. +05
3E
1. +05
5E
1. +05
7E
1. +05
9E
2. +05
1E
2. +05
4E
2. +05
6E
2. +05
8E
3. +05
0E
3. +05
2E
3. +05
5E
3. +05
7E
3. +05
9E
+0
5
Frequency
140
Annual Emissions (tonnes)
LAT data
NOx Emissions - Italy
1,000,000
900,000
800,000
700,000
[t]
600,000
500,000
400,000
300,000
200,000
100,000
Passenger Cars
Light Duty Vehicles
Heavy Duty Vehicles
Buses
2020
2015
2010
2005
2000
1998
1997
1996
1995
1994
1993
1992
1991
1990
1989
1988
1987
1986
1985
1984
1983
1982
1981
0
1E
2. +05
4E
2. +05
6E
2. +05
9E
3. +05
1E
3. +05
4E
3. +05
7E
3. +05
9E
4. +05
2E
4. +05
4E
4. +05
7E
5. +05
0E
5. +05
2E
5. +05
5E
5. +05
8E
+0
5
2.
Frequency
Uncertainty analysis - 2010
200
180
160
NOX
LAT value
140
120
100
80
60
40
20
0
Annual Emissions (tonnes)
LAT data
PM Emissions - Italy
50,000
45,000
40,000
35,000
[t]
30,000
25,000
20,000
15,000
10,000
5,000
Passenger Cars
Light Duty Vehicles
Heavy Duty Vehicles
Buses
2020
2015
2010
2005
2000
1998
1997
1996
1995
1994
1993
1992
1991
1990
1989
1988
1987
1986
1985
1984
1983
1982
1981
0
1E
1. +04
3E
1. +04
6E
1. +04
9E
2. +04
1E
2. +04
4E
2. +04
6E
2. +04
9E
3. +04
1E
3. +04
4E
3. +04
6E
3. +04
9E
4. +04
1E
4. +04
4E
4. +04
6E
+0
4
1.
Frequency
Uncertainty analysis - 2010
250
PM
200LAT
value
150
100
50
0
Annual Emissions (tonnes)
LAT data
CO2 Emissions - Italy
160,000,000
140,000,000
120,000,000
80,000,000
60,000,000
40,000,000
20,000,000
Passenger Cars
Light Duty Vehicles
Heavy Duty Vehicles
2020
2015
2010
2005
2000
1998
1997
1996
1995
1994
1993
1992
1991
1990
1989
1988
1987
1986
1985
1984
1983
1982
0
1981
[t]
100,000,000
Buses
1E
9. +07
8E
1. +07
1E
1. +08
1E
1. +08
2E
1. +08
2E
1. +08
3E
1. +08
4E
1. +08
4E
1. +08
5E
1. +08
6E
1. +08
6E
1. +08
7E
1. +08
8E
1. +08
8E
+0
8
9.
Frequency
Uncertainty analysis - 2010
200
180
LAT value
Annual Emissions (tonnes)
CO2
160
140
120
100
80
60
40
20
0
Summary of Uncertainty Analysis
METHOD
Morris
FAST
No RUNS
350
5499
YEAR
2000
2010
2000
2010
Mean
VC (%)
Mean
VC (%)
Mean
VC (%)
Mean
VC (%)
VOC
639
21
213
13
641
31
213
22
NOX
740
11
377
12
740
13
379
15
PM
39
22
21
21
40
25
21
26
CO2
117,621
7
136,406
6
113,852
9
133,350
9
second phase: quantitative sensitivity analysis technique
(extended-FAST) to apportion variance of emission estimates
back to input parameters.
2000
2010
VOC
VOC
1-SUM
1-SUM
delta
ltrip
MPC
VU
VH34
ltrip
MPC
eEF
eEF
delta
68% of VOC variance
explained by the topthree parameters
increase of ltrip and decrease of 
VU becomes important in 2010
The differences with the run conducted for 2000 are in the vehicle
Populations, fleet breakdown and in the use of new fuel.
2000
2010
PM
PM
1-SUM
1-SUM
VH34
VU
ltrip
MHDV
eEF
ltrip
MHDV
delta
delta
eEF
Uncertainty in diesel share of PC and LDV is important
2000
2010
NOX
NOX
1-SUM
1-SUM
VU
ltrip
eEF
UHDV
delta
MPC
eEF
MHDV
tau
MHDV
MPC
important variables are eEF and MPC
delta
 becomes important
2000
2010
CO2
CO2
1-SUM
1-SUM
MPC
MPC
MHDV
MHDV
UPC
VH34
UPC
ltrip
VU
ltrip
CO2 emissions are mostly influenced by MPC (SMPC=37%)
and ltrip. Situation remains unchanged in 2010
VU becomes important in 2010
Interpretation and conclusions
Output variability for each pollutant IS described by
three most influential input parameters.
ltrip, eEF, VU and  are common to almost all the pollutants.
Technological and fuel improvements will result in
reduced emissions for VOC, PM and NOX (2000  2010).
Quality of emission estimates can be enhanced if
we direct efforts to improve our knowledge on
average trip length, emission factors, diesel share
between PC and LDV and the annual mileage of
passenger cars
Importance of emission factors , with the current statistical model,
increases 2000  2010.
Uncertainty in emission factors should be explained by
a set of kinetic parameters (not only average speeds).
Acknowledge uncertainty
in the emission factors
at the level of driving cycles
When driving cycles are
combined to build TS,
it is straightforward to
calculate uncertainty
bounds for TS.