IMPLEMENTING UNIT VALUE INDICES IN THE
ANNUAL OECD INTERNATIONAL TRADE IN
COMMODITY STATISTICS (ITCS) DATABASE
Handling missing values, outliers in unit value
variations, representativity of the indices, data
conversion across classifications
Working Party on Trade in Goods and Services
7-9 November 2011
OECD Statistics Directorate
Outline of presentation
•
•
•
•
•
•
•
Background
Data universe and formulae
Supplementary units or net weights?
Conversion issues
Estimation of missing values
Dealing with outliers
Specific products
Background 1/2
Advantages of UVIs and Quantities Indices:
•
•
•
•
proxies for X & M price indices and for volume measures
available at product detail level and in a timely manner
international comparability
analysis of :
– terms of trade
– price and non-price competitiveness in X &M
– quantity effect from the transmission of inflation via foreign trade
• Context
• follow-up of the WPTGS 2010
• recommandation of the IMTS 2010
• results of the short Survey WPTGS 2010
Background 2/2
• Stocktaking of the implementation:
– on tracks with PWB
– process with SAS
– chained Laspeyres Paasche Fisher indices
– on 34 countries
– HS88 at a total level / partner World
• Issues that are being looking at :
– missing values (of quantities and trade values)
– outliers
– conversion across HS classifications
Data Universe 1/3
ITCS database :
– UNSD / OECD joint data processing system
– Contains Information on
– Trade values in US dollars
– Quantities (liters, m²…)
– Net Weight (kg)
– Data Availability
– HS (1988,1996,2002 & 2007)
– SITC (rev.2, rev.3)
– ISIC rev.3
Data Universe 2/3
• CIF / FOB Valuation
• Calculation based on 6 digits of HS1988
• Estimation of missing quantity/weight by
the UNSD method :
1) use of the available info (i.e. net weight proxied from quantity)
2) use of a Standard Unit Value
• Exclusion of 6 digits commodities without
information on weights (whole chapter 99 )
Data Universe 3/3
• UV = Value in US dollars
Quantity
(Pt)
• Unit Values Indices are sensitive
to exchange rates fluctuations;
not Quantity Indices …
Computation of Quantity indices
LASPEYRES Lt / t 1 (Q)
PAASCHE

P t / t 1(Q) 
P
P
t 1
t 1
Ft / t 1 (Q) 
* Qt 1
P *Q
P *Q
t
t
FISHER
* Qt
t
t 1
Pt 1 * Qt 1
Qt

*
 Pt 1 * Qt 1 Qt 1
Pt * Qt
Qt 1
 1/ 
*
 Pt * Qt Qt
Pt / t 1 (Q) * Lt / t 1 (Q)
Weighing system
Quantity ratio form
Computation of Unit Value indices
LASPEYRES Lt / t 1 ( P)
PAASCHE

P t / t 1( P) 
P *Q
P *Q
t 1
t
t 1
P *Q
P *Q
t
t 1
FISHER
Ft / t 1 ( P) 
t 1
t
t
Pt 1 * Qt 1
Pt

*
 Pt 1 * Qt 1 Pt 1
Pt * Qt
Pt 1
 1/ 
*
 Pt * Qt Pt
Pt / t 1 ( P) * Lt / t 1 ( P)
Weighing system
Quantity ratio form
Defining the denominator of the UV
Supplementary Units
or Net Weights ?
Should we use supplementary units?
• Supplementary units more accurate than
Net weights for some commodities BUT
 How to handle changes of quantity units
if for instance the quantity unit is one year, ‘’number of items (5)’’ and
the next year ‘’ thousands of items (9)’’
 On long series, net weights are more reported
 UNSD estimated that 75% of supplementary units is KG
=> OECD choice : net weight in kg
Representativity of the sample:
Issue with historical series
• 2009 :
33/34 OECD
countries with more
than 75% of total trade
values used in the
compilation of indices
for 2009
– <50% Israel X (80% for Israel M)
– >75% for All others OECD countries (M and X)
• 1999 : more problematic
8/34 OECD countries
have less than 75% total
trade used in the
compilation of indices for
1999
Half OECD
countries
have less
than 75% of
total trade
values used
to compiles
indices for
1989
– <25% Canada(M and X) , Australia M and USA M
– <60% + New Zealand (M), USA (X), Norway (X),
Australia (X)
– <75% + Japan (X), Norway (M), Mexico (M), New
Zealand (X) and Netherlands (M & X)
• 1989 / 21 countries
– <75% Canada (M & X), USA (M&X), New Zealand (M) and
Australia (M & X), Japan (X) , Norway (M & X)
=> Thresholds on UVI for coverage ?
Conversion issues
use of HS 1988
conversion of
HS 1996,
HS 2002
HS 2007
Conversion of HS
Exports of Poultry cuts and offal (HS 1988 cmd 020739) France
450
400
HS88 cmd 020739
350
HS1996 cmd 020713
300
HS1996 cmd 020713
250
200
150
HS1996 cmd 020726
100
50
HS1996 020735: Fresh or Chilled Cuts, Edible offal Ducks, Geese, Guinea Fowls, Spec. Dom.
HS1996 020713: Fresh or Chilled Cuts And Edible offal of Fowls of The Species
HS1996 020726: Fresh, Chilled Cuts, Edible offal of Turkeys, Species Domesticus
HS1988 020739: Poultry cuts & offal, except livers, fresh or chilled
2010
2009
2008
2007
2006
2005
2004
2003
2002
2001
2000
1999
1998
1997
1996
1995
1994
1993
1992
1991
1990
1989
0
1988
millions of USD
500
Conversion issues: looking at HS1988
020743: Duck, goose, guinea fowl cuts, offal not liver, froze
020750: Poultry livers, domestic, frozen
200
180
160
140
120
100
80
60
40
20
2010
2009
2008
2007
2006
2005
2004
2003
2002
2001
2000
1999
1998
1997
1996
1995
1994
1993
1992
1991
1990
1989
1988
0
Handling gaps in series
Estimating Missing
Values and Quantities:
Unconvincing tests of 2 methods
Estimation of Trade Values cmd 010111
140
120
millions USD
100
80
moving average
carry forward
60
40
20
0
1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009
Estimation of Quantities based on
Standard Unit Values (SUV)
• SUV
– Compiled by UNSD
– median unit value of
– each 6-digit commodity/year/flow
– after elimination of outliers
– of a sample of Unit Value of
– available data of the latest reporting year
– that respect a certain number of criteria (box 1)
Missing values estimation conclusions
• Thresholds for estimation for trade values:
1% of values estimated for the whole
chapter
• Calculated indices present dubious
movements :
• Large fluctuations for chained type Indexes at
6 digits level
=> Issues for more disaggregated indices (2&4 digits)
Outliers detection
Outlier detection
• “an unrealistic price growth in the
product specific distribution of unit
value ratios”
CEPII indices :
“ Unit values ratios are compared with the product
specific median change in unit values computed
over the whole period”
• .
Outlier detection
• “Value that lie far from the middle
of the distribution in either
direction”
• Mexico and Italy :
>100 obs : Asymmetric Fence Method
<100 obs : Mean Absolute Deviation
• UNSD : Tukey Method
AFM and MAD Formulae
• Assymetric Fence Method
i
 q1  uv hs
8  k AFM  max( q 2  q1 , c | q 2 |)


uv i  q  k
3
AFM  max( q 3  q 2 , c | q 2 |)
 hs8
• Mean Absolute Deviation
i
| uvhs
8  q2 | k MAD * MADhs 8


| uv i  q |  A* | q |
hs8
2
2

i
MADhs8  median(uvhs
8  q2 )
UV =logarithm of the unit value
Q1, Q2; Q3 1st 2nd 3rd quartiles of log unit values of trade distribution
c, k, A parameters
Specific products
%
chapters
84: Nuclear reactors, boilers,
machinery, etc
85:
Electrical,
electronic
equipment
29: Organic chemicals
of
outliers
within a chapter % of outliers within a
using symetric
chapter using AFM
yearly variation
MAD
detection
detection
method
method
15%
15%
3%
5%
1%
2%
Comparing OECD total level
indexes with those available
from other international
organisations and other
frameworks (SNA)
Percentage Change of Quantity Indices
for Iceland Imports
Percentage Change of Quantity Indices
for Turkey Imports
Percentage Change of Unit Values
Indices for Japan Import
Percentage Change of Unit Values
Indices for Italy Export
Next Steps
Following the Program of Work
Begining of 2012 :
• 34 countries at a 2 digits level applying methodologies
for outliers and missing values
• Finding some specific treatments for specific chapters
(including those that lose 30 % of their trade just by
changing classification (chapters 84 -85 )
Summer 2012 :
• Matrix of exports and imports unit value and quantity
indices available online for comments at the 2012
WPTGS
Questions to delegates
• Thresholds : on customs transactions ?
• Measure of Quantity: choice of supplementary quantity or
net weight values?
• HS Conversion issues : How to deal with cmd of HS change
?
• Estimation of Missing Values : What kind of methodology
do you recommend?
• More disaggregated indices (2 digits indices or more
detailed) do you have special warnings or experiences to share ?