Comparison of Simulation Methods
Using Historical Data in the
U.S. International Price Program
M.J. Cho, T-C. Chen, P.A. Bobbitt, J.A. Himelein,
S.P. Paben, L.R. Ernst, and J.L. Eltinge
U.S. Bureau of Labor Statistics
Cho.Moon@bls.gov
ICES III Session 14 – June 19, 2007
Disclaimer: The views expressed in this paper are those of the
authors and do not necessarily reflect the policies of the BLS
Outline
I.
U.S. International Price Program (IPP)
II.
Bootstrap Variance Estimator
III. Adequacy of Approximations in Simulation
Studies
IV. Simulation Methods Considered
V.
Numerical Results
VI. Conclusions
2
I.
U.S. International Price Program
(IPP)
A.
IPP is the nation’s primary source
of information on price trends in the
international trade of the U.S. economy
Export and Import Price Indexes
B.
IPP involves population structure,
sample design and estimation methods
with a high degree of complexity
3
C.
IPP Index Aggregation Tree
Upper Level Strata
Lower Level Strata
Classification Groups
Weight Groups (Company|CG)
Items
4
D.
IPP Sampling Design (Imported Goods)
1. Sample frame provided by U.S. Customs Border
Protection, and divided into two biannual panels
2. Stratified multistage sampling within panels
a. Within a broad product category (stratum), select
establishments proportional to trading dollar value
b. Within establishment, select detailed product
categories (CGs) using systematic PPS
c. Within Company|CG, select items
5
E.
IPP Index Formula
(modified Laspeyres index)
LTR
t
parent
t
  LTRchild
 Wchild
 child

t 1
LTR
child  Wchild

 child


t 1
  LTR parent


LTR  STR  LTR
t
t
t 1
6
F.
Weights
1.Item weight is weight group’s weight divided by
number of items
2.Weight group’s weight is a trade dollar value
divided by selection probability
3.CG weight is based on trade weights from the
Census Bureau
4.Stratum weight is aggregation of CG weights
7
II.
Bootstrap Variance Estimator
A.
Based on Rao, Wu and Yue (1992)
1.
Draw nh  1 PSUs (establishments) with replacement
from the nh sampled PSUs in each stratum
2.
Define bootstrap weights:
 nh  * 
nhi 
w  whik 
 nh  1  
where whik is original sample weights
*
hik
for the item k in PSU i of stratum h, and
nhi* is the number of times that the PSU i is selected
8
3.
Calculate the price STR using bootstrap weights
4.
Repeat steps 1-3 B times
5.
Compute the bootstrap variance estimator:
1
ˆ
VBT 
B

B
b 1
ˆb  ˆfull

2
where
ˆb is a bootstrap STR estimator and
ˆfull is the STR estimator from the original sample
9
B.
Properties of VˆBT Under Realistic
Approximations to IPP Population Structure,
Sample Design and Estimation Methods
1.
Over R replications of the “simulation design”
compute


1
ˆ
ˆ
 , VBT   R


R

r 1
1
ˆ
V VBT   R  1

ˆ r  , VˆBT  r 
R

r 1


VˆBT  r  - VˆBT 

2
10
2.
Evaluation Criteria
Relative Bias =

1
R  1 
r 1

R
ˆ  ˆ   Vˆ
 BT   1
r 


 
Degrees of Freedom =
  
~ ˆ
V VBT
1
2
1
 
2 VˆBT 
2
11
III.
A.
Adequacy of Approximations
in Simulation Studies
Notation
X 
ˆ 

Features of population and design
( X T  True, X A  Approximation )
Estimator, with cdf F (ˆ, X )

Q F (ˆ, X )  Specific functional of F (ˆ, X )
e.g. bias or MSE
12
B.
Adequacy of Simulation & Approximation
Qˆ {F{ˆ, X A )} for true Q{F{ˆ, X T )} ?
1. Components to include in X A
a. Population structure
b. Sampling steps
c. Nonresponse
d. Weighting
e. Variance estimation
2. Approximations for each component in (1)
1.
Taylor expansion: under conditions
Qˆ {F {ˆ, X A )}  Q{F {ˆ, X T )}
Q{F {ˆ, X )}

X
X X
 X A  XT 
T
2
ˆ, X )}
1

Q
{
F
{

  X A  X T 
 X A  XT 
2
X X 
X X*
 (Replication Error)
where
X *   X A  (1   ) X T for    0,1
14
2.
Four Cases
Case 1: Small first derivatives, small
X A  XT
Case 2: Small first derivatives, large
X A  XT
Case 3: Large first derivative
Case 4: Large second derivative
15
IV. Simulation Methods Considered
A.
Use a Single Method for Selection of
Sample Units and Related Weights:
Same sample units and weights for each month
B.
Three Methods to Construct Population of
Item-Level STRs
1. Resampling Method
2. Fixed-One-Rate Method
3. CDF Interpolation Method
17
V.
A.
Numerical Results
Selected Strata
Stratum
Stratum Description
P07
Edible vegetables, roots, and tubers
P08
Edible fruit and nuts; peel of citrus fruit or melons
P09
Coffee, tea, mate and spices
P22
Beverages, spirits, and vinegar
P61
Articles of apparel and clothing accessories
P74
Copper and articles thereof
P90
Optical, photographic, measuring and medical
instruments
Item STR1 of Design Strata
Original Variance of STR (P90)
Relative Bias of STR (P90)
Degrees of Freedom of STR (P90)
VI. Conclusions
A.
For Complex Establishment Surveys Like
IPP, Simulation-Based Evaluations of Est
Properties Require Consideration of
1.
Approximations X A to the true population,
design and estimation procedures
2.
Adequacy of the resulting
approximations Q{F (ˆ, X A )}
to the true properties Q{F (ˆ, X T )}
23
B.
Future Work
1.
For IPP: Other features of the pop and design
e.g. Independence of STRs from sample units
2.
Consider generalized variance estimator to
improve stability
3.
Explore the surface defined by


Q F (ˆ, X )
in “neighborhoods” of the true X T
24