Section 4AB: Segment data
Interpolation/Extrapolation
Andreea Erciulescu
CSSM - NRI Project
January 17, 2012
Andreea Erciulescu (Survey)
January 17, 2012
1 / 24
Part A : Interpolation
• Imputation is carried out for each segment coveruse
• The rotation segments are not sampled every year, hence the need for
interpolation
• The new assignment is done for an year if there is data for the
previous and for the next years.
• There are three steps:
• Small change or negative change in large urban
• Large change, not urban
• Large positive change in large urban
Andreea Erciulescu (Survey)
January 17, 2012
2 / 24
Notation
• yt,c,k = segment acres in coveruse c for segment k, year t
• B = survey year directly prior to the hole
• M = survey year directly after the hole
• u = permanent random number assigned to segments
• Ŷt,c,gls = the GLS estimate for coveruse c, for year t
Andreea Erciulescu (Survey)
January 17, 2012
3 / 24
Small change or negative change in large urban
Scenario
• |yM,c,k − yB,c,k | < 8 + (M − B) or
• Negative change in large urban
Procedure
• The change for each coveruse is assigned to one survey year with
probability ∝ the period covered by the change
• The change is assigned on the basis of u:
• If 0.000 ≤ u < 0.500, assign the change
• If 0.500 ≤ u < 0.667, assign the change
• If 0.667 ≤ u < 0.833, assign the change
• If 0.833 ≤ u < 1.000, assign the change
Andreea Erciulescu (Survey)
1997
2000
2001
2002
-
2000
2001
2002
2003
(prob=3/6)
(prob=1/6)
(prob=1/6)
(prob=1/6)
January 17, 2012
4 / 24
Small change or negative change in large urban
Example
Table: 4.1. PSU Data Before and After Imputation
2003 panel
Large urban
2003 panel
Small water
GLS estimates
Large urban
1997
(j=1)
2000
(j=2)
2001
(j=3)
2002
(j=4)
2003
(j=5)
80
80
80
85
85
14
14
14
10
10
140
150
160
180
200
Note: Using the random number u=0.68.
Andreea Erciulescu (Survey)
January 17, 2012
5 / 24
Large change, not in large urban
Scenario
• |yM,c,k − yB,c,k | ≥ 8 + (M − B) and
• The change is not in large urban
Procedure
• The change for each coveruse is assigned to one survey year
proportional to the number of years covered by the change
• Linear interpolation is used and:
yt,c,k = yB,c,k + (yM,c,k − yB,c,k )
Andreea Erciulescu (Survey)
t −B
M −B
January 17, 2012
6 / 24
Large positive change in large urban
Scenario
• (yM,11,k − yB,11,k ) ≥ 8 + (M − B)
Procedure
• The change for each coveruse is assigned to one survey year
proportional to the change in the GLS estimate of large urban:
yt,c,k = yB,c,k + (yM,c,k − yB,c,k )
Andreea Erciulescu (Survey)
Ŷt,11,gls − ŶB,11,gls
ŶM,11,gls − ŶB,11,gls
January 17, 2012
7 / 24
Large positive change in large urban
Example
Table: 4.1. PSU Data Before and After Imputation
2003 panel
Large urban
2003 panel
Small water
GLS estimates
Large urban
Andreea Erciulescu (Survey)
1997
2000
2001
2002
2003
20
30.83
41.66
63.33
85
14
13.33
12.67
11.33
10
140
150
160
180
200
January 17, 2012
8 / 24
Large positive change in large urban Example - Calculation
y2000,11,k = y1997,11,k + (y2003,11,k − y1997,11,k )(10/60) = 30.83
y2001,11,k = y1997,11,k + (y2003,11,k − y1997,11,k )(20/60) = 41.67
y2002,11,k = y1997,11,k + (y2003,11,k − y1997,11,k )(40/60) = 63.33
y2000,8,k = y1997,8,k + (y2003,8,k − y1997,8,k )(10/60) = 13.33
y2001,8,k = y1997,8,k + (y2003,8,k − y1997,8,k )(20/60) = 12.67
y2002,8,k = y1997,8,k + (y2003,8,k − y1997,8,k )(40/60) = 11.33
where:
Ŷ2003,11,gls − Ŷ1997,11,gls = 200 − 140 = 60
For example:
y2002,11,k = y1997,11,k +(y2003,11,k −y1997,11,k )(40/60) = 20+65∗40/60 = 63.33
Andreea Erciulescu (Survey)
January 17, 2012
9 / 24
Part B: Extrapolation
• Imputation is carried out for each segment coveruse
• The rotation segments are not sampled after some year T, hence the
need for extrapolation
• The new assignment is done for an year if there is data for the
previous years only.
• There are five steps:
• Linear extrapolation
• Adjustment to GLS level for year T
• Ratio adjustment to GLS changes for segment coveruse
• Ratio adjustment to compensate for change extrapolation
Andreea Erciulescu (Survey)
January 17, 2012
10 / 24
Notation
• Ŷt,c,gls = the GLS estimate for coveruse c, for year t
• T = the final year; for 2005 estimation, T = 2005
• p = the last year observed on panel p
• pmin is the minimum value of p for p < T ; pmin is 2001 for 2005
estimation.
• Ap is the set of indeces in panel p
• w1k is the ratio adjusted weight for segment k defined in section 4D
Andreea Erciulescu (Survey)
January 17, 2012
11 / 24
Linear extrapolation
Procedure
• Carried out for each segment coveruse by panel
• The value is:
ỹT ,c,k = yp,c,k + (yp,c,k − y1997,c,k )(p − 1997)−1 (T − p)
• If there is a real point going out of large urban, use:
ỹT ,10,k = yp,10,k ,for large urban and ỹT ,11,k = yp,11,k ,for small urban
• If
yp,11,k > 0 and ỹT ,10,k − yp,10,k < −0.9(ỹT ,11,k − yp,11,k )
, then
ỹT ,10,k = yp,10,k − 0.9(ỹT ,11,k − yp,11,k )
Note: There will remain some negative values, for example for small water
in some segments.
Andreea Erciulescu (Survey)
January 17, 2012
12 / 24
Linear extrapolation
Artificial Example
Table: 4.1. PSU Data Before and After Imputation
P0
P1
P2
P3
GLS
Andreea Erciulescu (Survey)
1997
20
20
20
20
80
2001
28
30
28
32
110
2002
30
32.5
30
35
118
2003
35
35
32
38
128
2004
35
37.5
34
41
136
2005
40
40
36
44
160
January 17, 2012
13 / 24
Linear extrapolation
Edit
• Performed on urban values
• If yp,11,k > 0 and there is no real point going out of large urban:
Define
H10,k = ỹT ,11,k + ỹT ,10,k − yp,11,k − yp,10,k
and
yT∗ ,11,k = ỹT ,11,k + max(0, H10,k )
and
yT∗ ,10,k = max(0, yp,10,k + H10,k − yT∗ ,11,k + yp,11,k )
• If yT∗ ,11,k > 0,yT ,12,k < 0 and there is no real point going out of large
urban:
yT∗∗,11,k = max(yp,11,k , yT∗ ,11,k + yT ,12,k )
and
yT∗∗,12,k = 0
Andreea Erciulescu (Survey)
January 17, 2012
14 / 24
Adjustment to GLS level for year T
Define the new linearly extrapolated values as:
z1,c,k = ỹT ,c,k − yp,c,k if ỹT ,c,k − yp,c,k > 0, and 0 ow
z2,c,k = ỹT ,c,k − yp,c,k if ỹT ,c,k − yp,c,k < 0, and 0 ow
Then the estimate change from p to T is
X
w1k (z1,c,k + z2,c,k ) =: ∆1,T ,c,p + ∆2,T ,c,p
∆T ,c,p =
k∈Ap
The GLS estimated change from pmin to T is
∆T ,c,pmin,gls = ŶT ,c,gls − Ŷp min, c, gls
Andreea Erciulescu (Survey)
January 17, 2012
15 / 24
Adjustment to GLS level for year T
Procedure:
• Adjust the linearly extrapolated values to get the GLS estimated
change:
∆T ,c,pmin,obs =
XX
w1k (yp,c,pk − ypmin,c,pk )
p k∈Ap
• The change remaining to be extrapolated is:
∆T ,c,pmin,gls − ∆T ,c,pmin,obs
and the total number of extrapolation periods is
P
(T − p).
panels
• In 2005,
P
(T − p) = (2005 − 2003) + (2005 − 2002) + (2005 − 2001) = 9
panels
Andreea Erciulescu (Survey)
January 17, 2012
16 / 24
Adjustment to GLS level for year T
Procedure (cont.):
• The remaining change per period is:
γave =
∆T ,c,pmin,gls − ∆T ,c,pmin,obs
P
(T − p)
panels
• The change from p to T to be extrapolated for panel p is:
∆∗T ,c,p = (T − p)γave and
• The modified extrapolated value is:
ŷT ,c,k = yp,c,k + (1 + αcp )z1,c,k + (1 − αcp )z2,c,k , where
αcp =
Andreea Erciulescu (Survey)
∆∗T ,c,p − ∆T ,c,p
∆1,T ,c,p − ∆2,T ,c,p
January 17, 2012
17 / 24
Adjustment to GLS level for year T
Artificial example
• GLS estimated change for total between 2001-2005:
∆05,01,gls = 160 − 110 = 50
• Observed or interpolated change for total between 01-p:
∆05,01,obs = (40−28)(core)+0(p1)+(30−28)(p2)+(38−32)(p3) = 20
• The remaining change per period:
γave =
∆05,01,gls − ∆05,01,obs
50 − 20
=
= 3.33
(5 − 1)(p1) + (5 − 2)(p2) + (5 − 3)(p3)
4+3+2
• Change from p to T that is to be extrapolated:
∆∗05,01 = 3.33·4 = 13.33, ∆∗05,02 = 3.33·3 = 10, ∆∗05,03 = 3.33·2 = 6.67
Andreea Erciulescu (Survey)
January 17, 2012
18 / 24
Adjustment to GLS level for year T
Artificial example
• Let:
13.33 − 10
= 0.33
10
10 − 6
= 0.67
α02 =
6
6.67 − 6
α03 =
= 0.11
6
• The modified extrapolated value for segment k of panel p in 05:
α01 =
p = 01 : ŷ05,k = 30 + (1 + 0.33) · (40 − 30) = 43.33
p = 02 : ŷ05,k = 30 + (1 + 0.67) · (36 − 30) = 40
p = 03 : ŷ05,k = 38 + (1 + 0.11) · (44 − 38) = 44.67
Andreea Erciulescu (Survey)
January 17, 2012
19 / 24
Adjustment to GLS level for year T=2005
Artificial example
Table: 4.1. PSU Data Before and After Imputation and GLS adjustment
P0
P1
P2
P3
GLS
1997
20
20
20
20
80
2001
28
30
28
32
110
2002
30
43.33
30
35
118
2003
35
43.33
40
38
128
2004
35
43.33
40
44.67
136
2005
40
43.33
40
44.67
160
(40 + 43.33 + 40.0 + 44.67) − (28 + 30 + 28 + 32) = 168 − 118 = 50 =
160 − 110 = ∆05,01,gls
- the interpolation procedure is used to create coveruse values for each
year between p and T=05
Andreea Erciulescu (Survey)
January 17, 2012
20 / 24
Ratio adjustment to GLS changes for segment coveruses
Procedure:
• The segment acres are first adjusted to the 1997 control level:
ynew ,t,c,k = Rcadj yt,c,k , where Rcadj = (
X
w1k yt,c,k )−1 Ŷ1997,c
k∈Acb
• Given the values at time T for every segment the interpolation
procedure is used to create values for each year.
• Repeat the steps above for T 7→ t and pmin 7→ t − 1:
ŷt,c,k = yt−1,c,k +(1+αt,c,t−1 )z1,t,c,t−1,k +(1−αt,c,t−1 )z2,t,c,t−1,k , where
∗
αt,c,t−1
=
Andreea Erciulescu (Survey)
∆t,c,t−1,gls − ∆t,c,t−1,obs
∆1,t,c,t−1,obs − ∆2,t,c,t−1,obs
January 17, 2012
21 / 24
Ratio adjustment to compensate for change extrapolation
Scenario
• Adjust the weights of temporary pseudo points for the added change
pseudo points.
• A total adjustment rather than a segment adjustment is required
because a segment adjustment that keeps segment size fixed is not
possible.
Procedure
Let the total weight for temporary pseudo points that are in control
coveruse c from 1997 to T be:
X
Tspsc,c =
δpsc,c,i w1,i ,where
i∈Sam
δpsc,c,i = 1 if point i is a temporary segment pseudo point and 0, ow
w1,i = weight assigned to temporary pseudo points by pseudo program
Andreea Erciulescu (Survey)
January 17, 2012
22 / 24
Ratio adjustment to compensate for change extrapolation
Procedure (cont.)
The weights for the extrapolated change pseudos are
w1i∗ = Rspsc,c δpsc,c,i w1,i + (1 − δpsc,c,i )w1,i ,where
Txt,cps,c
−1
Rspsc,c = (Tspsc,c − Txt,cps,c )Tspsc,c
,
X X
=
δcsup,i δc,t,i w1,i , total initial weight
t
i∈Sam
for the added temporary pseudo points
δc,t,i = 1 if point i is a temporary segment pseudo point and 0, ow
δcsup,j = 1 if point j is an added pseudo point
for extrapolated change from year t and 0, ow
The operation of creating temporary pseudo points for year-to-year
changes is completed as part of the pseudo point program.
Andreea Erciulescu (Survey)
January 17, 2012
23 / 24
End
Thank you!
Andreea Erciulescu (Survey)
January 17, 2012
24 / 24