NRCSE
Modelling non-stationarity
in space and time
for air quality data
Peter Guttorp
University of Washington
peter@stat.washington.edu
Outline
Lecture 1: Geostatistical tools
Gaussian predictions
Kriging and its neighbours
The need for refinement
Lecture 2: Nonstationary covariance estimation
The deformation approach
Other nonstationary models
Extensions to space-time
Lecture 3: Putting it all together
Estimating trends
Prediction of air quality surfaces
Model assessment
Research goals in
air quality modeling
Create exposure fields for health
effects modeling
Assess deterministic air quality models
Interpret environmental standards
Enhance understanding of complex
systems
The geostatistical setup
Gaussian process Z(s),s D  R2
m(s)=EZ(s) Var Z(s) < ∞
Z is strictly stationary if
d
(Z(s1),...,Z(sk )) (Z(s 1  h),...,Z(s k  h))
Z is weakly stationary if
m(s)  m
Cov(Z(s1),Z(s 2 ))  C(s1  s 2 )
Z is isotropic if weakly stationary and
C(s1  s2 )  C 0 ( s1  s2 )
The problem
Given observations at n sites
Z(s1),...,Z(sn)
estimate
Z(s0) (the process at an unobserved site)
or  Z(s)d(s) (a weighted average of the
A
process)
A Gaussian formula
If
X m X XX
 ~ N , 
Y  m Y YX
XY 

YY 
then (Y | X) ~ N(m Y  YX 1
XX (X  m X ),
YY  YX 1
XX XY )


Simple kriging
Let X = (Z(s1),...,Z(Sn))T, Y = Z(s0), so
mX=m1n, mY=m,
XX=[C(si-sj)], YY=C(0), and YX=[C(si-s0)].
Thus
1
Zˆ (s0 )  m  C(si  s0 )T C(si  sj ) X  m1n 
This is the best linear unbiased predictor for
known m and C (simple kriging).
Variants: ordinary kriging (unknown m)
universal kriging (m=Abfor
some covariate A)
Still optimal for known C.
Prediction error is given by

 C(si  s0 )
C(0)  C(si  s0 ) C(si  sj )
T
1
The (semi)variogram
( t ) 
1
Var(Z(s  t)  Z(s))  C(0)  C( t )
2
Intrinsic stationarity
Weaker assumption (C(0) need not
exist)
Kriging can be expressed in terms of
variogram
Estimation of
covariance functions
Method of moments: square of all pairwise
differences, smoothed over lag bins
Problem: Not necessarily a valid variogram
Least squares
Minimize
2

2
    Z(si )  Z(sj )  ( si  sj ; 


i j
Alternatives:
•fourth root transformation
•weighting by 1/2
•generalized least squares
Fitted variogram
 (t) 
 e2


 s2 1

 t  
e


Kriging surface
Kriging standard error
A better combination
Maximum likelihood
Z~Nn(m,) = a[r(si-sj;)] = a V()
Maximize
n
1
(m ,a,)   log( 2a )  log det V()
2
2
1

(Z  m )TV ()1 (Z  m )
2a
m
ˆ 1T Z / n
ˆ) / n
a
ˆ G(
G()  (Z  m
ˆ )TV()1 (Z  m
ˆ)
ˆ maximizes the profile likelihood
and 

n
G2 () 1
* ()   log
 logdetV()
2
n
2
A peculiar ml fit
Some more fits
All together now...
Effect of estimating
covariance structure
Standard geostatistical practice is to take
the covariance as known. When it is
estimated, optimality criteria are no longer
valid, and “plug-in” estimates of variability
are biased downwards.
ˆ (s 0 ; 
ˆ ) )  
ˆ)
Var( Z
ˆ 2 (
ˆ (s 0 ; 
ˆ )  cov(Z
ˆ )
2 trcov(
(Zimmerman and Cressie, 1992)
A Bayesian prediction analysis takes
proper account of all sources of uncertainty
(Le and Zidek, 1992)
Violation of isotropy
 e2  127.1(259)
 e2  154.6 (134)
 s2  68.8 (255)
  10.7 (45)
 s2  141.0 (127)
  29.5 (35)
General setup
Z(x,t) = m(x,t) + (x)1/2E(x,t) + e(x,t)
trend +
smooth
+ error
We shall assume that m is known or
constant
t = 1,...,T indexes temporal replications
E is L2-continuous, mean 0, variance 1,
independent of the error e
C(x,y) = Cor(E(x,t),E(y,t))
D(x,y) = Var(E(x,t)-E(y,t)) (dispersion)
 (x)(y)C(x,y) x  y
Cov(Z(x,t),Z(y,t))  
2
xy
 (x)   e
Geometric anisotropy
Recall that if C(x,y)  C( x  y ) we have
an isotropic covariance (circular
isocorrelation curves).
If C(x,y)  C( Ax  Ay ) for a linear
transformation A, we have geometric
anisotropy (elliptical isocorrelation
curves).
General nonstationary correlation
structures are typically locally
geometrically anisotropic.
The deformation idea
In the geometric anisotropic case, write
C(x,y)  C( f (x)  f(y) )
where f(x) = Ax. This suggests using a
general nonlinear transformation
f:R 2  Rd . Usually d=2 or 3.
G-plane D-space
We do not want f to fold.
Implementation
Consider observations at sites x1, ...,xn.
ˆ ij be the empirical covariance
Let C
between sites xi and xj. Minimize
2
ˆ
(,f )  w ij Cij  C(f(x i ),f (x j );)  J(f )
i,j
where J(f) is a penalty for non-smooth
transformations, such as the bending
energy
 2 2
2 2  2 2 

 f 
 f   f  


J(f)   

2
xy   2  dxdy
2 


 y  
x 
SARMAP
An ozone monitoring exercise in
California, summer of 1990, collected
data on some 130 sites.
Transformation
This is for hr. 16 in the afternoon
Thin-plate splines
f(s)  c  As  W' 
˜ (s)
Linear part

˜ (s)   (s  x1),...,(s  xn )'
2
(h)  h log( h )
1' W  0
X' W  0
J(f)  tr(W' S˜ W)
A Bayesian implementation
Likelihood:
 T

L(S | )  (2  )(T 1)/ 2 exp  tr 1S 
 2

Prior:
 1 2 '

˜
p(W)  exp 
W
S
W
 i i 
 2 i1

Linear part:
–fix two points in the G-D mapping
–put a (proper) prior on the remaining two
parameters
Posterior computed using Metropolis-Hastings
California ozone
63 Region 6 monitoring sites and their representation in a
deformed coordinate system reflecting spatial covariance
Thu Oct 30 00:12:36 PST 2003
55643
26
54
5 56
2530
43
2062
26
52
32
15
7
17
58 1147 8
18
6 946223727
41
12
42
3 51
35
45 49
44
53
57 55
40
5928636138
502 460
19
29
23
36
48
33
39
10 31 1 21
34 24
14
13
16
54
30
15
25
2062
32 478
52
17
7 11
69 22 27
41
12
5842
18
351
45 49
46 37
40
35
19
44
57
53
2 55
38
61
36 1 21
63
50 46059
23
28
29 10 31
48 24
33
39 34
13
16
14
Posterior samples
N=63, S. Calif: 4 samples from the posterior distribution of deformations reflecting spatial covariance
Tue Oct 28 22:18:29 PST 2003
56
5
2643
54
30
25
20
62
32
527 478
17 15
11
41
6922 27
58
12
18
351
42
46 37
45 49
40
35
19
44
57
53
121
61
38
63
55
59
502 460
231036
28
29
31
48 24
33
39 34
13
14 16
556
2643
54
30
62
20
25
17 15
327478
52
11
6922 27
41
12
58
18
351
42
35
4637 45 49
40
19
44
53
57
2460
61
38 36 121
55
59
63
50
231031
28
29
33
48
39 34 24
13
14 16
55643
26
54
30
25
20
62
17 15
327478
52
11
58
18
6
41
27
51
12
3
42922
4637 45 49
35
44
57
53
2 460
38 36
61
63
55
59
50
23
28
10
29
48 2431
33
39 34 13
16
14
40
19
121
56
5
2643
54
15
30
25
20
62
17
32
52
478
711
40
58
6
41
18
12
351
42
92227 45 49
35
46
19
37
53
44
57
21
2 460
61
38 36 1
63
59
50
55
23
28
31
10
29
48 24
33
39 34 13
16
14
Other applications
Point process deformation (Jensen &
Nielsen, Bernoulli, 2000)
Deformation of brain images (Worseley
et al., 1999)
Isotropic covariances
on the sphere
Isotropic covariances on a sphere are
of the form

C(p,q)   aiPi (cos  pq )
i 0
where p and q are directions, pq the
angle between them, and Pi the
Legendre polynomials.
Example: ai=(2i+1)ri
C(p,q) 
1 r 2
1 2rcos  pq  r
2
1
A class of global
transformations
Iteration between simple parametric
deformation of latitude (with
parameters changing with longitude)
and similar deformations of longitude
(changing smoothly with latitude).
(Das, 2000)
Three iterations
Global temperature
Global Historical Climatology Network
7280 stations with at least 10 years of
data. Subset with 839 stations with data
1950-1991 selected.
Isotropic correlations
Deformation
Assessing uncertainty
Gaussian moving averages
Higdon (1998), Swall (2000):
Let x be a Brownian motion without
drift, and X(s)  R 2 b(s  u)dx(u) . This is
a Gaussian process with correlogram
r(d)  R 2 b(u)b(u  d)du.
Account for nonstationarity by letting
the kernel b vary with location:
r(s 1,s 2 ) 
R 2 bs 1 (u)bs 2 (u)du
Kernel averaging
Fuentes (2000): Introduce orthogonal local
stationary processes Zk(s), k=1,...,K,
defined on disjoint subregions Sk and
construct
Z(s) 
K
 wk (s)Zk (s)
k 1
where wk(s) is a weight function related to
dist(s,Sk). Then
r(s1,s 2 ) 
K
 wk (s1 )wk (s 2 )rk (s1  s 2 )
1
A continuouskversion
has
Z(s)   w(x  s)Z (s ) (x)ds
Simplifying assumptions in
space-time models
Temporal stationarity
seasonality
decadal oscillations
Spatial stationarity
orographic effects
meteorological forcing
Separability
C(t,s)=C1(t)C2(s)
SARMAP revisited
Spatial correlation structure depends
on hour of the day (non-separable):
Bruno’s seasonal
nonseparability
Y(t,x)  m(t,x)   t (x)(a x Z1(t)  Z 2 (t,x))  e(t,x)
Nonseparability generated by
seasonally changing spatial term  t (x)
Z1 large-scale feature
Z2 separable field of local features

(Bruno, 2004)
A non-separable class of
stationary space-time
covariance functions
Cressie & Huang (1999):
Fourier domain
Gneiting (2001): f is completely
monotone if (-1)n f (n) ≥ 0 for all n.
Bernstein’s
theorem :
  rt
f(t)  0 e dF(r) for some nondecreasing F.
Combine a completely monotone
function  and a function y with
completely monotone derivative into a
space-time covariance
 h 2 
2
C(h,u) 

2 d/ 2 
2 
y( u )
y (u )
A particular case
C(h,u)  (u
2a
2


h
1
 1) exp 2a
 
 (u  1) 
a=1/2,=1/2
a=1/2,=1
a=1,=1/2
a=1,=1
Uses for surface estimation
Compliance
–exposure assessment
–measurement
Trend
Model assessment
–comparing (deterministic) model to
data
–approximating model output
Health effects modeling
Health effects
Personal exposure (ambient and nonambient)
Ambient exposure
outdoor time
infiltration
Outdoor concentration model for
individual i at time t
Cit  m(si ,t)  (si ,t)

Seattle health effects study
2 years, 26 10-day sessions
A total of 167 subjects:
56 COPD subjects
40 CHD subjects
38 healthy subjects
(over 65 years old, non-smokers)
33 asthmatic kids
A total of 108 residences:
55 private homes
23 private apartments
30 group homes
Ogawa sampler
HPEM
PUF
pDR
T/RH logger
CO2 monitor
Ogawa
sampler
Nephelometer
CAT
HI
Quiet
Pump Box
PM2.5 measurements
Where do the subjects
spend their time?
Asthmatic kids:
– 66% at home
– 21% indoors away from home
– 4% in transit
– 6% outdoors
Healthy (CHD, COPD) adults:
– 83% (86,88) at home
– 8% (7,6) indoors away from home
– 4% (4,3) in transit
– 3% (2,2) outdoors
Panel results
Asthmatic children not on antiinflammatory medication:
decrease in lung function related to
indoor and to outdoor PM2.5, not to
personal exposure
Adults with CV or COPD:
increase in blood pressure and heart
rate related to indoor and personal PM2.5
Spatial Analysis Region Definitions
Region 6: S Calif, all 94 sites, fitting and validation
Fitting (63)
Validation (31)
Los Angeles County
Trend model
m(si ,t)  m 1(si )  m 2 (si ,t)
m 1(si )  m 0 (si )    kVik
where Vik are covariates, such as
population density, proximity to roads,
local topography, etc.

m 2 (si ,t)   r j (si )fj (t)
where the fj are smoothed versions of
temporal singular vectors (EOFs) of the
TxN data matrix.

We will set m1(si) = m0(si) for now.
SVD computation
400
200
0
Singular value
600
800
Singular values of T=2912 x S=545 observation matrix
0
100
200
300
Index, 1:545
400
500
EOF 1
0.04
0.02
0.0
-0.04
Annual.svd$svd$u[1:1456, j]
Annual Trend Component 1
01/01/1987
10/01/1987
07/01/1988
04/01/1989
01/01/1990
10/01/1990
01/01/1994
10/01/1994
0.04
0.02
0.0
-0.02
Annual.svd$svd$u[1457:2912, j]
dates87to94[1:1456]
01/01/1991
10/01/1991
07/01/1992
04/01/1993
dates87to94[1457:2912]
EOF 2
0.06
0.02
-0.02
-0.06
Annual.svd$svd$u[1:1456, j]
Annual Trend Component 2
01/01/1987
10/01/1987
07/01/1988
04/01/1989
01/01/1990
10/01/1990
01/01/1994
10/01/1994
0.06
0.02
-0.02
-0.06
Annual.svd$svd$u[1457:2912, j]
dates87to94[1:1456]
01/01/1991
10/01/1991
07/01/1992
04/01/1993
dates87to94[1457:2912]
EOF 3
0.04
0.0
-0.04
Annual.svd$svd$u[1:1456, j]
Annual Trend Component 3
01/01/1987
10/01/1987
07/01/1988
04/01/1989
01/01/1990
10/01/1990
01/01/1994
10/01/1994
0.06
0.02
-0.02
-0.06
Annual.svd$svd$u[1457:2912, j]
dates87to94[1:1456]
01/01/1991
10/01/1991
07/01/1992
04/01/1993
dates87to94[1457:2912]
Region 6 : S. Calif
Starplot of temporal trend coefficients (LA)
061111003
061112003
060719004
060714003
060370113
060374002
0.4
0.2
0.0
sqrt(max 8hr O3)
60714003
01/01/1989
01/01/1990
01/01/1991
01/01/1992
01/01/1993
01/01/1994
01/01/1993
01/01/1994
01/01/1993
01/01/1994
1987-1994
0.4
0.2
0.0
sqrt(max 8hr O3)
60719004
01/01/1989
01/01/1990
01/01/1991
01/01/1992
1987-1994
0.4
0.2
0.0
sqrt(max 8hr O3)
60374002
01/01/1989
01/01/1990
01/01/1991
01/01/1992
1987-1994
0.4
0.2
0.0
sqrt(max 8hr O3)
60370113
01/01/1989
01/01/1990
01/01/1991
01/01/1992
01/01/1993
01/01/1994
01/01/1993
01/01/1994
01/01/1993
01/01/1994
1987-1994
0.4
0.2
0.0
sqrt(max 8hr O3)
61112003
01/01/1989
01/01/1990
01/01/1991
01/01/1992
1987-1994
0.4
0.2
0.0
sqrt(max 8hr O3)
61111003
01/01/1989
01/01/1990
01/01/1991
01/01/1992
1987-1994
Kriging of m0
Ordinary kriging prediction of mu
0.26
1.5
0.25
1.0
0.24
0.5
0.23
0.0
y
0.22
0.21
-0.5
0.20
-1.0
0.19
-1.5
0.18
-2
-1
0
1
2
Kriging of r2
Ordinary kriging prediction of b2
1.5
1.0
1.0
0.5
0.5
y
0.0
-0.5
0.0
-1.0
-0.5
-1.5
-2
-1
0
x
1
2
Quality of trend fits
0.4
0.2
0.0
sq rt Ozone
Fitted trend (solid) vs Predicted (dashed): 060371002
01/01/1989
01/01/1990
01/01/1991
01/01/1992
01/01/1993
01/01/1994
D ate
0.4
0.2
0.0
sq rt Ozone
Fitted trend (solid) vs Predicted (dashed): 060371301
01/01/1989
01/01/1990
01/01/1991
01/01/1992
01/01/1993
01/01/1994
D ate
0.4
0.2
0.0
sq rt Ozone
Fitted trend (solid) vs Predicted (dashed): 060375001
01/01/1989
01/01/1990
01/01/1991
01/01/1992
01/01/1993
01/01/1994
Observed vs. predicted
0.4
0.3
0.1
0.0
0.0
0.1
0.2
0.3
0.4
0.5
Zval[, j]
0.5
0.4
0.3
0.2
0.1
0.0
0.0
0.1
0.2
0.3
Zval[, j]
0.0
0.1
0.2
0.3
Zval[, j]
060831015
Zpredf[ , j]
0.2
Zpredf[ , j]
0.3
0.2
0.0
0.1
Zpredf[ , j]
0.4
0.5
060296001
0.5
060371103
0.4
0.5
0.4
0.5
Observed vs. predicted,
cont.
0.2
0.0
sq rt Ozone
0.4
Observ ed (points) v s Predicted (lines): 060371301
01/01/1989
04/01/1989
07/01/1989
10/01/1989
01/01/1990
04/01/1990
07/01/1990
10/01/1990
04/01/1992
07/01/1992
10/01/1992
04/01/1994
07/01/1994
10/01/1994
0.2
0.0
sq rt Ozone
0.4
D ate
01/01/1991
04/01/1991
07/01/1991
10/01/1991
01/01/1992
0.2
0.0
sq rt Ozone
0.4
D ate
01/01/1993
04/01/1993
07/01/1993
10/01/1993
01/01/1994
D ate
Conclusions
Good prediction of
day-to-day variability
seasonal shape of mean
Not so good prediction of
long-term mean
Need to try to estimate

m1
Other difficulties
Missing data
Multivariate data
Heterogenous (in space and time)
geostatistical tools
Different sampling intervals
(particularly a PM problem)
Southern California
PM2.5 data
01/01/2001
01/01/2001
s1003v1
s1103v1
3
2
1
0
1
2
3
log PM2.5
4
dates.99.01
01/01/2000
01/01/2001
01/01/1999
01/01/2000
01/01/2001
s1201v1
s1301v1
3
2
0
1
2
3
log PM2.5
4
dates.99.01
4
dates.99.01
1
log PM2.5
01/01/2000
dates.99.01
0
01/01/1999
3
01/01/1999
0
log PM2.5
01/01/1999
2
0
01/01/2000
4
01/01/1999
1
log PM2.5
3
2
1
0
log PM2.5
4
s1002v1
4
s0002v1
01/01/2000
01/01/2001
dates.99.01
01/01/1999
01/01/2000
01/01/2001
dates.99.01