Estimation and Model Selection
for Geostatistical Models
Kathryn M. Georgitis
Alix I. Gitelman
Oregon State University
Jennifer A. Hoeting
Colorado State University
R82-9096-01
The research described in this presentation has been funded by the U.S.
Environmental Protection Agency through the STAR Cooperative Agreement
CR82-9096-01 Program on Designs and Models for Aquatic Resource Surveys
at Oregon State University. It has not been subjected to the Agency's review
and therefore does not necessarily reflect the views of the Agency, and no
official endorsement should be inferred
Talk Outline
•
•
•
•
•
•
Stream Sulfate Concentration
G.I.S. Data Sources
Bayesian Spatial Model
Implementation Problems
What exactly is the problem?
Simulation results
Original Objective:
Model sulfate concentration in streams in
the Mid-Atlantic U.S. using a Bayesian
geostatistical model
Why stream sulfate concentration?
– Indirectly toxic to fish and aquatic biota
• Decrease in streamwater pH
• Increase in metal concentrations (AL)
– Observed positive spatial relationship with
atmospheric SO4-2 deposition
(Kaufmann et al 1991)
Wet Atmospheric Sulfate Deposition
http://www.epa.gov/airmarkets/cmap/mapgallery/mg_wetsulfatephase1.html
The Data
• MAHA/MAIA water chemistry data
– 644 stream locations
• Watershed variables:
– % forest, % agriculture, % urban, % mining
– % within ecoregions with high sulfate
adsorption soils
• National Atmospheric Deposition
Program
MAHA/MAIA Stream Locations
Map of NADP and MAHA/MAIA Locations
MAHA/MAIA
NADP
Sketch of watershed with overlaid
landcover map
Forest
Mining
Urban
Agriculture
Bayesian Geostatistical Model
Y ( s )  X ( s )   ( s )   ( s )
(1)
Where Y(s) is observed ln(SO4-2) concentration at stream locations
X(s) is matrix of watershed explanatory variables
 is vector of regression coefficients
Z ( s) ~ N n (0, 2 ( ))
( )  exp( D)
 ( s) ~ N n (0,  2 )
Where D is matrix of pairwise distances,
 is 1/range,
2 is the partial sill
2 is the nugget
Bayesian Geostatistical Model
Priors:
~Np(0,h2I)
~Uniform(a,b)
1/2 ~ Gamma(g,h)
1/2 ~Gamma(f,l)
(Banerjee et al 2004, and GeoBugs documentation)
Semi-Variogram of ln(SO4-2)
1.0
Range
0.6
0.4
Nugget
0.2
Semi-Variogram
0.8
Partial Sill
0
100
200
Distance (km)
300
400
500
Results using Winbugs 4.1
• n=644
– tried different covariance functions
– only exponential without a nugget worked
– computationally intensive
• 1000 iterations took approx. 2 1/4 hours
New Objective:
Why is this not working?
Large N problem?
Possible solutions:
SMCMC: ‘accelerates convergence by simultaneously
updating multivariate blocks of (highly correlated) parameters’
(Sargent et al. 2000, Cowles 2003, Banerjee et al 2004 )
   (1/range) did not converge
subset data to n=322
SMCMC & Winbugs:
  still did not converge and posterior intervals for all
parameters dissimilar
Is the problem the prior specification?
Investigated sensitivity to priors
Original Priors:
~Np(0,h2I)
~Uniform(a,b)
1/2 ~ Gamma(g,h)
1/2 ~Gamma(f,l)
 : Tried Gamma and different Uniform distributions (Banerjee et al
2004, Berger et al 2001)
– Variance components: Tried different Gamma distributions, halfCauchy (Gelman 2004)
Is the problem the presence of a
nugget?
• Simulations:
–
–
–
–
RandomFields package in R
Using MAHA coordinates (n=322)
Constant mean
Exponential covariance with and without a
nugget
– Prior Sensitivity (Berger et al. 2001, Gelman 2004)
Posterior Intervals for 
Using Different Priors
Prior ~Uniform (4,6)
Exponential without Nugget
Exponential with Nugget
3
4
5
Posterior Interv al
6
7
Prior ~Uniform (0,100)
Exponential without Nugget
Exponential with Nugget
0
20
40
60
Posterior Interv al
80
100
Posterior Intervals for Partial Sill
Using Different Priors for 
E
x
p
o
n
e
n
t
i
a
l
w
i
t
h
o
u
t
N
u
g
g
e
t
E
x
p
o
n
e
n
t
i
a
l
w
i
t
h
N
u
g
g
e
t
Prior ~Uniform (4,6)
Prior ~ Uniform (0,100)
0
.
0
0
.
5
1
.
0
P
o
s
t
e
r
i
o
r
I
n
t
e
r
v
a
l
1
.
5
2
.
0
Is the Spatial Signal too weak?
• Simulations were using nugget/sill = 2/3
• Try using a range of nugget/sill ratios
• Previous research:
– Mardia & Marshall (1984): spherical with and without nugget
– Zimmerman & Zimmerman (1991): R.E.M.L vs M.L.E. for
Exponential without nugget
– Lark (2000): M.O.M. vs M.L.E. for spherical with nugget
Is the Spatial Signal too weak?
= 10 and  = 2.5
nugget
0.3
1
1.5
2
2.7
partial sill ratio nugget:sill
2.7
1/10
2
1/3
1.5
1/2
1
2/3
0.3
9/10
100 realizations each combination
Simulation Results for 10
Bias for ML and REML Estimates
M
B
L
ia
E
s
sti
R
m
E
b
i
a
s
-2 -1 0 1
b
i
a
s
-2 -1 0 1
Bi as
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
r at
io
n u g gr
e
a
t
:
t
s
io
illn u g
M
B
L
ia
E
s
sti
R
m
E
b
i
a
s
0 20 40 60 80 10
b
i
a
s
-2 0 2 4 6 8
Bi as
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
r at
io
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
n u g gr
e
a
t
:
t
s
io
illn u g
Simulation Results for 10
Bias for ML and REML Estimates
M
B
L
ia
E
s
sti
R
m
b
i
a
s
0 10 20 30
b
i
a
s
0 10 20 30 40
Bi as
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
r at
io
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
n u g gr
e
a
t
:
t
s
io
illn u g
M
B
L
ia
E
s
sti
R
m
b
i
a
s
-1.5 -0.5 0.5 1.5
b
i
a
s
-1.5 -0.5 0.5 1.5
Bi as
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
r at
io
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
n u g gr
e
a
t
:
t
s
io
illn u g
Simulation Results for =2.5
Bias for ML and REML Estimates
M
Bias
L
Est
RE
im
b
i
a
s
-2 -1 0 1
b
i
a
s
-2 -1 0 1
Bias
0.1
0.33
0.5
0.66
0.9
rati o
nugget:si
rati o
llnugg
M
Bias
L
Est
RE
im
b
i
a
s
0 10 20 30
b
i
a
s
-2 -1 0 1 2
Bias
0.1
0.33
0.5
0.66
0.9
0.1
0.33
0.5
0.66
0.9
rati o
0.1
0.33
0.5
0.66
0.9
nugget:si
rati o
llnugg
Simulation Results for =2.5
Bias for ML and REML Estimates
M
B
L
ia
E
s
sti
R
m
E
b
i
a
s
0 10 20 30 40
b
i
a
s
0 20 40 60
Bi as
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
r at
io
n u g gr
e
a
t
:
t
s
io
illn u g
Bi as
M
B
L
ia
E
s
sti
R
m
E
b
i
a
s
-3 -2 -1 0 1 2 3
b
i
a
s
-3 -2 -1 0 1 2 3
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
r at
io
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
n u g gr
e
a
t
:
t
s
io
illn u g
Conclusions
• Covariance Model Selection Problem
• ML, REML, Bayesian Estimation
(Harville 1974)
• Infill Asymptotic Properties of M.L.E.:
– Ying 1993: Ornstein-Uhlenbeck without nugget 2-dim.;
lattice design
– Chen et al 2000: Ornstein-Uhlenbeck with nugget; 1-dim.
– Zhang 2004: Exponential without nugget; found increasing
range more skewed distributions
Simulation Results for =2.5
Bias for ML and REML Estimates
M
B
L
ia
E
s
sti
R
m
b
i
a
s
0 20 40 60 80
b
i
a
s
0 20 40 60 80 10
Bi as
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
r at
io
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
n u g gr
e
a
t
:
t
s
io
illn u g
M
B
L
ia
E
s
sti
R
m
b
i
a
s
0 10 20 30
b
i
a
s
0 10 20 30 40
Bi as
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
r at
io
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
n u g gr
e
a
t
:
t
s
io
illn u g
Simulation Results for =2.5
Bias for ML and REML Estimates
REM
Bias
L
RE
Es
b
i
a
s
0 20 40 60 80
b
i
a
s
0 5 10 15
Bias
0.1
0.33
0.5
0.66
0.9
rati o
nugget:si
rati o
llnug
REML
b
i
a
s
-2 0 2 4 6 8 10
Bias
0.1
0.33
0.5
0.66
0.9
0.1
0.33
0.5
0.66
0.9
rati o
nugget:si ll
Es
Simulation Results for 10
Bias for ML and REML Estimates
M
B
L
ia
E
s
sti
R
m
b
i
a
s
0 20 40 60 80 10
b
i
a
s
0 20 40 60 80 10
Bi as
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
r at
io
n u g gr
e
a
t
:
t
s
io
illn u g
RB
Ei
M
as
L R
EE
s
b
i
a
s
-2 0 2 4 6 8
b
i
a
s
0 10 20 30 40
Bi as
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
r at
io
0.
0
1
.
0
3
.
0
3
5
.
0
6
.
6
9
n u g gr
e
a
t
:
t
s
io
illn u g
Results from SMCMC and Winbugs
Intercept
Forest
Agriculture
Urban
Mine
SO4wetdep
SO4adsorption

nugget
partial sill
95% Posterior Interval
SMCMC n=644
(-.264, 2.735)
(.015, .047)
(.019, .051)
(.028, .069)
(0.129, .202)
(.005, .021)
(-.019, -.01)
(.021, .062)
( .084, .43)
(.537, 1.08)
SMCMC n=322
(-.821, 2.735)
(.014, .053)
(.016, .056)
(.036, .096)
(.131, .236)
(.004, .023)
(-.018, -.008)
(.011, .045)
(.241, .691)
(.266, .872)
Winbugs n=322
(1.997, 11.18)
(-.063, .027)
(-.06, .03)
(-.037, .068)
(.063, .203)
(.002, .019)
(-.015, -.005)
(.001, .02)
(.418, .763)
(.368, 2.894)