Validation of Coastwatch Ocean Color products
S. Ramachandran, R. Sinha (SP Systems Inc @ NOAA/NESDIS)
Kent Hughes and C. W. Brown (NOAA/NESDIS/ORA, Washington, DC)
NRT QA of Coastwatch products
ABSTRACT: NOAA Coastwatch routinely
Residual analysis
-4.30
6.39
35
-0.53
2.57
57
0.07
7.69
56
1.02
8.19
Balch
234
-1.67
4.10
234
-0.11
1.82
277
-0.30
3.08
282
0.23
2.59
NMFS
121
-0.56
3.18
126
0.17
1.90
221
-1.22
5.26
231
-0.14
3.89
EMAP
7
-3.42
5.83
8
-2.23
3.74
29
4.82
9.22
29
7.30
9.86
CBP
20
-7.35 10.23
29
-4.49
7.48
181
1.62
9.95
191
2.99
9.41
SE
33
-1.67
4.33
33
-0.69
1.47
40
-0.43
1.07
42
-0.30
0.97
GoMex
38
-2.88
6.20
40
-1.66
4.05
55
-1.00
3.23
55
-0.32
2.50
6
-2.06
3.36
6
-0.52
0.72
13
-0.06
1.91
12
0.18
1.88
Grt_lakes
Table 1. Residual mean and sigma values for four algorithms using eight in-situ data
sets.
Great Lakes
Carribean
WC05 ocean %
Extended Pacific
180
100
180
160
90
160
140
80
140
120
70
100
Terra
80
Aqua
120
SWFS
60
SWFS
SWFR
50
Terra
40
100
Aqua
30
40
Conclusions:
NE05 ocean pixels %
SE05 ocean pixel %
60
Aqua
20
10
0
0
0
0
0.2
0.4
0.6
0.8
1
0
0.2
0.4
0.6
0.8
35
50
30
SWFS
40
Terra
30
Aqua
35
SWFS
SWFR
Terra
Aqua
15
10
5
30
0.8
Terra
20
Aqua
15
5
0
0.6
SWFS
25
10
0
0.4
0
1
0.2
0.4
0.6
0.8
0
1
0
0.2
0.4
cloud %
cloud %
0.6
0.8
1
cloud %
Negative nLwr fraction SE05
WC05 negative nLw
NE05 negative nLw
100
140
1
40
20
10
0.8
45
25
20
0.6
NE05 Cloud fraction
40
60
0.4
% ocean pixel
Granule stats - SE05
70
0.2
0.2
ocean %
WC05 cloud stats 90 days
0
0
-20
1
120
90
120
• Issues with negative values for nLw is more prominent for SeaWiFS compared to MODIS and is associated with high
aerosol (affecting atmospheric correction) and suspended sediments in coastal waters.
100
80
70
100
SWFS
80
AQUA
60
80
SWFS
60
TERRA
SWFR
50
AQUA
40
SWFS
40
20
10
0
0
0
0
0.2
0.4
0.6
0.8
0
1
0.2
0.4
0.6
0.8
0
1
0.2
0.4
%
SE05 Absorbing aerosol fraction
140
0.8
1
NE05 Absorbing aerosol
100
140
90
120
0.6
%
neg nLwr fraction
WC05 Absorbing aerosol
• Cloud flags across sensors for each region in overall agreement, yet MODIS has more pixels per scene flagged as cloudy
than in the case of SeaWiFS.
AQUA
30
40
• Overall agreement across all sensors for % of clear ocean pixels in each scene.
60
TERRA
TERRA
20
Fig 4b. Day 189, 2005
with Stumpf algorithm
Terra
60
20
20
20
Fig 4a. Day 189, 2005
with SEADAS OC4v4
SWFS
80
40
ocean %
• It can be inferred from the residual versus the insitu plot how well the algorithm derives the surface chlorophyll over a
given range of possible chlorophyll values. In the case of Rick Stumpf (NOS) algorithm it does very well in the Gulf of
Mexico and Southeast coastal waters.
Westcoast
Freq
Figure 3a. Day 16, 2002
with SEADAS OC4v4
Gulf of Mexico
Fig 10. Time series of the mean of the residuals above calculated for daily composite and their
respective sigma is shown for the different regions
• Sparse sampling of insitu data makes it difficult to validate the large dynamic range of satellite derived values seen for
the North east coast using both the OC4v4 and Rick Stumpf algorithm, though both algorithms seem to underestimate
chlorophyll values in the range greater than 10 mg m-3.
Figure 3b. Day 16, 2002
with Stumpf algorithm
Southeast
Freq
33
Freq
MWRA
Freq
N
Stumpf algorithm
oc2v4
Stumpf
N mean sigma N
mean sigma
120
80
100
100
70
80
AQUA
60
TERRA
40
SWFS
60
SWFS
SWFR
50
AQUA
40
SWFS
80
AQUA
60
TERRA
TERRA
40
30
20
20
Freq
The slope and intercept of the line relating remotely
derived estimates to in-situ measurements was
computed using a Type II, Reduced Major Axis
regression analysis (Fig 2). Type I analysis was not
used because of the uncertainty in both the in-situ
measurements and the remotely derived estimates.
Statistical comparisons of chlorophyll concentration
were based on log-transformed data.
seadas default algorithm
oc2v4
oc4v4
mean sigma N
mean sigma
Freq
Linear regression analysis between in-situ and remotelyderived values of chlorophyll was performed to assess
the accuracy of the NOAA-generated ocean color
products. The satellite-derived value represents the mean
of valid pixels extracted from the 5 x 5 box centered on
the sampling location.
Sample chlorophyll image products created for
the coast watch region using different algorithms
currently available. Figure 3a uses the default
SEADAS OC4v4 with the default atmospheric
correction. Figure 3b uses the atmospheric
correction and bio-optical algorithm developed
by the NOS group (Rick Stumpf et. al.). Fig 4a
and 4b show results from the latest versions
being used.
Atmos corr
chlorophyll
Figure 8. The above Balch data using
the Stumpf atmospheric correction and
chlorophyll algorithm (units: mg m-3)
Freq
Satellite derived values of chlorophyll concentration
were extracted from a 5 x 5 pixel box centered on the
position of contemporaneous in-situ measurements.
SeaWiFS Level L1A files were acquired and processed
using the climatologic ancillary meteorological and
ozone data. A bio-optical and atmospheric correction
algorithm developed by Stumpf as well as the
NASA/GSFC SeaWiFS Project OC4v4 algorithm was
employed to estimate chlorophyll concentrations in all
Coast Watch regions.
Figure 7 . NMFS data (North east) using the
Stumpf algorithm shows overestimation for
low chlorophyll values and underestimation
for higher chlorophyll concentration. (units:
mg m-3)
Fig 9. Shows the different sensor retrievals (Aqua, Terra & SeaWiFS) for the same day for two different
Coastwatch regions. The scatter plots (Aqua vs. Terra, Aqua vs. SeaWiFS, Terra vs. SeaWiFS and the
corresponding residuals are also shown.
Freq
Method of Analysis
Figure 2 . The large scatter makes it difficult to determine
the quality of the model used in deriving the chlorophyll
values. The global fit also shows insufficient discriminating
power between different algorithms as the coastal waters in
different regions of the CONUS have different behavior.
Table 1 lists the residual mean and
sigma values from various algorithms
studied so far for the different insitu data
sets collected along the US east coast.
The Great Lakes, South east coast(SE)
and Gulf of Mexico (GoMex) region
surface chlorophyll values are well
modeled by the Stumpf algorithm. The
OC4v4 does quite well for the North
East coastal waters (MWRA, Balch,
NMFS). In the estuaries (EMAP, CBP)
around the Chesapeake bay the land
contamination makes comparisons
difficult.
Figure 6. Balch data set for the North
east coast using default SEADAS
OC4v4 algorithm (units: mg m-3)
Freq
Bio-optical measurements used in this analysis were
provided by numerous investigators. Cruises were
conducted in all seasons in optically diverse waters
ranging from the extremely shallow and turbid
Pamlico Sound to the deep and clear Sargasso Sea
(Fig. 1). Numerous agencies and programs sponsored
these cruises. Details of data collection and
processing for each cruise can be obtained from
individual Principal Investigators or their publications.
The vast majority of in-situ chlorophyll concentrations
employed in this evaluation were determined
fluorometrically.
Figure 5. Gulf of Mexico region Stumpf
Chlorophyll algorithm residual as
function of insitu (units: mg m-3)
Freq
Sea Truth Data
Products: Histograms, scatter plots, time series of residuals of Chlorophyll, nLw, quality
flags (Individual granule, Daily composite, by region)
Freq
Figure 1. A map of all the insitu chlorophyll data used in
the analysis. The data sample used in the study adequately
represents the US east and west coastal regions. More
samples are required for the Great Lakes region.
An alternate analysis would be to study
the distribution of the residuals. We
define the residual to be the difference
between the insitu value and the satellite
derived value of the pixel closest in
match to the latitude and longitude of the
insitu point. We present below plots of
the residual versus the insitu value for
different sets of ground truth data as well
as for different satellite derived
chlorophyll algorithms used. Any bias in
the data from the ideal mean of zero
shows clearly in such a plot. It even
separates the region where chlorophyll
values are either underestimated or over
estimated by the various algorithms. For
the Gulf of Mexico region it shows that
the Stumpf algorithm adequately models
the in-situ values as the scatter of the
residuals is around the mean of zero (Fig.
5). In all the other regions the
performance is not optimal. Even
SEADAS oc4v4 shows a significant bias
as the mean value has a slope different
from zero (Figs. 6, 7, 8).
Freq
produce Ocean Color products which need
validation. We present results from validation
activities with in situ data match up of
satellite derived values and residual analysis
used in the evaluation of several existing biooptical algorithms designed to estimate
surface chlorophyll concentration from
SeaWiFS data for the U.S. coastal waters. We
also show results from the Near-Real Time
QA of our products from various sensors.
Comparisons of various sensor products is
also presented.
The goal is to be able to make quality assessments for Coastwatch Ocean Color products
in a timely manner to be of use to NOAA managers and other users.
20
10
0
0.1
0.2
0.3
aerosl flagged pixel %
Acknowledgement: We thank the Principal Investigators for providing their in-situ measurements to us. Support for this
project was provided by NOAA/NESDIS funding to C.W.B.
0
0
0
0.4
0.5
0
0.1
0.2
0.3
absorbing aerosol flagged pixels %
0.4
0.5
0
0.1
0.2
0.3
0.4
0.5
aerosol flagged pixel %
Fig 11. Summary statistics on clear ocean pixel % (1st row); Cloud fraction (2nd row) ; negative
nLw fraction (3rd row); Absorbing aerosol fraction (4th row) are shown for the Westcoast,
Southeast and Northeast regions respectively.