EMODnet Chemistry
WP2: Data aggregation and products generation
in the Atlantic Sea: results and criticalities
With IFREMER, IEO, IHPT and RIHMI‐WDC contributions
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Outline
• Data inventory of the second harvesting
• Data aggregation / validation phase : issues and solutions
• Regional data buffer : space and time distribution by
parameter
• DIVA maps production
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Data inventory (1)
Data collected by MARIS robot (28/10/2014) :
Vertical
profiles
13613
Time
Series
99
Updated CDIs
2601
27
2628
Old CDIs
7040
231
7271
23254
357
23611
CDI type
New CDIS
Total
Total
13712
Annual Meeting, June 15-16 2015, Istanbul, Turkey
3/15
Data inventory (1)
Data collected by MARIS robot (28/10/2014) :
CDI type
New CDIS
Vertical
profiles
Time
Series
Total
13613
99
13712
Updated CDIs
2601
27
2628
Old CDIs
7040
231
7271
23254
357
23611
Total
More than twice the first harvesting !
Annual Meeting, June 15-16 2015, Istanbul, Turkey
4/15
Data inventory (2)
First harvesting
Second harvesting
Old
Updated
Enlarged
data coverage
New
Annual Meeting, June 15-16 2015, Istanbul, Turkey
5/15
Data inventory (3)
Distribution by partners
CDI
type
IEO
Ifremer
IHPT
RIHMINODC
IMR
UniHB
New
CDIs
2396
40
0
1670
3
9
Updat
ed
CDIs
5242
0
3677
146
0
0
Old
CDIs
7764
6465
876
2676
33
0
Total
15402
6505
4553
4492
36
9
Annual Meeting, June 15-16 2015, Istanbul, Turkey
6/15
Data inventory (3)
Distribution by partners
CDI type
IEO
Ifremer
IHPT
RIHMINODC
IMR
UniHB
New CDIs
2396
40
0
1670
3
9
Updated CDIs
5242
0
3677
146
0
0
Old CDIs
7764
6465
876
2676
33
0
Total
15402 6505
4553
4492
36
9
Annual Meeting, June 15-16 2015, Istanbul, Turkey
7/15
Data inventory (4)
Distribution by parameters
2nd harvesting
CDI type
Phos
Amon Silicate
Nox
New CDIs
2230
250
762
835
Updated CDIs
2551
1835
2241
2416
Old CDIs
6538
2916
4107
4221
Total
11319
5001
7110
7472
Annual Meeting, June 15-16 2015, Istanbul, Turkey
8/15
Data aggregation and QC
Quality Checks - common rules
•
Use only data with Quality Flag = 0, 1, 2, 6 as initial collection to use only
good data
•
Unit conversion and aggregation (P35 vocabs): mol/l
•
Deletion of empty lines in a station : some levels with no measurements
for the parameter
•
Deletion of lines with negative depth (incorrect missing value)
•
Perform QC with ODV
•

Search for out of “broad range” values with Flag = 1

Set a QC for all measurements with Flag = 0

Set default values with Flag = 0, 1 to Flag = 9

Deal with Flag 6 and zero values
Write a report on QC and send results to MARIS and data providers
Annual Meeting, June 15-16 2015, Istanbul, Turkey
9/15
Data aggregation and QC
Quality checks – problems faced
• Problem of unit conversion : ug/l to umol/l
 e.g. reload russian data and some spanish data
• Problem in CDI updating
 e.g.high spanish Phosphate to delete
• Some of the reported errors to data centres have not
been corrected :
 we cannot just import the new collection in the
collection built up after the first harvesting.
 We have to re-make the whole QC work.
Annual Meeting, June 15-16 2015, Istanbul, Turkey
10/15
Data aggregation and QC
 5 Aggregated Data Collection Qced produced and sent to MARIS :
-
Phosphate : Global_PHOS_collection.txt
Ammonium : Global_AMON_collection.txt
Nox : Global_NOX_collection.txt
Silicate : Global_SLCA_collection.txt
Timeseries of these 4 parameters : Global_nutrients_TS_CDIs.txt
Annual Meeting, June 15-16 2015, Istanbul, Turkey
11/15
Data aggregation and QC
Perform QC with ODV
 Search for out of range values with Flag = 1
 Set a QC for all measurements with Flag = 0
 Set default values with Flag = 0, 1 to Flag = 9
 Deal with Flag 6 and zero values
Annual Meeting, June 15-16 2015, Istanbul, Turkey
12/15
Data aggregation and QC
Perform QC on New Phosphate with ODV
 Search for out of range values with Flag = 1
 Set a QC for all measurements with Flag = 0
 Set default values with Flag = 0, 1 to Flag = 9
 Deal with Flag 6 and zero values
Flag
value
PHOS
Before
QC
PHOS
after
QC
0
1
43
29994
0
29417
2
3
4
19
40
4
19
114
5
0
30100
2
29557
9
total
Annual Meeting, June 15-16 2015, Istanbul, Turkey
13/15
Regional data buffer
Phosphate data
a) Station map
d) Data bins
b) Vertical plot
e) X/Y distribution
Annual Meeting, June 15-16 2015, Istanbul, Turkey
c) Time histogram
f) Season histogram
14/15
Regional data buffer
Nox data
a) Station map
d) Data bins
b) Vertical plot
e) X/Y distribution
Annual Meeting, June 15-16 2015, Istanbul, Turkey
c) Time histogram
f) Season histogram
15/15
Regional data buffer
Silicate data
a) Station map
d) Data bins
b) Vertical plot
e) X/Y distribution
Annual Meeting, June 15-16 2015, Istanbul, Turkey
c) Time histogram
f) Season histogram
16/15
Regional data buffer
Ammonium data
a) Station map
d) Data bins
b) Vertical plot
e) X/Y distribution
Annual Meeting, June 15-16 2015, Istanbul, Turkey
c) Time histogram
f) Season histogram
17/15
DIVA maps production
DIVA maps settings :
settings
Atlantic area
DIVA version
4.6.9 + weighting patch
Time period
10-years running periods with yearly step for the 4 seasons
Time range
1962-2014
Seasons
winter (0103), spring (0406), summer (0709), autumn (1012)
Vertical layers
IODE standards (33 levels)
parameters
PO4, SiO4, Nox
Annual Meeting, June 15-16 2015, Istanbul, Turkey
18/15
DIVA maps production
Problems faced :
•
Some difficulties in installing DIVA on a Window PC via cygwin :
inconsistencies between netcdf and gfortran librairies
•
Handle the different steps to produce an analyse
•
Optimizing the parameter estimation:
 Do not run on the 46 periods – 4 seasons
 chose a decade with high coverage
 Some SN profiles very noisy
 test binning and weighting options
•
Very long time-consuming runs
 reduce the space and time resolution
•
Pb of generation of the big 4D netcdf file from the 3D files
Annual Meeting, June 15-16 2015, Istanbul, Turkey
19/15
DIVA maps production
Phosphate at surface (0m) – summer (0709)
DIVA – 2005/2014
Annual Meeting, June 15-16 2015, Istanbul, Turkey
WOA – 1955/2012
20/15
DIVA maps production
Phosphate at surface (0m) – summer (0709)
DIVA 2005/2014
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Data bins
21/15
DIVA maps production
Silicate at surface (0m) – summer (0709)
DIVA 2002/2011
Annual Meeting, June 15-16 2015, Istanbul, Turkey
WOA – 1955/2012
22/15
DIVA maps production
Silicate at surface (0m) – summer (0709)
DIVA 2002/2011
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Data bins
23/15
Planning for the next weeks
 DIVA maps production
Data extraction
(and weighting)
Parameters
optimization
Run analysis
Internal expert
validation of products
Phosphate
Silicate
Nox
4D Netcdf
files ready
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Planning for the next weeks
 DIVA maps production
Data extraction
(and weighting)
Parameters
optimization
Run analysis
Internal expert
validation of products
Phosphate
Silicate
Nox
4D Netcdf
files ready
If needed
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Planning for the next weeks
 DIVA maps production
Data extraction
(and weighting)
Parameters
optimization
Run analysis
Internal expert
validation of products
Phosphate
Silicate
Nox
4D Netcdf
files ready
If needed
 Work with other parameters : oxygen, chlorophylle, …
Annual Meeting, June 15-16 2015, Istanbul, Turkey
Thanks!
Annual Meeting, June 15-16 2015, Istanbul, Turkey