Training Workshop – Ostende – 1-4 March 2010
Training course – Ostende – 2-6 July 2012
Practical work on NEMO
Converting files
M. Fichaut
Training course – Ostende – 2-6 July 2012
From ASCII data to XML files
Collection
of
ASCII files
ODV
files
XML
CDI files
summary_CDI_NEMO.xml
Delivered with NEMO
Coupling table
Export
CDI summary
CSV file
Coupling table
sdn-userdesk@seadatanet.org – www.seadatanet.org
MIKADO
SeaDataNet
CDI
Training course – Ostende – 2-6 July 2012
NEMO installation
•
•
•
•
•
Open the install_nemo_1.4.4_windows.zip file (on the
desktop)
Unzip it
Double click on launcher_nemo.bat
Follow the installation procedure
Don’t forget to add the NEMO shortcut on your desktop
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
Collection of ASCII files
•
Training data set
– 6 CTDs raw files under
SDN tools/Practical work/data/input files
– ASCII files with measured parameters in
column
– Measured parameters = Depth, Temperature,
Salinity, Fluorescence and Dissolved oxygen
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
First step : conversion to ODV with NEMO
• Run NEMO to convert the files to ODV format.
1. Modify the settings of NEMO
• Default directories for data set to
SDN tools/Practical work/data
– Default directories for data set to
SDN tools/Practical work/models
– Enter your EDMO ID in the field below using the
search function (right click)
2. Proceed the 4 steps of NEMO
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
NEMO – File description
•
•
•
•
•
•
•
•
Choose the type of file (cruise directory, cruise file, …)
Browse your PC to input the data files directory
Once the file is open in NEMO, have a look to its content
Choose file type : Profile
Choose the output format : ODV, one unique file for all
stations or One file per station
Enter the Station header information
– there is a specific character string at the end of each
Station header : *END*
The Data termination indicator is EOF (End of file)
Validate the step when description is OK
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
NEMO – Station description
•
•
•
•
•
•
•
•
•
Station number is given on line “**Station”
Data type is ‘CTD stations’
Time is U.T.
Latitude format is NDD MM.mm
Longitude format is NDDD.MM.mm
Bottom Depth is not available
Input all the mandatory fields : station number, time, date,
latitude, longitude
Use the Test function to verify your input
Validate the step when description is OK
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
NEMO – Data description (1)
•
•
•
•
Select the parameter list to use : P011 via P021
Add the parameters : P021 codes are
– Depth : AHGT, Temperature : TEMP, Salinity :
PSAL, Fluorescence : FVLT, Dissolved oxygen :
DOXY, Input the units of the parameters (for
salinity, use Dimensionless)
– Add a text label to each parameter
Set Start and End positions for the parameters
Tell NEMO which code is the vertical reference
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
NEMO – Data description (2)
•
•
•
•
Format is mandatory
– Describe the output format that you want
– %6.1f : means 6 digits total (decimal point included),
1 digit for decimal, f for float (decimal number)
– Use the contextual menu ‘Select the format’ for
predefined format
Use the test fields to validate your input
Validate the step when description is OK
Save you model in the model directory
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
File conversion
•
•
•
•
•
You will be asked to give :
A dataset reference (can be a cruise name, for
example)
A name for the output file (all the stations in
the same file) or name for the output
directory (one station per file) in the
SDN tools\Practical work\output directory
Have a look to the output files
Drag and drop the ODV files in ODV to test
their conformity
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
Practical work on NEMO
Generating CDI summary and coupling table
M. Fichaut
Training course – Ostende – 2-6 July 2012
From ASCII data to XML files
Collection
of
ASCII files
ODV
files
XML
CDI files
summary_CDI_NEMO.xml
Delivered with NEMO
Coupling table
Export
CDI summary
CSV file
Coupling table
sdn-userdesk@seadatanet.org – www.seadatanet.org
MIKADO
SeaDataNet
CDI
Training course – Ostende – 2-6 July 2012
Creation of the CDI-SUMMARY and the
coupling table
•
Input the information in NEMO SETTINGS
– Mapping
• Click in the box Generate mapping for SeaDataNet
download Manager
– SeaDataNET CDI summary
• Unfold the SeaDataNet CDI Summary block
• Click in the box Generate SeaDataNet CDI summary
• Fulfill all the mandatory fields (in red), using right click for
the Search function
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
Run the file conversion
•
•
•
•
•
•
Same file than in the previous exercise
You will be asked to give :
A dataset reference (can be a cruise name, for example)
A name for the CDI summary file
A name for the output file (all the stations in the same file)
or name for the output directory (one station per file)
An output directory prefix (for the coupling table). This
prefix will be subtracted from the file name in the mapping
table.
For example if
– output file name =
C:\username\NEMO\cruise_name\file_name
– and output directory prefix = C:\username\NEMO
– file name in the mapping table will be :
cruise_name\file_name
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
CDI summary file
•
Convert it to an EXCEL file (for future use by MIKADO)
– Open the CDI summary text file with EXCEL (field
delimited by tabulations)
– Rename the folder into : STATION
– Save it under XLS or XLSX format, it will be used
later on during the practical work on MIKADO
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
Coupling table
•
•
Have a look at the coupling table
– Menu : Coupling_table>Edit
– It gives the link between the LOCAL_CDI_ID and the file in
which the LOCAL_CDI_ID will be found by the download
manager of SeaDataNet
– MODUS = 1 if the LOCAL_CDI_ID is in a mono-station file
– MODUS = 3 if the LOCAL_CDI_ID is in a multi-station file
– One LOCAL_CDI_ID and one format is unique in the coupling
table
– Add and delete records
Export the coupling table in a flat file
– Menu : Coupling_table>Export
– Have a look at the flat coupling.txt file
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training Workshop – Ostende – 1-4 March 2010
Training course – Ostende – 2-6 July 2012
Practical work on MIKADO
Creation of XML files for CDI
M. Fichaut
Training course – Ostende – 2-6 July 2012
Exercise 1
a.CDI Automatic generation from
the CDI summary file created this
morning with NEMO – Validation
of the XML files
b.Creation of the corresponding
coupling file
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
From ASCII data to XML files
VALIDATOR
a
b
Collection
of
ASCII files
ODV
files
Coupling
file for
DM
XML
CDI files
summary_CDI_NEMO.xml
Delivered with NEMO
Coupling table
Export
CDI summary
CSV file
Coupling table
sdn-userdesk@seadatanet.org – www.seadatanet.org
MIKADO
SeaDataNet
CDI
Training course – Ostende – 2-6 July 2012
a) Configuration file for CDI XML
•
•
Run MIKADO
Open the configuration file delivered with NEMO :
– MIKADO => Automatic/Open/CDI
the configuration file is in the HOME_DIRECTORY of NEMO,
but for the training it is under \Practical work\config , file
summary_CDI_NEMO.xml
•
•
•
•
•
Make the connection with the Excel file that you have
created with NEMO (the Excel file must be closed to avoid
conflict access)
Check the connection (you must have a green message)
Go to the Queries tag and have a look at the queries
Check them (green message after check all)
Save your configuration file
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
a) XML file generation
•
Run Automatic/Generate/CDI
– Open the configuration file
\Practical work\config\summary_CDI_NEMO.xml
• Choose the export directory (where CDI XML files will
•
be written)
– Choose the type of export file you want :
• XML files
• Zip file containing the XML files
• Both (XML and Zip)
Open one XML export file to have a look at it with MIKADO
Manual>Open>File>CDI
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
a) Web validation service: Validator
•
•
•
•
•
•
Check the XML file with the Web validation Service at :
http://www.seadatanet.org/validator
Select the CDI catalogue
Validate file with CDI Xml schema
Upload one of the XML file that has been generated
Validate the file
Look at the result of the validation : you must have no error,
you can have some warnings depending on the vocabulary
versions.
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
b) Coupling configuration file
•
•
Run MIKADO
Open the configuration file delivered with NEMO :
– MIKADO => Tools/Coupling table for download manager/ Import
CDI configuration
the configuration file is the same than in a) ) in the config directory
•
Check the connection to the excel file
\Practical work\config\CDI_summary_with_address.xls
•
•
•
•
•
Go to the main query tag and check the query (you must have
a green message)
Go to Modus 1 or Modus 3, choose the output format and
fulfill the query which retrieve the file address in the excel file
Check it
Preview the coupling file
Save your coupling
configuration
file
sdn-userdesk@seadatanet.org
– www.seadatanet.org
Training course – Ostende – 2-6 July 2012
b) Coupling file generation
•
Run the generation of the coupling file
– MIKADO => Tools/Coupling table for download manager/
generate/ Modus 1 or Modus 3 depending on your ODV file(s)
•
•
•
•
•
Select the configuration file for coupling that you just create
Select the output directory
Choose the mode for writing
Generate the coupling file
Open it with a text editor
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
Exercise 2
• EDIOS manual input of
– 1 Program
– 1 Series
– 1 Platform
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
Under SDN_tools\practical_work
•
•
Under \EDIOS__POSEIDON_Example\EDIOS_Data
Open the EDIOS_POSEIDON.xls file describing the EDIOS program,
series and platforms (1 program, 15 series,15 platforms)
• Input the program information into MIKADO and create the
corresponding XML file
• Input 1 series information into MIKADO and create the corresponding
XML file
• Input the platform related to the previous series in MIKADO and create
the corresponding XML file
•
Why can’t you first enter a platform, then a series then a
program?
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
Exercise 3
• EDIOS Automatic generation of XML
files for:
– Program
– Series
– Platforms
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
Under SDN_tools\practical_work
•
•
•
\EDIOS__POSEIDON_Example\EDIOS_Data
Open the EDIOS_POSEIDON.xls file describing the EDIOS program, series
and platforms (1 program,15 series, 15 platforms)
Run MIKADO to create the automatic query to retrieve the program
information
– Or open the already written XML configuration for MIKADO for EDIOS program :
\EDIOS__POSEIDON_Example\queries\poseidon_edios_program.xml
•
Run MIKADO to create the automatic query to retrieve the series information
– Or open the already written XML configuration for MIKADO for EDIOS program :
\EDIOS__POSEIDON_Example\queries\poseidon_edios_series.xml
•
Run MIKADO to create the automatic query to retrieve the program
information
– Or open the already written XML configuration for MIKADO for EDIOS platform :
\EDIOS__POSEIDON_Example\queries\poseidon_edios_platform.xml
sdn-userdesk@seadatanet.org – www.seadatanet.org
Training course – Ostende – 2-6 July 2012
Generate the XML EDIOS files
• EDIOS PROGRAM
– Run MIKADO : Automatic>Generate>EDIOS Program
• Choose the \queries\poseidon_edios_program.xml
• Choose the export directory \results\program
• EDIOS SERIES
– Run MIKADO : Automatic>Generate>EDIOS Series
• Choose the \queries\poseidon_edios_series.xml
• Choose the export directory \results\series
• EDIOS PLATFORMS
– Run MIKADO : Automatic>Generate>EDIOS platforms
• Choose the \queries\poseidon_edios_platforms.xml
• Choose the
export directory \results\platforms
sdn-userdesk@seadatanet.org
– www.seadatanet.org