Appendix III
FishFrame 5.0
Exchange format specification
VMS data
Table of content
TABLE OF CONTENT ......................................................................................................................... 2
INTRODUCTION .................................................................................................................................. 3
DATA FORMATS .................................................................................................................................. 3
DATA TYPES ......................................................................................................................................... 3
FILE FORMATS ...................................................................................................................................... 3
File names ....................................................................................................................................... 4
Example files ................................................................................................................................... 4
RECORD TYPES ..................................................................................................................................... 4
FIELD DEFINITIONS ................................................................................................................................ 5
Missing / optional values ................................................................................................................ 6
XML format consideration .............................................................................................................. 6
EXCHANGE FORMAT FOR COMMERCIAL EFFORT BASED ON MERGING OF VMS
DATA AND LOGBOOK DATA. .......................................................................................................... 7
COMMERCIAL FISHERIES EFFORT STATISTICS RECORD (VE)BASED ON VMS DATA ............................... 7
SPECIESLIST RECORD (VSL) IN COMMERCIAL FISHERIES BASED ON VMS AND LOGBOOK DATA ........... 8
Introduction
This document describes the formats in which VMS (Vessel Monitoring System)
based data can be uploaded to FishFrame. The document is in the context of not just
the VMS data but also the other data types which is hold by FishFrame.
Data formats
Data types
Following data types can be uploaded:
Commercial fisheries (CF):
 CS = Commercial Samplings
Sampling of commercial catches by observers or fishermen at sea, in the port
or at the market.
 CL = Commercial Landing statistics
 CE = Commercial Effort statistics
VMS data (VD):
This data are compiled by merging information derived from GPS (Global
Possition System), catch registration and effort registration.
Trawl surveys (TS):
This follows the ICES DATRAS format. FishFrame receives data uploaded to
DATRAS from ICES HQ on a regular basis. COST accepts the DATRAS format for
uploading of CA records.
Each of these seven data types consists of data of specific record types, the record
types are explained later.
File formats
The data can be uploaded in the following file formats:
 CSV = Comma Separated Values
ASCII-file with the values separated by either semicolon or comma. Decimal
separator should be comma when delimiter is a semicolon and a point when
delimiter is a comma. This format is preferred for Human-to-Server interaction
(manual upload).
 XML = Extendible Markup Language
XML-file following the W3C-standard for exchanging data over the internet
(see http://www.w3.org/XML/). This format is preferred for Server-to-Server
interaction (automated upload).
 ZIP = Zipped CSV or XML file. Only one exchange file per zip-file.
File names
The filename is required to indicate which data type the file contains. This is done by
following the naming convention: Last two letters in the filename should be the data
type i.e. CS, VD, CL or CE. The extension of the file name indicates the format of the
file. Notice that zip-files should also follow the naming conventions. Files that do not
follow this naming convention are not accepted by the system.
It is recommended that the files follow a systematic naming convention, e.g:
The name of the data file must indicate the key variables like: Country, year etc.
following the rules below:
Example of the file name for Sweden 1997 Q1&2 commercial samplings in a
semicolon separated file: “SWE_1997_Q1_Market_CS.csv” or
“BIAS_GFR_2005_HER_AB.csv”.
The only required naming convention is the file extension and the last two letters that
state the data type (e.g. following filenames would be accepted by the system, but
might not be advisable for keeping track of own national data files: “CS.csv” or
“DataOKCL.zip”.
Example files
Example files are available for each file type. The files can be downloaded from
FishFrame under “Tools -> Test files”. Select “UploadValidation”. The files where
“No errors” is in the name are correct example-files. The folder also contains
numerous files with all the possible kinds of errors, these files are for testing purposes
and their use are described in the test documentation.
Record types
Each of the four data types consists of data of specific record types.
CS (Commercial samplings) consists of five different record types:
 TR = Trip
A fishing trip that has been sampled on board or a sample on a market.
 HH = Haul Header
Detailed information about a fishing operation e.g. a haul or a net set.
 SL = Species List
The sortings strata defined by species, catch category etc.
 HL = Haul Length cell
Length frequency in the stratum subsample
 CA = Catch Aged = SMAWL (Sex-Maturity-Age-Weight-Length)
Sex-Maturity-Age-Weight distribution sampled representatively from the
length groups
CL (Commercial Landing statistics) consists of just one record type:
 CL = Commercial Landing statistics
CE (Commercial Effort statistics) consists of just one record type:
 CE = Commercial Effort statistics
VD (VMS data) consists of two record types:
 VE = Commercial Effort statistics
Total effort, total landing weight (all species) and total value (all species) by
high spatial resolution (based on VMS).
 VSL = Species List
Weight and value by species by high special resolution (based on VMS)
The record types are given in a specific hierarchy and order within the data file. The
required order is given below. For convenience when working with the data files it is
recommended to have the records sorted by all the key variable fields.
CS (Commercial samplings):
The structure is nested. Special ordering is required.
The records must be given trip (TR) by trip, with each trip being followed with all the
records belonging to that trip. Within each trip each of the stations (HH) must be
followed by the corresponding specieslists. Each species list record must be followed
by the corresponding length records. The SMAWL records (CA) can be placed either
after the corresponding set of lengths (HL) or at the very end of a trip.
TR is a mandatory record type. It should be followed by at least one HH record.
CL (Commercial Landing statistics):
The structure is simple. There is only one kind of record. No ordering is required.
CE (Commercial Effort statistics):
The structure is simple. There is only one kind of record. No ordering is required.
VD (VMS data)
The structure is nested. Special ordering is required.
The records must be given by each high spatial resolution effort sum (VE), with each
effort sum being followed with a number of Species List records (VSL) each
representing the species caught in that effort sum.
Field definitions
The fields follow the definitions in the tables below. Each table contains the
definitions, lists the basic data checks performed when uploading the data and gives
the field ordering information for CSV-files. For structure of XML-formatted files
please refer to the XSD- and XML-files in Error! Reference source not found..
XSD-files contain complete definitions and the empty xml-files is given as a help in
order to interpret the XSD.
The columns in the tables below are:
 Order: The order in the CSV-line
 Name: A descriptive name of the field. A “*” after the name indicates that the
field is a part of the key.
 Type: The data type. Integer = whole number, String = text string, Dec =
decimal (the number after “Dec”, e.g. “4” in “Dec(4)”, describes the number of
decimals). If more decimals than allowed are given, then FishFrame off the
value.
 Req.: Required information. M = mandatory, O = optional. See text above on
how to handle optional fields without values.
 Basic checks: The range or list of valid values. For code lists, please refer to
Error! Reference source not found. for valid codes (can also be found as an
online report).
 Comments: Additional comments and clarifications.
Column description for exchange format tables below:
Missing / optional values
When information is missing in optional fields, then an empty zero-length string is
added between the comas/semicolons, like the 3 missing information’s in:
“54,,EXPO,,,”.
When information is missing in optional fields, then the attributes are still added, but
with a zero-length value like “”.
XML format consideration
(Skip this if you exchange your data in CSV format)
The XML formats are defined in the XSD schemas. The XSD schemas can be
downloaded from the documentation page under “Help -> Documentation” in the
menu or copied from Error! Reference source not found.. They can also be used for
validating the data file locally before uploading. The validation can be done in a XML
editor like XML-spy or in own software.
The philosophy behind the XML format in FishFrame is that the record types are
elements; their defining unique properties (like country, year, sampling type, project
and trip number for a commercial sampling trip) are attributes on the elements while
the values are sub-elements.
Exchange format for commercial effort based on merging of
VMS data and logbook data.
Commercial fisheries effort statistics record (VE)based on VMS data
Order
1
2
3
4
5
Name
Type
Req.
Basic checks
Record type *
Vessel Flag Country *
String
String
M
M
Code list
Year *
Quarter *
Month *
Area*
Integer
Integer
Integer
String
M
M
M
M
Code list
Code list
Code list
Code list
C-square *
String
M
Code list
Fishing activity category
National *
String
O
Code list
Fishing activity category
European lvl 6 *
String
M
Code list
Fishing hour
Decimal
numeral
decimal
numeral
Decimal
numeral
Decimal
numeral
M
1 to 9999999999
M
1 to 9999999999
M
1 to 9999999999
M
1 to 9999999999
6
7
8
9
10
11
12
13
kW*fishing hour
Tot weight
Tot value
* = The field
Comments
Fixed value VE
ISO 3166-1 alpha-3 codes.
The flag country of the vessel.
1900 to 3000
1 to 4.
1 to 12
Area level 3 (level 4 for Baltic,
Mediterranean, Black Sea) in
the data Collection regulation
(EC, 2008a, 2008b).
0.05x0.05 degree, C-square
reference
XXXX:XXX:XXX:X
Fishing activity category –
National coding system. Bound
to the Nantes matrix level 4 as
children i.e. an alternative level
5+6.
Fishing activity category – Level
6 in the Nantes matrix (SGRN
06-03)
Fishing hour calculated from
VMS data.
Total landings of all species
caught. In kg
Total value of all species
caught. In Euro
is a key field.
The references to code lists are the lists in the regular FishFrame exchange format description (ICES
Cooperative Report 296, 2009)
SpeciesList record (VSL) in commercial fisheries based on VMS and
logbook data
Order
1
2
3
4
5
Name
Type
Req.
Basic checks
Record type *
Vessel Flag
Country *
Year *
Quarter *
Month *
Area*
String
String
M
M
Code list
Integer
Integer
Integer
String
M
M
M
M
Code list
Code list
Code list
Code list
C-square *
String
M
Code list
Fishing activity
category National *
String
O
Code list
Fishing activity
category European
lvl 6 *
Species
String
M
Code list
String
M
Weight
Integer
M
Value
Integer
O
Closed list of
species 1)
1 to
9999999999
1 to
9999999999
6
7
8
9
10
11
12
* = The field
Comments
Fixed value VSL.
ISO 3166-1 alpha-3 codes.
The flag country of the vessel.
1900 to 3000
1 to 4.
1 to 12
Area level 3 (level 4 for Baltic, Mediterranean,
Black Sea) in the data Collection regulation (EC,
2008a, 2008b).
0.05x0.05 degree, C-square reference
XXXX:XXX:XXX:X
Fishing activity category – National coding system.
Bound to the Nantes matrix level 4 as children i.e.
an alternative level 5+6.
Fishing activity category – Level 6 in the Nantes
matrix (SGRN 06-03) see appendix I.
Scientific name in Latin (genus species).
Whole weight in gram. Decimals not allowed.
In Euro. Decimals not allowed.
is a key field.
1) Type 1 species (DCF (ref) plus a fixed number of added species (the final list to be decided
on).