Migrating_maris_to_va_eloha

advertisement
Migrating MARIS data to the VA ELOHA database
Erik.Leppo@tetratech.com
2012-05-31
Purpose
The MARIS dataset (in Access) is a data sharing format common to several states and organizations. The
VA ELOHA (Access) database is for calculating metrics for fish and benthic macroinvertebrates (benthos)
for use in the ELOHA process. The purpose of this document is to create a roadmap for migrating data
from MARIS to the VA ELOHA database. This roadmap highlights the major steps of the data migration
process but is not intended as a step by step guide.
Assumptions
1. The VA ELOHA database is based on EDAS but has been modified for VA ELOHA. That is to say
the VA ELOHA database will not have the same structure as other EDAS databases (e.g., the
version of EDAS used by VA DEQ’s probmon program).
2. The MARIS data format is current as of Spring 2011.
a. The process described in this document only covers data to be migrated to the VA
ELOHA database from MARIS. Any non-MARIS data can be added to MARIS and then
added to the VA ELOHA database using the procedures outlined below.
Procedure Overview
1. MARIS data modifications
a. Additional fields are added to the MARIS data tables to aid in the migration.
b. Details below under “MARIS Table Modifications”.
2. From the VA ELOHA database link to the tables of the MARIS database (see “Linking ELOHA
database to MARIS”).
3. Then append the new data as necessary (see “Add MARIS data to ELOHA database tables”).
Data
Data Needed
1. Fish Master Taxa
a. Will need phylogeny and autecological information for new taxa.
b. Will have to resolve name differences. Use combination of Scientific and Common
Name.
c. Translation table is used for new taxa.
2. Station Location Information
a. Projects
3. Sample Collection Information
a. Collection Method
4. Replicate Information
5. Sample Contents (taxa and counts)
Page 1 of 3
MARIS Table Modifications






Table “Entity”
o Take note of the owner/collector/source of the data. Sometimes the same data resides
in multiple data sets.
Table “FISH INFO”
o Add field “StationID_PK” (text, length=50) that was created in Table.LOCATION. This will
be a unique location identifier across all states and will be used in the ELOHA database.
o Add field “CollDate”. Same as “SAMPLE_BEGIN_DATE” but format will be consistent and
required(short date; YYYY-MM-DD). Missing dates must be filled in with placeholder
dates. For example, if the day is missing make it the 1st of the month. If day and month
are missing use “July 04”. If all information is missing use “1776 JUL 04”.
o Add field “SampID”. After adding data to ELOHA database add the ELOHA SampID to
the MARIS data. This field is used to match data in the future and to help identify which
samples were transferred between the two databases.
o Add field “CollectionMethod”. Use the existing ELOHA database table
“lu_CollectionMethod” to add collection methods for each sample. This could be as
simple as an identifier for the organization or more complex, this is up to the user. This
can be accomplished by linking to the ELOHA table so updates or additions, if necessary,
are made there at the same time.
o Add field “TaxaID”. Use the “TaxaID” field that was created in the MARIS table “FISH
SPECIES LOOKUP” that was created in MARIS table “FISH SPECIES LOOKUP”
Table “FISH SPECIES LOOKUP”
o Add field “TaxaID” that is unique to each taxon. Equivalent taxon names in the table will
receive the same “TaxaID” code so this field may have duplicates.
o Add field “TaxaName_Sci”. Consistent database-wide scientific naming convention to
be used. Equivalent to the MARIS field “SCIENTIFIC NAME” but needs to be consistent
across all data in the table.
 Text field, length = 100
o Add field “TaxaName_Com”. Consistent database-wide common naming convention to
be used. Equivalent to the MARIS field “COMMON NAME” but needs to be consistent
across all data in the table.
 Text field, length = 100
Table “LOCATION “
o Add field “StationID_PK” (text, length=50)and create a unique site identifier based on
the existing “WaterID”, “Station_ID”, “State” and “Originator” fields. This identifier
must be unique in the database not just by waterbody, originator, or state. To ensure
all records are unique mark the field as a “Primary Key” in Access with no duplicates.
This can be done as needed in the table or through the addition of a new table that is
linked on the existing 4 fields that make up the primary key. “StationID” is the
convention used in the VA ELOHA database.
Table “State”
o No modifications necessary. This is a look up table only.
Table “WATER CHARACTERISTICS”
o No modifications necessary. This data was not migrated to the ELOHA database.
Page 2 of 3
Linking ELOHA database to MARIS
Open the ELOHA database and import the tables from MARIS using the “link” option. Per user
preference the tables can be prepended with the “_import_MARIS_” to make them easier to distinguish
in the list of tables in Access.
Add MARIS data to ELOHA database tables
The relationships window in the VA ELOHA database can be viewed to get an overview of the structure
of the database and how the tables are associated with one another (Figure 1).

Data from the MARIS table “Location” goes into the corresponding fields in “Stations”. Query
“Import_MARIS_Location2Stations”.

Check master taxa for Fish. Query “Import_MARIS_FishTaxa_QC”. Any new taxa will need to be
added to the table “TaxaMasterFish”. Include the MARIS taxa field “MARIS_FISHSPECIES_ID”.

Append Sample and Replicate information to the tables “Samples” and “Reps”.

Append sample taxa data to the table “Taxa_Samples_Fish”.

The MARIS tables can be removed when done. It is recommended that the linked tables remain
as the linked database can be changed for new data.
Figure 1. Relationships window for VA ELOHA database.
Page 3 of 3
Download