Migrating MARIS data to the VA ELOHA database Erik.Leppo@tetratech.com 2012-05-31 Purpose The MARIS dataset (in Access) is a data sharing format common to several states and organizations. The VA ELOHA (Access) database is for calculating metrics for fish and benthic macroinvertebrates (benthos) for use in the ELOHA process. The purpose of this document is to create a roadmap for migrating data from MARIS to the VA ELOHA database. This roadmap highlights the major steps of the data migration process but is not intended as a step by step guide. Assumptions 1. The VA ELOHA database is based on EDAS but has been modified for VA ELOHA. That is to say the VA ELOHA database will not have the same structure as other EDAS databases (e.g., the version of EDAS used by VA DEQ’s probmon program). 2. The MARIS data format is current as of Spring 2011. a. The process described in this document only covers data to be migrated to the VA ELOHA database from MARIS. Any non-MARIS data can be added to MARIS and then added to the VA ELOHA database using the procedures outlined below. Procedure Overview 1. MARIS data modifications a. Additional fields are added to the MARIS data tables to aid in the migration. b. Details below under “MARIS Table Modifications”. 2. From the VA ELOHA database link to the tables of the MARIS database (see “Linking ELOHA database to MARIS”). 3. Then append the new data as necessary (see “Add MARIS data to ELOHA database tables”). Data Data Needed 1. Fish Master Taxa a. Will need phylogeny and autecological information for new taxa. b. Will have to resolve name differences. Use combination of Scientific and Common Name. c. Translation table is used for new taxa. 2. Station Location Information a. Projects 3. Sample Collection Information a. Collection Method 4. Replicate Information 5. Sample Contents (taxa and counts) Page 1 of 3 MARIS Table Modifications Table “Entity” o Take note of the owner/collector/source of the data. Sometimes the same data resides in multiple data sets. Table “FISH INFO” o Add field “StationID_PK” (text, length=50) that was created in Table.LOCATION. This will be a unique location identifier across all states and will be used in the ELOHA database. o Add field “CollDate”. Same as “SAMPLE_BEGIN_DATE” but format will be consistent and required(short date; YYYY-MM-DD). Missing dates must be filled in with placeholder dates. For example, if the day is missing make it the 1st of the month. If day and month are missing use “July 04”. If all information is missing use “1776 JUL 04”. o Add field “SampID”. After adding data to ELOHA database add the ELOHA SampID to the MARIS data. This field is used to match data in the future and to help identify which samples were transferred between the two databases. o Add field “CollectionMethod”. Use the existing ELOHA database table “lu_CollectionMethod” to add collection methods for each sample. This could be as simple as an identifier for the organization or more complex, this is up to the user. This can be accomplished by linking to the ELOHA table so updates or additions, if necessary, are made there at the same time. o Add field “TaxaID”. Use the “TaxaID” field that was created in the MARIS table “FISH SPECIES LOOKUP” that was created in MARIS table “FISH SPECIES LOOKUP” Table “FISH SPECIES LOOKUP” o Add field “TaxaID” that is unique to each taxon. Equivalent taxon names in the table will receive the same “TaxaID” code so this field may have duplicates. o Add field “TaxaName_Sci”. Consistent database-wide scientific naming convention to be used. Equivalent to the MARIS field “SCIENTIFIC NAME” but needs to be consistent across all data in the table. Text field, length = 100 o Add field “TaxaName_Com”. Consistent database-wide common naming convention to be used. Equivalent to the MARIS field “COMMON NAME” but needs to be consistent across all data in the table. Text field, length = 100 Table “LOCATION “ o Add field “StationID_PK” (text, length=50)and create a unique site identifier based on the existing “WaterID”, “Station_ID”, “State” and “Originator” fields. This identifier must be unique in the database not just by waterbody, originator, or state. To ensure all records are unique mark the field as a “Primary Key” in Access with no duplicates. This can be done as needed in the table or through the addition of a new table that is linked on the existing 4 fields that make up the primary key. “StationID” is the convention used in the VA ELOHA database. Table “State” o No modifications necessary. This is a look up table only. Table “WATER CHARACTERISTICS” o No modifications necessary. This data was not migrated to the ELOHA database. Page 2 of 3 Linking ELOHA database to MARIS Open the ELOHA database and import the tables from MARIS using the “link” option. Per user preference the tables can be prepended with the “_import_MARIS_” to make them easier to distinguish in the list of tables in Access. Add MARIS data to ELOHA database tables The relationships window in the VA ELOHA database can be viewed to get an overview of the structure of the database and how the tables are associated with one another (Figure 1). Data from the MARIS table “Location” goes into the corresponding fields in “Stations”. Query “Import_MARIS_Location2Stations”. Check master taxa for Fish. Query “Import_MARIS_FishTaxa_QC”. Any new taxa will need to be added to the table “TaxaMasterFish”. Include the MARIS taxa field “MARIS_FISHSPECIES_ID”. Append Sample and Replicate information to the tables “Samples” and “Reps”. Append sample taxa data to the table “Taxa_Samples_Fish”. The MARIS tables can be removed when done. It is recommended that the linked tables remain as the linked database can be changed for new data. Figure 1. Relationships window for VA ELOHA database. Page 3 of 3