Attachment 2 CFEV database – Access and management protocol Conservation of Freshwater Ecosystem Values (CFEV) Program Data access and management protocol ‘A strategic framework for statewide management and conservation of Tasmania’s freshwater ecosystem values’ Conservation of Freshwater Ecosystems Values Program Water Assessment Branch Water Resources Division Department of Primary Industries, Parks, Water and Environment July 2010 1 CFEV database – Access and management protocol © Department of Primary Industries, Parks, Water and Environment, July 2010 Published by: Water Resources Division Department of Primary Industries, Parks, Water and Environment GPO Box 44 Hobart Tas 7001 Telephone: (03) 6233 6328 Facsimile: (03) 6233 8749 Email: cfev@dpipwe.tas.gov.au Website: www.dpipwe.tas.gov.au/cfev This document outlines the processes for accessing, maintaining and revising the CFEV database. Citation: DPIPWE. (2010). Conservation of Freshwater Ecosystem Values Program. Data access and management protocol. v1.0. Conservation of Freshwater Ecosystem Values Program. Department of Primary Industries, Parks, Water and Environment, Hobart, Tasmania. Copyright All material published in the report by the Department of Primary Industries, Parks, Water and Environment, as an agent of the Crown, is protected by the provisions of the Copyright Act 1968 (Cwlth). Other than in accordance with the provisions of the Act, or as otherwise expressly provided, a person must not reproduce, store in a retrieval system, or transmit any such material without first obtaining the written permission of the Department of Primary Industries, Parks, Water and Environment. Disclaimer Whilst the Department of Primary Industries, Parks, Water and Environment makes every attempt to ensure the accuracy and reliability of information published in this report, it should not be relied upon as a substitute for formal advice from the originating bodies or Departments. DPIPWE, its employees and other agents of the Crown will not be responsible for any loss, however arising, from the use of, or reliance on this information. i CFEV database – Access and management protocol Conservation of Freshwater Ecosystem Values Program Data access and management protocol v1.0 Contents 1 Purpose ......................................................................................................................... 1 2 Background .................................................................................................................... 1 3 Definitions and interpretations ........................................................................................ 1 4 Data development .......................................................................................................... 1 4.1 Metadata ................................................................................................................................. 2 5 The CFEV database....................................................................................................... 3 6 Users of the CFEV database .......................................................................................... 4 7 Data access arrangements ............................................................................................ 5 7.1 Read only access to the CFEV database ............................................................................... 5 7.1.1 GIS software........................................................................................................................ 5 7.1.2 Corporate interface.............................................................................................................. 6 7.1.3 WIST ................................................................................................................................... 7 7.2 Read/write access to the CFEV database .............................................................................. 7 7.2.1 GIS software........................................................................................................................ 7 7.2.2 Database software .............................................................................................................. 7 7.3 Data requests for individual CFEV data sets .......................................................................... 7 8 User training................................................................................................................... 8 9 Data management.......................................................................................................... 8 9.1.1 Original data cross-checks and field validation ................................................................. 10 10 Updating the CFEV database ....................................................................................... 10 10.1 General approach ................................................................................................................. 10 10.2 Updating protocols ................................................................................................................ 11 10.3 Data processing and analysis ............................................................................................... 11 10.3.1 Quality assurance/Quality control ................................................................................. 12 10.3.2 Documentation of updating procedures ........................................................................ 12 11 Updating CFEV access portals and documentation ...................................................... 12 11.1 Metadata ............................................................................................................................... 12 11.2 Corporate interface ............................................................................................................... 12 11.3 WIST ..................................................................................................................................... 13 ii CFEV database – Access and management protocol 12 References................................................................................................................... 13 Appendix A – Creating a new login for external CFEVCI users ........................................... 15 Appendix B – Connecting to Oracle SQL Developer and GeoRaptor .................................. 17 Appendix C – Data permission forms and Conditions of Use .............................................. 22 Appendix D – Geoprocessing Worksheet ............................................................................ 28 iii CFEV database – Access and management protocol 1 Purpose The purpose of this document is to outline a set of guidelines and principles for the access, management and revision of the Conservation of Freshwater Ecosystem Values (CFEV) database. This protocol is primarily written for the CFEV Program team within the Water Assessment Branch, Department of Primary Industries, Water and Environment (DPIPWE) to ensure a consistent approach to the management of the data to improve quality and integrity of the data. 2 Background DPIPWE have invested a significant amount of time and resources into developing an assessment framework and database to support the conservation and management of Tasmania’s freshwaterdependent ecosystems, through the CFEV Program. Spatial data has been collected and analysed to strategically identify conservation values and management priorities for freshwater ecosystems across the State. The CFEV database (an Oracle Spatial database) stores the assessment results to form an objective decision-making tool for water and land managers. It is supported by various documentation available on the CFEV website at: www.dpipwe.tas.gov.au/cfev. 3 Definitions and interpretations The following definitions apply to this protocol: Custodian: The agency, program or person of a specific data set, recognised as having the responsibility of collection, management and updating of that data set. Program: Data licence: means a document/letter that authorises the use of data for specified purposes, and specifies any restrictions on the use of data. Spatial data: 4 Data development Early in the CFEV Program, it was realised that a large amount of data was to be sourced and collated in order to carry out the CFEV assessment. To start this process, a review of available data sets relevant to freshwater-dependent ecosystems in Tasmania was undertaken. This included evaluating the data sets for relevance to various aspects of the CFEV assessment. An inventory of available data sets was compiled for all of the State’s freshwater ecosystems (rivers, lakes and lentic wetlands, estuaries, saltmarshes, karst and other .groundwater-dependent ecosystems), including information on current water management. The range of environmental and biological data available was summarised and an indication of geographic scope (statewide, regional or patchy coverage), currency of data and an estimate of data reliability were included. Data sets were sourced by the CFEV Program during the course of the statewide audit and conservation evaluation process, or through consultation with members of the CFEV Scientific Working Groups. Data sets were sourced in the following forms: 1. selected categories/values from already existing layers (e.g. TASVEG, GTSpot) 2. point data collected through other existing programs (e.g. Australian River Assessment System (AusRivAS) assessments) 3. published or unpublished point data (in some cases obtained by review of literature, and/or consultation with relevant expert) 4. existing statewide data sets that required minor modification to suit the CFEV objectives (e.g. IBRA tree assemblage map, wetland polygons). In many instances, original data sets were modified to suit the requirements of the CFEV assessment using mapping rules and/or statistical modelling to fill gaps or ensure that the data could be applied across the entire State. 1 CFEV database – Access and management protocol CFEV data were ultimately developed as spatial layers (points, lines or polygons) and/or attribute data. The spatial data layers were created in or converted to the GDA 1994 MGA Zone 55 projection. In some instances, spatial data layers were developed to describe a particular assessment component (e.g. ecosystem component or condition variable) and then intersected with ecosystem spatial unit data layers to create a new attribute in the latter layer. Some data however, were assigned directly to the ecosystem spatial units using GIS rules. Additional data sets, such as data on the conservation value rankings and conservation management priorities, were derived specifically as part of the CFEV assessment. All of the CFEV data were subsequently housed in the CFEV database – a Geographic Information System (GIS). Figure 1. shows a conceptual diagram of the flow of data. Further information on the types of data that was sourced and how various data sets were developed for input into the CFEV assessment (i.e. the spatial unit, catchment, classification, condition, Special Values and Land Tenure Security data) can be found in the CFEV Project Technical Report (DPIW 2008). Raw input data (external sources e.g. NVA, LIST, TASVEG, WIMS, etc.) Raw data store (M: drive) Spatialise raw data Input raw data Source spatial data store (M: drive) MatLab analysis Ecosystem/ catchment attribution CFEV database (TS/TSDEV) Spatial selection algorithm output data store (M: drive) Execute conservation rule sets Run Spatial Selection Algorithm External access (GIS data files, Corporate Interface, WIST Figure 1. Conceptual diagram illustrating the flow of data development in the CFEV assessment. 4.1 Metadata Detailed metadata for each of the CFEV spatial and attribute data sets are provided in the CFEV Metadata report v1.0 (DPIPWE 2010b). This is an evolving document, which is to be updated every time a data set is amended or created. The details provided in the CFEV Metadata report may consist of any (but not necessarily all) of the elements listed in Table 1. Table 1. Elements of the CFEV Metadata. Title The name of the data set developed by the CFEV Project. Where relevant, these names relate to the names given in the CFEV database. Custodian The custodian is the agency who owns the data and is responsible for its management, including data collection, storage, update, maintenance and distribution. 2 CFEV database – Access and management protocol Creator The name of the individual or agency or individual who created the data set. Description A brief summary of the data. Input data Individual source data sets used in the production of the data set or the name (and document reference) of the CFEV spatial data layer(s) used to create the attribute data. Lineage The history of the data set and the steps taken to produce the data set including mapping and modelling rules. Data limitations Known limitations of the data and constraints associated with using and/or interpreting the data. Date created The date when the data was first produced. Scale and coverage Scale: The relationship between the size of an object in the developed data set and its actual size in the real world. Coverage: Geographic extent of the data set. References Any literature or further information relating to the production of the data set. Column heading Name of the column heading within a specific spatial data layer relating to the attribute data (e.g. RS_TYPE). Type of data This details whether the attribute data is categorical or continuous. Number of classes The number of unique classes within the attribute data set. Assigning values to ecosystem spatial units The mapping rules for assigning the attribute data to specific spatial data layers. CFEV assessment framework hierarchy Describes where the attribute data fits in the CFEV assessment framework and what data sets it was used to derive (e.g. Rivers>Statewide audit>Classification). Other comments Any additional information that is related to the data sets, particularly in the use and interpretation of the data. The metadata document (CFEV Metadata_10June10.docx) is currently stored on the M: drive (M:\WRM_Water_Resources\WAP_Water_Assess_Plg\WAP_Shared\CFEV Project\CFEV Database\Documentation). It can be converted to a PDF file and provided to data users on request. The metadata document is also used to provide documentation to users of the CFEV Corporate Interface (CFEVCI) via web-based HTML links. Periodically or whenever the metadata is updated, this document is to be used to update the web links using a purpose-built application (name?) (stored on the Corporate Information Technology (CIT) server). The application converts the document to HTML format, splits it up into individual sections and creates links to the relevant headings in the CFEVCI. The HTML files created from this application are also stored on the CIT server and can be changed directly if required or if the application fails to perform. 5 The CFEV database Spatial and attribute data collected and developed by the CFEV Program has been integrated and is housed in the CFEV database. The database exists as a relational Oracle Spatial database housed on a server maintained by the CIT Unit. Housing all the data in the CFEV database ensures the efficient storage and access of the data. Interrogation of the CFEV database is via three different portals aimed to meet the access needs of a range of users (refer to Section 7 for details) (Error! Reference source not found.). 3 CFEV database – Access and management protocol GIS interface e.g. ArcGIS, MapInfo (password read only access for DPIPWE users) Corporate Interface Web interface Corporate interface (password read only access for professional users) cfev.dpiw.tas.gov.au, WIST (general read only access for public users) water.dpiw.tas.gov.au Web Feature Service (WFS) TSDEV2 (Production?) Web Map Service (WMS) GIS interface ArcGIS geodatabase (password read/write access for CFEV Program staff) WIST Oracle SQL Developer (password read/write access for CFEV Program staff) ArcSDE CFEV Database (Oracle Spatial) Figure 2. Infrastructure set-up for the CFEV database including portals for accessing the data. The CFEV database stores spatial data on all the freshwater-dependent ecosystems as well a set of three nested catchments for the State; the River Section Catchments, sub-catchment and the major catchments. The database also contains descriptive data to provide the user with contextual information to aid the interpretation of the CFEV assessment data. Until recently, only the output data which represented the results of the CFEV assessment was stored in the database (e.g. features within the rivers spatial layer attributed with CFEV assessment data). A recent upgrade of the structure of the database has not only improved the data design by removing (where practicable) redundant data but has also enabled an extensive amount of ‘source’ data to be input into the database. This ‘source’ data includes data sets representing various components and variables for the CFEV assessment (such as biophysical classifications of fluvial geomorphology, macroinvertebrate assemblages, native fish assemblages, etc., and condition variables, e.g. catchment disturbance, nutrient inputs from land use impacts). This data exists as spatial data which illustrates the geographic distribution of the particular variable which was subsequently attributed to the CFEV spatial units (e.g. river sections, waterbodies, wetlands). Inclusion of this ‘source’ data alongside the CFEV base data (e.g. river spatial data, waterbodies spatial data) consolidates the CFEV data to the one location, which allows improved look-up capabilities, as well as better version control and more efficient updating potential. The CFEV database is backed-up periodically by CIT on a weekly basis.. 6 Users of the CFEV database The CFEV database is used by a wide range of statewide organisations and the Tasmanian community as an information tool for identifying which freshwater ecosystems are a priority for conservation management. The Conservation Management Priority – Immediate (CMPI) output is best used for determining the most suitable freshwater sites for targeting on-ground management activities, monitoring programs or covenanting. The Conservation Management Priority - Potential 4 CFEV database – Access and management protocol (CMPP) output is best used for assessing freshwater areas when proposing actions that will potentially change the way in which a catchment is currently managed, as in water management planning, and also dam and other development proposals. From this top level of information, users can drill down to retrieve input data to support the overall CMP rankings (e.g. naturalness, Representative Conservation Value (RCV), Integrated Conservation Value (ICV), Special Values, etc.). While it is the high level information (e.g. ICV, CMP) that the CFEV assessment framework was primarily designed to produce, a user can also make use of individual input data sets for other purposes (e.g. catchment data, condition data, etc.). An initial assessment of potential users of the CFEV data identified a wide range of users with varying data needs. Retrieval of data from the CFEV database is needed at a variety of spatial scales (e.g. statewide, catchment, property, individual spatial unit) and levels of information (e.g. classification, condition assessment or conservation value data). While not exhaustive, groups of users include members from: State government agencies government business enterprises local government councils Australian government agencies non-government environmental organisations agricultural groups Natural Resource Management (NRM) groups community groups the general public. 7 Data access arrangements The CFEV data is accessible via several means. There is read only access to various aspects of the CFEV database for the general public and professional users (both DPIPWE staff and external users). In some instances, a password is required to access the data (see below Sections below for more details). Read/write access is restricted to CFEV Program staff for editing and updating of the CFEV database. In addition to accessing the CFEV database, in some cases, individual data sets from the database (formatted as ESRI shapefiles or MapInfo tables) may also be available subject to data licensing agreements being established with DPIPWE. 7.1 Read only access to the CFEV database Due to the varying needs identified, three options were made available to provide read only access to the CFEV database (see also Figure 2): 1. GIS software (e.g. MapInfo, ArcGIS) linked to the Oracle Spatial database 2. Corporate interface (cfev.dpipwe.tas.gov.au) 3. Water Information System of Tasmania (WIST) (water.dpipwe.tas.gov.au) These portals provide direct access to the CFEV database and as such users will always be viewing the most current data. Interpretive information and a catalogue of CFEV-related literature which support the CFEV database are available on the CFEV website: http://www.dpipwe.tas.gov.au/cfev. 7.1.1 GIS software Users within DPIPWE have read-only, password access, to the CFEV database to import or view spatial and non-spatial data using their existing GIS software (commonly ArcGIS and MapInfo). An advantage of this system is that the spatial data can then be manipulated or analysed with a user’s own data on a related subject and users get access to the entire database including ecosystem base data and the ‘source’ data; disadvantages are that the users require GIS skills to manipulate the data 5 CFEV database – Access and management protocol and access to the descriptive information is not always streamlined (i.e. requires querying look-up tables). This option is limited to users who have internal access to the DPIPWE computer network. ArcGIS To be finalised once user settings have been resolved. MapInfo MapInfo will require tables to be translated before it works. This canbe done by saving a local copy, then coverting files using the “Universal Translator” in MapInfo. This is not a preferred mrethod, and will become rarer as DPIPWE now has an ESRI site license to use ARC info. When using MapInfo to view the data, a user is prompted to create a copy of the table(s) by saving them to a desired location. It is believed this assists data viewing by speeding up the map display. 7.1.2 Corporate interface Access to the CFEV database for professional users is through the CFEV Corporate Interface (CFEVCI) a web-based interface (cfev.dpipwe.tas.gov.au). CFEVCI is ‘purpose-built’ to suit the wide range of uses and queries required by land and water managers in Tasmania. The interface was designed by a panel of selected users, representing a range of people and interest groups requiring access to the CFEV database. Data is navigable at a variety of spatial scales and users will be able to interrogate all the different levels of CFEV assessment information. The main advantages/functions of the corporate interface are described in the following points: While the corporate interface has the look and some functionality of a GIS interface, users do not require technical GIS skills. Users can access high level CFEV outputs (such as CMP, ICV and RCV) as well as drilldown to get specific information on ecosystem character and condition. Access to descriptive data is streamlined, requiring only one or two clicks on a feature to get detailed information, rather than several separate queries of look-up tables. Summaries at various catchment or regional scales provide statistical information (e.g. proportion of rivers within the South Esk Catchment that are ranked as Very High Conservation Management Priority). Quick navigation to sites or regions using eastings/northings or property identity numbers. The user can download customised reports. The CFEVCI provides users with access to spatial and attribute data associated with all the ecosystems; however, the source data is not available via this portal. Users of the CFEVCI can also obtain metadata for individual components of the CFEV assessment by clicking on the relevant headings in the Data View of the interface. Access to the CFEVCI can only be made using a password. All DPIPWE users can use their usual Windows log-in username and password, while external users need to request one from the CFEV Program (email: cfev@dpipwe.tas.gov.au) by providing a brief explanation of how they will use the data. A username and password can be created by CFEV Program staff using JXplorer LDAP Browser, which can be installed by CIT. CFEV Program members administering usernames and passwords for the CFEVCI require specific administration rights for the LDAP database, which can be obtained from CIT. Instructions for creating new users are provided in Appendix A – Creating a new login for external CFEVCI users. 6 CFEV database – Access and management protocol New users should also be added to the ‘Users trained in CFEVCI.xls’ spreadsheet (located at on the departmental M drive) to keep track of which CFEVCI users have also been trained in using the interface (see also Section 8 on user training). 7.1.3 WIST The simplest and easiest way to access the CFEV database is via the Water Information System of Tasmania (WIST); DPIPWE’s access point to comprehensive water-related data sets. This webbased portal (water.dpipwe.tas.gov.au) provides access to information on water flow, water quality, water licensing, aquatic biota, and a range of relevant departmental publications. The CFEV module of this interface provides access to high-level information (conservation management priorities, conservation value rankings (and associated values) and condition variables). Two options exist within WIST that enable a user to: 1. identify significant values of priority freshwater-dependent ecosystems associated with a potential land or water development proposal 2. browse for natural values information relating to specific ecosystem themes at a variety of spatial scales. This portal is aimed at the general public user that requires occasional access to the CFEV data. Due to the size and complexity of the CFEV database, the WIST only provides access to a sub-set of the CFEV data. 7.2 Read/write access to the CFEV database Read/write access is limited to CFEV Program members only. Password access allows staff to view all of the spatial and attribute data in the CFEV database, either using GIS software (ArcGIS) and, for non-spatial data only, Oracle SQL developer. Staff members with the appropriate access privileges, have permission to edit and update the CFEV data (including descriptive data) but cannot make structural changes to the database. These changes can only be made by CIT staff. 7.2.1 GIS software ArcGIS TO BE assessed when available ArcSDE ArcInfo 7.2.2 Database software Oracle SQL Developer Access to non-spatial data of the CFEV database is available to CFEV Program administrators using Oracle SQL Developer which can be installed by CIT. The GeoRaptor add-on is also required to provide better efficiency in viewing some of the data sets. Instructions to install GeoRaptor are provided in Appendix B – Connecting to Oracle SQL Developer. The fields for each data set are listed (and some instances a brief summary of the data is provided under ‘Comments’) in the ‘Columns ‘ tab. The raw data is viewed in the ‘Data’ tab. Using the program to execute queries requires some SQL programming experience. CFEV Program staff also have read/write access to the 7.3 Data requests for individual CFEV data sets Individual data sets in GIS format (ESRI shapefiles or MapInfo tables) can be requested by contacting the CFEV Program. External users – ESRI shapefiles (same data that is in the database but as files) 7 CFEV database – Access and management protocol Process – Condition of use for CFEV and LIST (contact Craig Smith, Senior Client Services Officer (Agreements), Geodata Services, ILS) or agreements – Data requests spreadsheet. NRM (Rhys Stickler, Section Leader (NRM Liaison and Info Management), Land Conservation Branch, RMC) has all data and distributes it to NRM groups and consultants. We are notified. No cost as present but may be introduced to cover costs of preparation of data which would then be put into funding data updates... DocOne 8 User training Main training sessions done. From now on will be done Ad hoc. An Excel spreadsheet of all users that have been trained exists and is updated as people are added to the list.. 9 Data management At each step in the data set development and analysis, procedures were adopted to ensure data quality, consistency of process, transparency and replicability of the analysis. When new fields were added, some boundaries were set in the properties to ensure the correct type of data was entered into the data set (e.g. numbers or text, field lengths, etc.). Different versions of the data sets were tracked using dates in the file names to ensure all members of the project team and GIS officers were working on the most up-to-date data set. Each time a spatial data layer or attribute data set was created, a review was carried out by the CFEV Program team and, if required, by external scientific experts. This was needed to check for consistency in the application of GIS rules and/or errors in the data. Upon detection of any errors, either rules were modified and re-run, or if minor, errors were corrected manually. Where possible, some data sets were also validated against external data sources. All of the attribute tables (or d-base tables (.dbf)) for each ecosystem theme were structured with a unique identifier for each individual spatial unit (e.g. ES_ID = 1, 2, 3, 4, etc.). Over the course of the CFEV project, various attributes were calculated for each ecosystem theme (created as individual attribute data sets) and ultimately combined to create a final ecosystem theme table. Column headers in the table denote each different attribute data set (e.g. ES_BPCLASS = estuary biophysical class). Table 2 shows an example of the final attribute table layout for estuaries. Sub-sets of this data file were used for input to the expert rule systems and the spatial selection algorithm. Data in the ecosystem theme attribute tables were primarily in coded form (either as alpha-numeric tags or numeric scores) and as such, require additional descriptive information for useful interpretation. This information was developed and added to the CFEV database in the form of look-up tables. Continuous condition variables were also put into categories using look-up tables (e.g. if ES_NSCORE = 0.678, which is between ES_NSCORE_MIN_SCORE of 0.61 and ES_NSCORE_MAX_SCORE of 0.85, then ES_NATURALNESS_CATEGORY is Medium). Table 2. Attribute table layout for estuaries. Column heading Attribute description GEOM Spatial component of data set ES_ID Unique estuary identifier ES_AREA Area of estuary (m2) RSC_ID Unique identifier for the River Section Catchment relevant to the estuary ES_MAP 1:25 000 map sheet the estuary is located within ES_EAST Easting (centroid of estuary) ES_NORTH Northing (centroid of estuary) ES_OZEST Ozestuary Number ES_BCLASS Biological class for the estuary ES_PCLASS Physical class for the estuary 8 CFEV database – Access and management protocol Column heading Attribute description ES_NSCORE Naturalness score for the estuary ES_PDENC Human population density (within entire upstream catchment) from Edgar et al. (1999) ES_PDEND Human population density (associated with the immediate area around the estuary) from Edgar et al. (1999) ES_SVDIV The number of different Special Values associated with the estuary ES_OUTSV Number of outstanding Special Values within estuary ES_NONSV Number of non-outstanding Special Values within estuary ES_UNDIFSV Number of undifferentiated Special Values within estuary ES_ICV Integrated Conservation Value class for the estuary ES_RCV Representative Conservation Value class for the estuary ES_LTENSEC Predominant Land Tenure Security for the estuary (accumulated upstream) ES_LTSMAP Composition of Land Tenure Security for the estuary (accumulated upstream) ES_LTS_L Proportional area of low tenure security land in the upstream accumulated catchment ES_LTS_M Proportional area of medium tenure security land in the upstream accumulated catchment ES_LTS_H Proportional area of high tenure security land in the upstream accumulated catchment ES_CMPI1 Priorities for improved conservation management for the estuary (using RCV as input to rules) ES_CMPP1 Priority for protection and maintenance for the estuary (using RCV as input to rules). This applies in the situation where further development is proposed within the catchment which may contribute to a change in aquatic ecological condition or status ES_CMPI2 Priorities for improved conservation management for the estuary (using ICV as input to rules) ES_CMPP2 Priority for protection and maintenance for the estuary (using ICV as input to rules). This applies in the situation where further development is proposed within the catchment which may contribute to a change in aquatic ecological condition or status ES_NRCLASS Naturalness – Representativeness class for the estuary ES_SORTIT Spatial selection algorithm iteration string (A sort in descending order puts all the estuaries in order of selection) ES_RARCL Ranked rarity class of the rarest biophysical class within the estuary ES_CLASSN Classification code of the unit which initiated the selection of the estuary ES_HPOPCL Human population class ES_NAME Name of estuary ES_BPCLASS Biophysical class for the estuary (used in estuaries classification assessment) ES_SELORD Order of selection by the spatial selection algorithm for the estuaries ES_REPCLAS Biophysical class (and iteration score) which is driving the selection of the estuary. An iteration score of 100% denotes the best example for that biophysical class ES_TAFICL Level of conservation significance of the estuary, as assessed by TAFI (Edgar et. al. 1999) ES_RAMSAR Is the estuary listed as an internationally significant wetland on the 'List of Wetlands of International Importance' (Ramsar Convention)? (Yes = 1; No = 0) ES_DIWA Is the estuary listed as a nationally significant wetland in the 'Directory of Important Wetlands of Australia' (DEWHA)? (Yes = 1; No = 0) Since the original development of the CFEV data, CIT have undertaken a restructure of the database to check for and rectify de-normalised and replicate data. In addition, many of the ‘source’ data sets have been entered into the database for safe storage and improved data efficiencies. This will 9 CFEV database – Access and management protocol improve data management and enhance the ability to update the CFEV data in the future. Information on the re-design of the CFEV database is provided in DPIPWE (2010a). 9.1.1 Original data cross-checks and field validation The CFEV data was cross-checked whenever there was relevant data available for a comparison and where time permitted. Appendix 7 of the CFEV Project Technical report (DPIW 2008) summarises the cross checks that were undertaken during the CFEV assessment process. Various types of cross-checking were conducted: cross-checks using external data sets (i.e. data not directly used in the development of the attribute) internal cross-checks (i.e. using the real data that was used to produce the modelled data) acceptance of the validation processes already in place for the adopted data sets (e.g. TASVEG). In addition to data cross-checks, the CFEV Program has undertaken a sub-project to validate the CFEV data in the field in priority Water Management Planning catchments. This provides groundtruthed information on selected significant freshwater-dependent ecosystems to progress Water Management Plans across Tasmania. This work was carried out during 2006-2007 and was funded through the Australian Government Water Fund, under ‘Water Smart Australia’. A series of six reports were produced from this work. These included an overall validation report and appendices (Davies et al. 2007a, Davies et al. 2007b), summaries from the various catchments involved (Davies et al. 2007c, Davies et al. 2007d, Davies et al. 2007e, Davies et al. 2007f), and a document dealing with the creation of a Bayesian Decision Network (BDN) using the data for rivers and wetlands (Davies 2007). The latter document is accompanied by a series of data files and the BDN freeware GeNIe to allow the analysis of different management scenarios. NRM South also initiated a project to ground-truth the CFEV results with respect to NRM issues and priorities in Tasmania, as well as review and cross-check some of the internal rule sets used to produce the CFEV data. The results of this project are documented in a report (Hydro Tasmania Consulting 2008) which is available from NRM South. The recommendations from these two projects will help to refine the CFEV database and increase confidence in the data for land and water managers. 10 Updating the CFEV database 10.1 General approach The design of the CFEV assessment framework and development of the supporting data sets have been produced in a manner that will cater for various revisions as new data becomes available. The first version of the CFEV data was produced and put into the CFEV database by March 2006. Updating and maintaining the CFEV database is imperative for its ongoing adoption and use by various agencies. The CFEV Program of the Water Assessment Branch, DPIPWE, is the custodian of many of the data sets, particularly the output data sets specific to the CFEV assessment (e.g. RCV, CMP, etc.) and as such is responsible for the management and revisions of those data sets. Input data sets developed and managed by other sections or agencies (e.g. TASVEG, GTSpot (now the Natural Values Atlas)) will remain the responsibility of their custodians. Future revision and updating of the CFEV database consists of several elements: Revising and making improvements to the spatial (mapping) data. Making correction to errors in the attribute data that have been identified in the existing data. Updating of existing data based on more current input data (e.g. new Special Value records, changes to land tenure), including re-running of expert rule systems, the spatial selection algorithm and other conservation assessment rule sets. Assessment of potential new data sets for possible incorporation into the CFEV assessment 10 CFEV database – Access and management protocol Reviewing and making changes to existing rule sets and/or algorithms and potentially creating new ones to integrate new data sets into the CFEV assessment. Each of these components should be considered annually (by a newly established expert panel) with the view of developing a work plan for the implementation of specific changes to be carried out during that year. In the first instance, data and recommendations from the ground-truthing projects (see Section 9.1.1) will trigger an initial review and update of the CFEV database. Feedback from current users will also contribute to the revision of the data. In addition to this, data relevant to the CFEV assessment is being collected through various programs and scientific studies, such as those being conducted through DPIPWE and the State’s NRM framework. Many of the tasks to be considered for updating the CFEV database from these resources have been summarised in DPIPWE (2010b). These will be assessed by the expert panel and prioritised based on a set of criteria. As part of the initial revision of the CFEV database (i.e. Version 2) processes should be established to allow for updating of data that needs to occur periodically. Examples might include: The re-development of Special Values and Land Tenure Security data sets based on updated information, and re-running rule sets to produce ICV and CMP outputs on an annual basis. review and re-developing statewide audit input data sets and re-running the spatial selection algorithm and subsequent rule sets to produce a new version of the CFEV database on a 5-yearly basis. Areas of the CFEV assessment process and related data development that could potentially be automated in some way should also be explored. 10.2 Data processing and analysis Drainage and catchment creation Landscape mapper Data needed for input into Landscape mapper for creating/adjusting catchment or drainage are: ?? ?? ?? The field names need to be named accordingly. Expert rules systems Data needed for input into MatLab for the execution of the expert rules systems are: The field names need to be named accordingly. Spatial selection algorithm Data needed for input into the spatial selection algorithm are: The field names need to be named accordingly. 11 CFEV database – Access and management protocol 10.3 Updating protocols Process – update scripts are run overnight? In down times. Errors found by CFEV program or from user feedback should be entered into the CFEV error register (CFEV_error_register.xls located at M://???). List what need to record. Include urgency 10.3.1 Quality assurance/Quality control Quality assurance In general, quality control refers to all actions, procedures, checks and decisions undertaken to ensure the representativeness and integrity of samples, and the accuracy and reliability of analytical results. Quality assurance refers to all actions that monitor and measure the effectiveness of quality control procedures. Quality control Process for checking data before going on server and before going to production Project Officer/GIS consultant – Program manager – test server (to check links) – production server. Project Officer/GIS consultant – Program manager to fill out checking form to track errors found. 10.3.2 Documentation of updating procedures Project Officer/GIS consultant – Program manager to fill out checking form to track errors found. Each task done requires a geoprocessing form (Appendix B) to be filled out and each output data set need to be documented with metadata and a list of all fields and their description. 11 Updating CFEV access portals and documentation 11.1 Metadata Updating of metadata report 11.2 Corporate interface 12 CFEV database – Access and management protocol Insight GIS – separate contract/quote as required...as a software upgrade. 11.3 WIST Any updates to WIST are to be negotiated through CIT. 12 References Davies, P. E. (2007). Bayesian Decision Networks for Management of High Conservation Assets. Report to the Conservation of Freshwater Ecosystem Values Project. Report 6 of 6. Department of Primary Industries and Water, Hobart. 19 pp. Davies, P. E., Cook, L. S. J., Koehnken, L. & Barker, P. (2007a). Conservation of Freshwater Ecosystem Values Validation Report. Report to the Conservation of Freshwater Ecosystem Values Project. Report 1a of 6. Department of Primary Industries and Water, Hobart. 87 pp. Davies, P. E., Cook, L. S. J., Koehnken, L. & Barker, P. (2007b). Conservation of Freshwater Ecosystem Values Validation Report Appendix. Report to the Conservation of Freshwater Ecosystem Values Project. Report 1b of 6. Department of Primary Industries and Water, Hobart. 72 pp. Davies, P. E., Cook, L. S. J., Sloane, T., Koehnken, L. & Barker, P. (2007c). Macquarie River (including Brumbys Creek and Lake River) Catchment: High Conservation Value Asset Descriptions Report. Report to the Conservation of Freshwater Ecosystem Values Project. Report 2 of 6. Department of Primary Industries and Water, Hobart. 194 pp. Davies, P. E., Cook, L. S. J., Sloane, T., Koehnken, L. & Barker, P. (2007d). Meander River Catchment: High Conservation Value Asset Descriptions Report. Report to the Conservation of Freshwater Ecosystem Values Project. Report 3 of 6. Department of Primary Industries and Water, Hobart. 106 pp. Davies, P. E., Cook, L. S. J., Sloane, T., Koehnken, L. & Barker, P. (2007e). South Esk River Catchment: High Conservation Value Asset Descriptions Report. Report to the Conservation of Freshwater Ecosystem Values Project. Report 4 of 6. Department of Primary Industries and Water, Hobart. 112 pp. Davies, P. E., Cook, L. S. J., Sloane, T., Koehnken, L. & Barker, P. (2007f). Montagu and Welcome River Catchment: High Conservation Value Asset Descriptions Report. Report to the Conservation of Freshwater Ecosystem Values Project. Report 5 of 6. Department of Primary Industries and Water, Hobart. 117 pp. DPIPWE. (2010a). Conservation of Freshwater Ecosystem Values Database. Database Design. Conservation of Freshwater Ecosystem Values Program. Department of Primary Industries and Water, Hobart, Tasmania. DPIPWE. (2010b). Conservation of Freshwater Ecosystem Values Database. Review of updating requirements. Conservation of Freshwater Ecosystem Values Program. Department of Primary Industries, Parks, Water and Environment, Hobart, Tasmania. DPIPWE. (2010c). Conservation of Freshwater Ecosystem Values (CFEV) Metadata Report. Conservation of Freshwater Ecosystem Values Program. Department of Primary Industries and Water, Hobart, Tasmania. DPIW. (2008). Conservation of Freshwater Ecosystem Values (CFEV) Project Technical Report. Conservation of Freshwater Ecosystem Values Program. Department of Primary Industries and Water, Hobart, Tasmania. 13 CFEV database – Access and management protocol Hydro Tasmania Consulting. (2008). NAP07 Validation of the Conservation of Freshwater Ecosystem Values (CFEV) Database. NRM North and South, Hobart. 14 CFEV database – Access and management protocol Appendix A – Creating a new login for external CFEVCI users Adding or updating user log in details in the CFEV LDAP directory Installation: Go to: http://www.jxplorer.org/ Click: "Download" from the menu Click: "pre-compiled java package" Click the windows option: "Download (4.6M)" Run the .exe file and follow the instructions. Connection: Open the application once installed (JXplorer LDAP Browser) Click "Connect" in the File menu. Enter the following fields: Port: 389 Host: Ripple Protocol: LDAP v3 Level: User + Password User DN: uid=YOURWINDOWSUSERNAME,ou=adpeople,dc=dpiw,dc=tas,dc=gov,dc=au Password: YOURWINDOWSPASSWORD Click “Save” to create a template name and save settings for future connections. Administration: All external users will be listed under "extpeople". This is where you add and remove users. To add a user: Right click "extpeople" and select "New". A box appears, enter a username in the box called "Enter RDN" after "uid=" (so that it is something like: "uid=someuser" Click OK. Fill in the fields: - cn (Full name) - sn (Surname) - givenName (First Name) - userPassword - and any other fields that you like (no others are necessary but you may wish to do it anyway) Click "Submit" (if you don't click Submit, the user account will not be created) To remove a user: 15 CFEV database – Access and management protocol Right click on their username and select "Delete" NOTE: If different symbols come up against each different user, then disconnect and connect again. 16 CFEV database – Access and management protocol Appendix B – Connecting to Oracle SQL Developer and GeoRaptor An Oracle user has been created. It has no tables of its own, but it has select and update privileges on the cfevmig schema (the CFEV database). The details are as follows: user: cfevup password: eire8UP When you connect to it via arc, you should enter host: tsdev2.dpipwe.tas.gov.au user: cfevup password: eire8UP schema: cfevmig 17 CFEV database – Access and management protocol Installing GeoRaptor 1. In Oracle SQL Developer, click menu item "Help" and "Check for updates". 2. Dialog "Welcome to the Check for updates wizard" Click on button "Next". If the following dialogue box comes up, then click on ‘Use HTTP Proxy Server’ and then ‘OK’?. 18 CFEV database – Access and management protocol 3. Dialog "Check for Updates - Step 2 of 5: Source" Add new Update center. Click on button " Add " and insert values: Name: GeoRaptor 19 CFEV database – Access and management protocol Location: http://georaptor.sourceforge.net/install.xml When new update center is added make sure that GeoRaptor is selected and click "Next". 4. Dialog "Check for Updates - Summary" Click button "Finish". 20 CFEV database – Access and management protocol 5. After restart of SQL Developer, GeoRaptor should be installed. 21 CFEV database – Access and management protocol Appendix C – Data permission forms and Conditions of Use 22 CFEV database – Access and management protocol Department of Primary Industries, Parks, Water & Environment WATER ASSESSMENT BRANCH Hobart GPO Box 44, Hobart, Tasmania, 7001 Launceston PO Box 46, Kings Meadows, Tasmania, 7249 Devonport PO Box 303, Devonport, Tasmania, 7310 Ph 1300 368 550 Web www.dpipwe.tas.gov.au Inquiries: Danielle Hardie Phone: (03) 6233 2627 Fax: (03) 6233 8749 E-mail: Danielle.Hardie@dpipwe.tas.gov.au Our Ref: AW-TT-DM-201695 16 October 2010 [Full name] [Title] [Address 1] [Town State Postcode] Dear [First name], PERMISSION TO USE CONSERVATION OF FRESHWATER ECOSYSTEM VALUES (CFEV) DATA LAYERS This letter is by way of permission to use the requested data layers developed for the CFEV Program, Department of Primary Industries, Parks, Water and Environment, Tasmania. These include: Attribute data (dbf file) Spatial data (shp file) Classification Condition Conservation Value Conservation Management Priority Special Values Rivers Y Y Y Y Y Y Wetlands Y Y Y Y Y Y Waterbodies Y Y Y Y Y Y Saltmarshes Y Y Y Y Y Y Estuaries Y Y Y Y Y Y Karst Y Y Y Y Y Y Groundwater dependent ecosystem Y NA NA NA NA NA NA NA NA NA NA Y Data set Special Values Data set Spatial data (shp file) 23 CFEV database – Access and management protocol Catchments Y Sub-catchments Y River Section Catchments N The CFEV data has been supplied to you in accordance with a mutually beneficial Service Level Agreement between the XXXXXX and the Department of Primary Industries and Water. Conditions of Use of these data layers are attached (including CFEV developed data and data obtained by the CFEV Project from the LIST) which apply to the use of this data by yourself and other members of your organisation for the term of the XXXXXXXXXX Project as discussed. Data must be discarded once this has been completed. You will be notified of any revisions to these data layers for the duration of the CFEV Program and thank you for requesting to include them as part of your project. On acceptance of these conditions, please sign and return this agreement to me and I will forward the requested data. For any queries relating to the requested data layers or the CFEV Program in general, please contact me on 6233 2627. Yours sincerely Danielle Hardie Program Manager Conservation of Freshwater Ecosystem Values Program ACCEPTANCE OF CONDITIONS: _____________________________ Dated: _____________2009 XXXXXXXXXX 24 CFEV database – Access and management protocol CFEV DATA CONDITIONS OF USE July 2009 In this document unless the contrary intention appears: “Data” means and includes Conservation of Freshwater Ecosystem Values (CFEV) products produced by, and supplied to you via Water and Marine Resources Division, Department of Primary Industries, Park, Water and Environment (DPIPWE). “You” or “Your” means and includes the organisation or individual to whom the Data is supplied. By receiving the Data you are granted a non-exclusive, non-transferable licence for the use of the Data and you indicate an acceptance of the following Conditions of Use. 1) All Data supplied is subject to Crown Copyright provisions as defined by the Copyright Act 1968. 2) The Data is to be used internally within your organisation to support the execution of your usual business activities. 3) The Data may be presented on the internet site providing: a) appropriate acknowledgment is made as to its source; and b) security measures are in place to ensure the Data cannot be reconverted to vector format. 4) The Data shall be used in a manner consistent with the Personal Information Protection Act 2004. 5) The Data shall not be on-sold, traded given away or used to prepare a product for commercial gain in any way, without prior written approval be the General Manager, Water Resources Division. 6) The Data received under this Agreement shall not be distributed outside your organisation, except for where the third party assisting or contracted by you requires the Data to perform tasks on your behalf. However a third party cannot use the Data for any other purpose, nor prepare a product for further commercial gain. 7) You must ensure your employees, agents and sub-contractors are aware of and comply with the Conditions of Use of the Data. 8) You may modify the Data and/or combine the Data with your data, however this will not: a) derogate from the ownership of the Data; or b) operate as an assignment of the copyright in the Data; or c) negate the Conditions of Use applicable to the Data. 25 CFEV database – Access and management protocol 9) A Data Licence Agreement must be negotiated to include rights to produce outputs; or on-sell the Data; or prepare a product for commercial gain or receive updates. A Data Licence Agreement is obtainable through the Manager, Geodata Services Branch. 10) Any product or service derived from the Data whether for commercial, non-commercial gains, internally of externally of your organisation must be accompanied by the relevant acknowledgment statement (s). The statements must be in a form visible to the human eye. Acknowledgment statements: Raster Data: (maps and images) Base image by CFEV, © State of Tasmania Vector Data: Base data by CFEV, © State of Tasmania 11) The Data has been acquired from documents from various sources and therefore is recorded and stored at different levels of reliability. If you wish to make decisions based on the Data you should consult with the relevant authorities. 12) DPIPWE does not warrant that the Data is free of errors and defects, therefore you release DPIPWE from all liability whether in contract or tort arising from any defects and errors in the Data. 13) You acknowledge that the Data has not been prepared to meet your individual requirements and that it is therefore your responsibility to ensure that the Data meets your own individual requirements. 14) DPIPWE is under no obligation to provide training or assistance in the use of the Data or associated hardware/software to utilise the Data. 15) Failure to accept or adhere to any of the above Conditions of Use, terminates any granted permission of use of the Data. Consequently all use of the Data must cease and the Data returned to DPIPWE. The Data and any related material and all copies thereof shall be erases from your storage media. This requirements applies to all copies of the Data in any form, partial or complete and whether or not modified or merged into other materials. CONTACTS GENERAL CFEV DATA CFEV DATA ERRORS Any questions regarding the Conditions of Use or Crown Copyright should be directed to: Any questions regarding CFEV products may be emailed to: Any errors within the CFEV products may be emailed to: cfev@dpipwe.tas.gov.au cfev@dpipwe.tas.gov.au Program Manager (CFEV) Water Assessment Branch Water and Marine Resources Division 13 St Johns Avenue NEW TOWN TAS 7008 26 CFEV database – Access and management protocol CONDITIONS OF USE April 2006 In this document unless the contrary intention appears: ‘Data’ means and includes LIST and/or TASMAP products supplied to you via the Information & Land Services Division (ILS) of the Department of Primary Industries and Water (DPIW). By receiving the Data, Client Name (‘you’), are granted a non-exclusive, non-transferable licence to use the Data and indicate your acceptance of the following Conditions of Use. 1) All Data supplied is subject to Crown Copyright provisions as defined by the Copyright Act 1968. 2) The Data is to be used internally within your organisation to support the execution of your usual business activities. 3) The Data may be used on up to 3 single user computers or a server with up to 3 interfaces to the Data to carry out the function described at Item 2, unless otherwise approved in writing by the Manager, Geodata Services Branch, ILS. 4) The Data may be presented on an internet site providing: a) appropriate acknowledgment is made as to its source; and b) security measures are in place to ensure the Data cannot be reconverted to vector format. 5) The Data shall be used in a manner consistent with the Personal Information Protection Act 2004. 6) The Data shall not be on-sold, traded, given away or used to prepare a product for commercial gain in any way, without prior written approval by the General Manager, Information & Land Services Division. 7) The Data received under this Agreement shall not be distributed outside your organisation, except for where a third party assisting or contracted by you requires the Data to perform tasks on your behalf. However a third party cannot use the Data for any other purpose, nor prepare a product for further commercial gain. 8) You must ensure your employees, agents and sub-contractors are aware of and comply with the Conditions of Use of the Data. 9) You may modify the Data and/or combine the Data with your data, however this will not: a) derogate from the ownership of the Data; or b) operate as an assignment of the copyright in the Data; or c) negate the Conditions of Use applicable to the Data. 10) A Data Licence Agreement must be negotiated to include rights to produce outputs; or on-sell the Data; or prepare a product for commercial gain or receive updates. A Data Licence Agreement is obtainable through the Manager, Geodata Services Branch, ILS. 11) Any product or service derived from the Data whether for commercial or non-commercial gains, internally or externally of your organisation must be accompanied by the relevant acknowledgment statement(s). The statements must be in a form visible to the human eye. Acknowledgment Statements: Raster Data: Base image by TASMAP, © State of Tasmania Vector Data: Base data from theLIST, © State of Tasmania 12) The Data has been acquired from various sources and therefore is recorded and stored at different levels of reliability. Accordingly DPIW warrants the Data only within the performance criteria specified within the Metadata statements found at the Tasmanian Spatial Data Directory, and via the information sheets accompanying the Data. If you wish to make decisions based on the Data you should consult with the relevant authorities. 13) DPIW does not warrant the Data is free of errors or defects, therefore you release DPIW from all liability whether in contract or tort arising from any defects and errors in the Data. 14) The Data has not been prepared to meet your individual requirements and it is therefore your responsibility to ensure that the Data meets your own individual requirements. 15) DPIW is under no obligation to provide training or assistance in the use of the Data or associated hardware/software to utilise the Data. 16) Failure to accept or adhere to any of the above Conditions of Use, terminates any granted permission of use of the Data. Consequently all use of the Data must cease and the Data returned to DPIW. The Data and any related material and all copies thereof shall be erased from your storage media. This requirement applies to all copies of the Data in any form, partial or complete and whether or not modified or merged into other materials. 27 CFEV database – Access and management protocol Appendix D – Geoprocessing Worksheet Date Input/s Process Comment/Result Output/s Where stored? Checking? Any further issues? Date task/process was undertaken Name and formats of input data files Individual steps that were carried out. What was the result? Are there any issues? Name of output files. Location (either database or location drive) of the output data Does this step require checking and if so, what? And who by? Any issues that arose from the checking process. E.g. 31/08/06 cfev_a_gouldi_main _catchments (ESRI shapefile provided by Peter Davies) Check projections. Need adjusting – change to GDA_1994_MGA_Z one_55 a_gouldi_main_ catchments_gda 94 D:\Other data\Crayfish_clas ses_Aug06 CFEVRivers_crayfis h_310806 (ESRI shapefile – new copy of CFEVRivers to be used for this process) CFEVRivers_crayfis h_310806 OK crayfish_310806 _gda94 crayfish_final (ESRI shapefile) A_gouldi_main_catc hment_gda94 Merge all boundaries of catchments (Edit, select and merge polygons) crayfish_310806_g da94 Edit polygons so that they covers the north west Results in two polygons- one in the north-west and one in the northeast A_gouldi_main_ catchment_gda9 4 crayfish_310806 _gda94 28 CFEV database – Access and management protocol 29