Metadata Whole of Victorian Government Metadata System August 2010 What is Metadata? Metadata is defined as “data about data”. Metadata is structured information that describes data, information, services or other resources. Metadata is the key management mechanism for Victoria’s spatial information environment. Metadata provides fundamental information management tools at three levels. Discovery: enabiling users to locate and evaluate information. Management: enabling custodians to better manage their spatial information. Utilisation: enabling users to access and manipulate information by means of automated / distributed systems. What is the Metadata System? The Metadata System is a single, corporate, Whole of Victorian Government (WOVG) facility for the management of metadata for spatial data and related resources. It is a critical building block in the drive to improve awareness, access and usability of data by all levels of government, agencies, the general public and private enterprise, both in Australia and worldwide. The Metadata System consists of a database and a set of tools that custodians will use to store, manage and share their metadata efficiently. It will support custodians in the management of their data, opening up access to unclassified data assets while keeping classified ones secure. The Metadata System complies with the ANZLIC Profile, an Australian standard for metadata, which is based on the International ISO 19115 Metadata Standard. The Metadata System will be the ‘one stop shop’ for everything you need to know about a dataset: what geographic area it covers, when it was created, the map projection system used, code lists, tables, legends, the user licence, how to get it, how to use it. Metadata for what types of spatial information? Vector Imagery Surface Model Web Mapping Services Web Feature Services Tables Maps Non-geographic resources such as: Charts Textual Documents Research results The repository (information store) will principally hold Victorian Government metadata but DSE is also encouraging third parties such as utilities to share their data through a formal custodianship framework. Their metadata will be included and non Government custodians will have access to the same tools for metadata entry and management. A set of system requirements has been defined which includes all the functionality of the Victorian Government’s current spatial metadata systems but in a more user friendly and cohesive environment that complies with current international standards. The Metadata System will integrate with existing and future business systems such as User Authentication, rather than duplicating their functionality. The new Metadata System will have the openness and flexibility to allow custodians to add functionality to manage specific types of data such as imagery, to create reports or to record project details and quality assurance results. Dynamic to meet changing storages and processing environments. Manage published and unpublished data. Efficient metadata entry The Metadata System will have a simple, intuitive entry tool that makes the process of entry and update of metadata as easy and automated as possible. For example, it will be able to extract contact information from the custodian’s log-in profile, default values from a template and then duplicates another metadata record or inherit information from a parent record. Other elements can be extracted directly from the dataset such as geographic area covered, creation data and the map projection system used. Rule-based checking with meaningful error messages, examples and contextsensitive help will ensure metadata is entered according to the ANZLIC Profile and is consistent with operational business rules. Of course, automatically populated fields can be overwritten by the metadata author but much time will be saved and inconsistencies avoided by assigning, for example, the same attributes to all mineral tenement datasets. The metadata author will be prompted for metadata elements in a logical manner with colour coding to identify obligatory elements. This will allow data collectors such as scientists or archaeologists to create metadata that is meaningful, capturing their knowledge and allowing their data to be reused when they are long gone. There will be a single metadata entry tool which will be user friendly and intelligent. It will offer meaningful defaults, context sensitive help and examples, advanced validation and inheritance from similar metadata records to save retyping. Migration of existing metadata There will be a once-off migration of existing metadata to the new system in which Custodians will be supported by specialists from the Spatial Information Infrastructure division of DSE. Custodians require the migration to the new system to be as seamless as possible so that users experience minimal interruptions and no metadata is lost. After migration, a search for any and all records from the old system will be found in the new. Systems dependant on metadata (e.g. Map Share, Spatial Datamart, etc) can successfully access metadata from the new system. Custodians asked for seamless integration of data management and delivery, so that a user who meets the pre-assigned criteria can acquire data either as physical files or via online web services, without the intervention of the custodian. Reflecting operational needs The Metadata System will support the practical operational needs of the custodian throughout the process of capture, documentation, publication and physical delivery of the data whether it is a group of physical files or an online mapping service. While the delivery process is automated, custodians will be able to track and monitor who has received their data, change access and notify them of changes. The Metadata System will store or have links to Pricing and Supply conditions and user licences to relieve the load on the Custodian and make the delivery of data transparent and automatic when access criteria are met. Data users will receive all the associated information they need to get the most out of the data such as code lists, symbology, legends or Lookup Tables. These may be files that from part of the dataset or series, or a URL link. The system will support the concepts of resources, data products and dataset series. The custodian will be able to create, update or retire metadata for a whole dataset series in a single action. The metadata system will support versioning and archiving. Custodians will be able to extend the system to document and manage resources and processes that are not covered by the ANZLIC Profile including search words that are specific to one department (e.g. a geological thesaurus), alias names for data (“Threatened Flora 100”), attributes or data quality audit results. Trigger events Changes in other business systems will trigger metadata changes. For example when John replaces Anna in the Identity Management System, all metadata records managed by Anna, will be updated with John’s contact details. Planned updates, significant changes such as the realignment of a waterway, or the publication of a new dataset will trigger email notifications to users or interested parties. This will avoid the current reliance on informal communications that is a burden on Custodians. When the Custodian retires a dataset or authorises a new user or group of users to acquire the data, the change in metadata will be automatically reflected in the delivery system (e.g. Spatial Datamart). Tracking, Reporting and Bulk editing The custodian will be able to report on metadata status, getting reminders when metadata is due for review and report on who is using the data. A spreadsheet type of environment will enable the custodian to review, enter or update multiple records at one time. This will make it easy to spot inconsistencies and correct misinformation. All additions, updates or retirements of metadata, datasets, users or groups will be logged. If someone accessed a dataset two years ago and made decisions based on it, the custodian needs to be able to extract the metadata record for that specific version and find the dataset itself even if the dataset has been retired. Search and Discovery The Metadata System will be very user friendly so users can search with Google-like simplicity, retrieve required data or find online mapping services with minimal training. For example they will search from a map or place name (even if that place name is mis-spelt). Results with thumbnail views will be sorted in multiple ways such as ‘best match’ or ‘most recent’. There will be an advanced option to exclude datasets where holes mean there is not data in the area of interest. e.g. a bore dataset covers the whole state but there are not bores in the area of search. Data custodians searching for data to help them or their colleagues will use the same interface to search not only the Victorian repository but other national (e.g. ASADD) and international sites. Authorised portals in Australia (GeoVic, ASDD or Data Search Victoria) and around the world will be able to harvest metadata making it accessible to potential investors, scientists or climate change researchers. This automated harvesting is only possible through adoption of the new ANZLIC Profile standard which also enables metadata to be exported to standard formats such as AGLS, HTML and XML. Managing user access The Metadata System will be a single point of access to all Victorian Government metadata. A user will sign on only once via an established Authentication system, and will not have to re-authenticate to perform different tasks such as metadata entry, reporting or publishing. User permissions, roles and groups will be administered centrally with custodians managing which user groups, including the public, have access to their metadata and data. Access to metadata and access to data are separate issues because a custodian such as a utility may give general access to the metadata for the location of underground pipes or cables but will restrict access to the actual data. Who will use the Metadata system? Users will come from Government departments, private companies, utilities, emergency services, local councils, non government (e.g. Landcare), research, academia and the community generally. They may be members of the public or online Portals that automatically harvest metadata such as Datasearch Victoria, Victorian and Australia Spatial Data Directories; international portals for minerals or climate change research. Users fall into the following types: Public user Accesses the on-line discovery system without a login to find metadata for resources. r.g. Landcare, bushwalking or bird observer groups. Registered user Authorised users who access the on-line discovery system with a login to find a wider number of resources depending on their user groups. e.g. Emergency Services comprising CFA, Melbourne Fire Brigade (MFB), VicPolice. Custodian The party that accepts accountability and responsibility for the data and ensures appropriate care and maintenance of the resource. Establishes quality standards and determines who may access the metadata and data resource. Assigns responsibility for Metadata authoring. Metadata Author Has a good knowledge of the data and creates, updates, imports or retires metadata. These are typically the people who collect and process data such as scientists, researchers and archaeologists. Point of Contact A party who provides “front-line” support for enquiries regarding the resources and how to acquire it. e.g. a data manager or data supplier. Metadata System Administrator Manages the metadata system including tools and repository, assures performance and modify users and groups. Supports users and solves technical problems. Resource Repository Administrator Managers, stores, accesses and delivers resources on behalf of the Custodian. e.g. GIS Data Managers, Data Base Administrator (DBAs), GIS Administrators. Contact Information Spatial Information Infrastructure Department of Sustainability and Environment Phone: 03 8636 2355 Email: Sii.CustodianshipProgram@dse.vic.gov.au For more information about DSE visit: www.dse.vic.gov.au or contact the Customer Services Centre on 136 186.