Introducing Geospatial Metadata A Two-day Workshop About Metadata Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---1 What’s Metadata? What Metadata Is and Why You Need to Know About It Topic 1 Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---2 What’s Metadata? Two similar paintings by Picasso sold for vastly different prices Why? One had metadata. One didn’t Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---3 What’s Metadata? Metadata is information about data If you had two cans without labels, which would you eat? Without a label, how would you know which was tuna and which was cat food? Tuna? Cat Food? Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---4 What’s Metadata? We often use metadata without even knowing it Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---5 What’s Metadata? A published map contains the elements of metadata… • Publisher • Publication date • Type of map • Description • Spatial references • Scale and accuracy Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---6 What’s Metadata? Metadata is a critical component of a complete data set Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---7 What’s Metadata? It’s data about a data set Metadata Title Data set GIS files Scale Source Content Location Publication Don’t Imagery Geospatial databases GPS data Access Duck Metadata November 2000 Introducing Geospatial Metadata ---8 What’s Metadata? Metadata is the “stuff” to help you find and understand data — whether the data is a… • • • • • list spreadsheet database CAD drawing GIS map Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---9 What’s Metadata? Metadata helps those who need geospatial data find what they need and determine how to use it Don’t Geospatial data: any data with a spatial component, including maps, charts, in situ data Duck Metadata November 2000 Introducing Geospatial Metadata ---10 What’s Metadata? Metadata describes… content, quality, condition, and other characteristics about data who, what, where, why, and how of a data set Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---11 What’s Metadata? Metadata answers questions… • Who created and maintains the data? • Why were the data created? • What is the content and structure of the data? Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---12 What’s Metadata? Metadata answers questions… • When collected? When published? • Where is the geographic location? Storage location? • How were the data produced? Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---13 What’s Metadata? Metadata includes information you need to… • determine which data exist for a geographic location • determine if a set meets your need • acquire the set you identified • process and use the set Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---14 Why Bother with Metadata? Why Metadata Is Essential for the Long Term Topic 2 Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---15 Why Bother with Metadata? You probably already have metadata in some form: • Manila folders stuffed with notes • Readme files • Hardcopy data catalogs Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---16 Why Bother with Metadata? Documenting data is critical to preserving its usefulness over time Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---17 Why Bother with Metadata? Without proper documentation, no data set is complete Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---18 Why Bother with Metadata? Data users Metadata helps… Organizations Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---19 Why Bother with Metadata? For data developers, metadata helps: Avoid duplication Share reliable information Publicize efforts Reduce workload Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---20 Why Bother with Metadata? For data users, metadata: Facilitates understanding Focuses on key elements Enables discovery — inside and outside organizations Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---21 Why Bother with Metadata? For organizations, metadata helps: • Protect investments • Counter personnel changes • Create institutional memory • Share data with other agencies Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---22 Why Bother with Metadata? For organizations, metadata helps: Reduce costs Limit potential liability Save time and money Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---23 What Does Good Metadata Look Like? Metadata at Its Best Topic 3 Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---24 What Does Metadata Look Like? Metadata in the FGDC format Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---25 What Does Metadata Look Like? Informative title Geographic location Sections of the standard Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---26 What Does Metadata Look Like? Identification Information Originator Abstract Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---27 Why Share Your Metadata? Why Clearinghouses Are Necessary to Our Work Topic 4 Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---28 Why Share Your Metadata? A metadata clearinghouse is a location — typically accessed through the Internet — to search for spatial data sets Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---29 Why Share Your Metadata? A clearinghouse is a decentralized system of Internet servers you can search Client Servers with metadata Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---30 Why Share Your Metadata? Clearinghouses make metadata records easy to find Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---31 Why Share Your Metadata? Why clearinghouses and not web indexes? • You can’t do a complete search • You can only do full-text searches • You can’t limit your search to particular fields Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---32 Why Share Your Metadata? • You can’t perform date, numeric, geographic searches • Place names don’t always relate to lat/lon • You don’t have search access to existing databases Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---33 Why Share Your Metadata? The National Geospatial Data Clearinghouse has more than 100 spatial data servers with digital geographic data Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---34 Why Share Your Metadata? Some specific examples: • FGDC Geospatial Data Clearinghouse • Montana State Library Natural Resource Information System GIS • Nebraska Geospatial Data Clearinghouse • Wisconsin Land Information Clearinghouse Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---35 Why Share Your Metadata? The NGDC has the people and infrastructure to help you find out who has what geographic information Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---36 Why Share Your Metadata? The NGDC is a set of information services that use hardware, software, and telecommunications networks to provide searchable access to information Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---37 Why Share Your Metadata? The clearinghouse includes federal, state, university, and vendor participants in the United States and abroad Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---38 Why Share Your Metadata? A digital geospatial data set is the target unit of description defined in the clearinghouse Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---39 Why Share Your Metadata? You can search all or part of the community in a single session x Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---40 Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---41 Why Share Your Metadata? You can define your criteria Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---42 Why Share Your Metadata? You can select a server Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---43 Why Share Your Metadata? Don’t You can view your search results Duck Metadata November 2000 Introducing Geospatial Metadata ---44 Why Share Your Metadata? Help us keep metadata accurate and up to date Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---45 Why Share Your Metadata? Make creating metadata part of the process of developing your data Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---46 Why Have a Standard? What the CSDGM Standard Is and How It Helps Us All Topic 5 Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---47 Why Have a Standard? The FGDC Metadata Workbook, Version 2.0 is free and can be downloaded www.fgdc.gov/metadata/csdgm/ Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---48 Why Have a Standard? The standard helps you determine: • if a set of geospatial data is available and fit for your use • how to access and transfer the set Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---49 Why Have a Standard? A standard helps to create: • Common terms • Common definitions Process step entity • Common language domain • Common structure Don’t lineage Duck Metadata November 2000 Introducing Geospatial Metadata ---50 Why Have a Standard? The standard… • establishes names of data elements and compound elements • defines information about values provided for data elements Don’t entity Duck Metadata November 2000 Introducing Geospatial Metadata ---51 Why Have a Standard? The standard serves as a uniform summary description of the data set Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---52 Why Have a Standard? If you’re creating metadata for the first time, the standard may seem complex Stick with it Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---53 Why Have a Standard? The key to using the standard… • Find the fields that are pertinent to your data and your organization • Build a template • Use the template Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---54 Why Have a Standard? Don’t create your own standard — you’ll only confuse people Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---55 Why Have a Standard? The standard will benefit all of us — for years to come Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---56 Why Have a Standard? Online systems rely on data being predictable in form and content Predictability is assured only when we conform to a standard Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---57 All About the Green Book Details About the Sections and Terms of the Standard Topic 6 Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---58 All About the Green Book FGDC’s Metadata Workbook: a.k.a. the Green Book 334 metadata elements and their production rules Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---59 All About the Green Book A view from the FGDC Metadata Workbook Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---60 All About the Green Book Metadata Main Sections Supporting Sections 1. Identification 8. Citation 2. Data Quality 9. Time Period 3. Spatial Data Org. 0. Metadata 4. Spatial Reference 10. Contact 5. Entity and Attrib. 6. Distribution Don’t Duck Metadata 7. Metadata Ref. November 2000 Introducing Geospatial Metadata ---61 All About the Green Book By Susan Stitt Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---62 All About the Green Book Compound Element Data Element Mandatory: must be provided Mandatory if applicable: must be provided if the data set exhibits the defined characteristic Optional: provided at the discretion of the data producer Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---63 All About the Green Book Hierarchical structure shows relationship Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---64 How Do You Write Good Metadata? Rules and Tips for Creating Quality Metadata Files Topic 7 Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---65 How Do You Write Good Metadata? Organize information Write your metadata file Review your file Have someone review Steps to Quality Metadata Revise it Don’t Publish Duck Metadata November 2000 Introducing Geospatial Metadata ---66 How Do You Write Good Metadata? Before creating your metadata… Get organized! Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---67 How Do You Write Good Metadata? Keep your readers in mind • Write simply but completely • Document for a general audience • Be consistent in style and terminology Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---68 How Do You Write Good Metadata? Think about the long-term effects • Don’t use jargon • Define technical terms and acronyms — CA, LA, GPS, GIS • Clearly state data limitations Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---69 How Do You Write Good Metadata? • Don’t use ALL CAPITAL LETTERS • Use subheadings • Use bulleted lists • Cite examples Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---70 How Do You Write Good Metadata? The title is critical in helping readers find your data Which is better? Greater Yellowstone Rivers from 1:126,700 Forest Visitor Maps (1961-1983) Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---71 How Do You Write Good Metadata? Useful titles Near Real Time Advanced Very High Resolution Radiometer (AVHRR) Data-Satellite Imagery from NOAA CSC Coastal Remote Sensing Division Ace Basin, South Carolina National Estuarine Research Reserve Digital Line Boundary Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---72 How Do You Write Good Metadata? Write a complete title, including: • What • Where • When • Scale • Who Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---73 How Do You Write Good Metadata? An informative title includes: • Topic • Timeliness of the data • Specific information about place and geography Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---74 How Do You Write Good Metadata? If the data are officially published, in the title include: • Series name • Issue number • Name of publisher • Location of publisher Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---75 How Do You Write Good Metadata? Be specific. Quantify when you can. Vague: We checked our work and it looks complete. Specific: We checked our work using 3 separate sets of check plots reviewed by 2 different people. We determined our work to be 95% complete based on these visual inspections. Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---76 How Do You Write Good Metadata? Select your key words wisely • Use unambiguous words • Use descriptive words • Fully qualify geographic locations Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---77 How Do You Write Good Metadata? Remember, a computer will read your metadata: • Don’t use symbols that might be misinterpreted !@#%{}|/\<>~ • Don’t use characters with dual interpretations Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---78 How Do You Write Good Metadata? • Don’t use tabs or indents • Be careful with carriage returns • Use “none” or “unknown” meaningfully Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---79 How Do You Write Good Metadata? Review your final product • Have someone else read it • If you’re the only reviewer, put it away and read it again later • Check for clarity and omissions Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---80 How Do You Write Good Metadata? When you review your work, ask: • Can a novice understand what you wrote? • Are your data properly documented for posterity? Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---81 How Do You Write Good Metadata? • Does the documentation present all the information needed to use or reuse the data? • Are any pieces missing? Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---82 How Do You Write Good Metadata? • Think like a novice • Put yourself in someone else’s shoes Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---83 Tools to Help You Tools That You Can Use to Create Quality Metadata Topic 8 Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---84 Tools to Help You No tool can check the accuracy of metadata Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---85 Tools to Help You You don’t need specific hardware or software to meet the requirements of the standard Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---86 Tools to Help You But different tools can help you create, validate, or distribute your metadata Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---87 Tools to Help You Tools can make your job easier: • Intelligent tools extract information from data sets • Form-based tools have a userinterface to guide you Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---88 Tools to Help You • ASCII and word-processor templates - Used to edit template documents • Utilities - Produce and process metadata in some form Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---89 Tools to Help You Look for… • Guidance without forced compliance • Stability • An OS/interface you know • No restriction on length of text field Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---90 Tools to Help You Consider… • Cost • Platform • Data-entry method • Level of compliance • User support Don’t • Capabilities Duck Metadata November 2000 Introducing Geospatial Metadata ---91 Tools to Help You Don’t www.fgdc.gov/metadata/toollist/metatool.html Duck Metadata November 2000 Introducing Geospatial Metadata ---92 Tools to Help You • USGS Metadata Editor • Corpsmet 95 metadata creation freeware • NBII Metamaker tool • NOAA CSC’s ArcView® Metadata Collector Extension Don’t November 2000 Duck Metadata November 2000 Introducing Geospatial Metadata ---93 Tools to Help You • USGS Metamaker • DataTracker • Spatial Metadata Manager Software Don’t November 2000 Duck Metadata November 2000 Introducing Geospatial Metadata ---94 Tools to Help You Reviews are available online… • Wisconsin NSDI Clearinghouse • Ohio State University, Ohio Geographically Referenced Information Program Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---95 Using the Clearinghouse Using the Clearinghouse to Distribute Your Data Topic 9 Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---96 Using the Clearinghouse The FGDC Clearinghouse selected the search and retrieve software ANSI Z39.50-1995 (ISO 10163-1995) Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---97 Using the Clearinghouse The clearinghouse has environmental and base cartographic information predominately Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---98 Using the Clearinghouse 127 nodes (servers) online Fed er al 28% Loc al 17% Don’t I n t e rn a t i o n a l 20% as of 7/99 Duck Metadata State 35% November 2000 Introducing Geospatial Metadata ---99 Using the Clearinghouse Federal participants • NOAA (10) • DOT NTAD • USGS (6) • National Park Service • FEMA (sampler) • Army Corps of Engineers • NRCS (sampler) • Tri-Services Center • CIESIN/EPA • National Wetlands Inventory • CIESIN/NASA • Census (sampler) Don’t as of 7/99 Duck Metadata November 2000 Introducing Geospatial Metadata ---100 Using the Clearinghouse How can you participate? • Set up your own node • Send metadata to an existing node • Have another organization host your node • Put metadata on a web page for future “harvesting” to a node Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---101 Using the Clearinghouse Why set up a node? • You control your metadata • It’s easy to set up • You choose the software • The software operates on typical web server platforms Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---102 Using the Clearinghouse • In setting up a node, most work is in preparing metadata for available data sets • Store these data in a structured form, according to the standard Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---103 Using the Clearinghouse Check your metadata for completeness, accuracy, and quality Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---104 Using the Clearinghouse Meet clearinghouse requirements: •Proper format •Proper field names and values •Appropriate formats: text, HTML, and SGML Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---105 Using the Clearinghouse Parsing and validation tools check for structural or content errors mp metadata documents metadata parser software error reports Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---106 Using the Clearinghouse Validate your metadata by using cns and mp software Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---107 Using the Clearinghouse Run original file through cns Create Metadata Correct original file YES Errors in Errorserror.txt? in error.txt? NO Run cns output through mp Correct original file YES Errors in error.txt? NO Run cns output through mp to make text, SGML, HTML files Don’t Place 3 files on searchable system Duck Metadata November 2000 Introducing Geospatial Metadata ---108 Using the Clearinghouse The tool produces metadata in HTML, SGML, and TEXT mp HTML TEXT SGML Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---109 Using the Clearinghouse Then indexes files by searchable fields into document database HTML TEXT index searchable document database SGML Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---110 Using the Clearinghouse <html> <head> <title>1989 Land Cover/Land Use on the Upper Mississippi River System</title> </head> <body> <h1>1989 Land Cover/Land Use on the Upper Mississippi River System</h1> <h2>Metadata:</h2> <ul> <li><A HREF="#Identification_Information">Identification_Information</A> <li><A HREF="#Data_Quality_Information">Data_Quality_Information</A> <li><A HREF="#Spatial_Data_Organization_Information">Spatial_Data_Organization_Informati on</A> mp HTML file Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---111 Using the Clearinghouse mp HTML file Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---112 Using the Clearinghouse <!DOCTYPE METADATA PUBLIC "-//FGDC//DTD METADATA 1.0//EN"> <metadata> <idinfo> <citation> mp SGML file <citeinfo> <origin>Kurt P. Kowalski and Douglas A. Wilcox</origin> <origin>Great Lakes Science Center</origin> <origin>Biological Resources Division</origin> <pubdate>1997</pubdate> <title>Coastal Wetland Vegetation Analysis (Metzger Marsh) </title> <pubinfo> Don’t <pubplace>Ann Arbor, MI</pubplace> Duck Metadata November 2000 Introducing Geospatial Metadata ---113 Using the Clearinghouse Identification_Information: Citation: mp TEXT file Citation_Information: Originator: Environmental Management Technical Center Publication_Date: 19950829 Title: 1989 Land Cover/Land Use on the Upper Mississippi River System Geospatial_Data_Presentation_Form: Map Publication_Information: Publication_Place: Environmental Management Technical Center Publisher: Environmental Management Technical Center (EMTC) Don’t Other_Citation_Details: White, B. M. and T. W. Owens. 1991. Geographic Information System Pilot Project For The Upper Mississippi River System. U.S. Fish and Wildlife Service, National Ecology Research Center, Fort Collins, Colorado, June 1991. LTRMP 91-05. 48 pp. + Appendix. Duck Metadata November 2000 Introducing Geospatial Metadata ---114 Using the Clearinghouse cns and mp command line validation Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---115 Using the Clearinghouse To incorporate your metadata into a clearinghouse… • Spell out the element names exactly • Keep the parent/child relationships • Don’t omit intermediate compound element headings Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---116 Where Do You Go from Here? Where to Find Help While You’re Back at Work Topic 10 Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---117 Other Resources • FGDC metadata website • CSDGM Workbook (The Green Book), Version 2 • Image map of the CSDGM Version 2 by Susan Stitt, USGS-NBS Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---118 Other Resources • Mentor Software Freeware converter • Chuck Taylor Freeware Toolbox • FGDC tutorial on setting up a clearinghouse node • Search for Geographic Data, FGDC, National Geospatial Data Clearinghouse Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---119 What Now? • Don’t invent your own standard • Consider data granularity • Prioritize your data • Document to preserve value within your organization Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---120 What Now? • Use the standard • Spread the word Don’t Duck Metadata November 2000 Introducing Geospatial Metadata ---121