Sharing Your Data – An Update on AIRS XML and Helpful Tools Marianne Galleon – 211 LA County (626) 350-1841 x2122, marianne@211LA.ORG Teri Perkins – University of Nebraska Public Policy Center 402-472-5620, tperkins@nebraska.edu Today’s workshop Data sharing between I&R databases – – – – – – – – 2 The big picture AIRS XML Schema Vendor role Tool for automated processing of partner data Things to think about when sharing data Examples of on-line databases Resources Contact information Goal and Objective Comprehensive databases that take advantage of the expertise of multiple database “owners” to more efficiently deliver referrals to those who need them Enable routine sharing of information between I&Rs regardless of the software products they use 3 Current Practice In our communities and across our regions and states, there are many organizations keeping information about resources. Organizations maintain information to serve their clients: – – – 4 Geographic Age group Type of service The Conundrum 5 I&Rs have specific expertise But, clients may have needs beyond precise information that any one I&R can reasonably maintain… Duplication of effort? 6 Many I&Rs collecting information from many agencies Many agencies providing information to many I&Rs Groups of I&Rs working together… 7 Recognizes expertise Reduces duplication Joint promotion More responsive referrals Alliance of Information and Referral Systems Accreditation… I&R services within the system shall endeavor to participate in local database collaboratives as a means of avoiding duplication of database maintenance activities and achieving broader coverage of different types of community resources. The I&R services within the system shall: Appropriately divide information gathering tasks and oversee the regular and systematic exchange of resource database information; and Maintain comprehensive, accurate, and up-to-date information on the community resources for which they have maintenance responsibility. (p. 18) 8 Exchanging Information Between Databases That Use Different Software Need a standard way to express data – – XML allows users to create their own tagging names and definitions (eXtensible Markup Language) AIRS volunteers developed and refined the AIRS XSD (XML Schema Definition) over the past three years – 9 The AIRS XSD describes the structure and content of the data in an XML document AIRS Board approved the AIRS XML Schema AIRS XML Schema Three levels of data Agency Site Services 10 Services Site Services Services Site Services Services Services Services Services Services AIRS Commitment Standing Subcommittee of AIRS Technology Committee – – 11 Annual updates to the AIRS XSD (XML Schema) Working on a “Style Guide” Continuing to look at additional ways to promote interoperability through certifications and standards Data Sharing Database – I&R A I&R A’s Vendor utility program Compliant XML export of I&R A’s data AIRS XSD Database – I&R B I&R B’s Vendor utility program Comprehensive Database Compliant XML export of I&R B’s data 12 Vendor Role in Data Sharing Vendors develop utilities for their software that allow mapping of customer data to an XML file that is compliant with the AIRS XSD – 13 Eliminates manual conversion of the data into an XML file by comprehensive database administrator Valid XML File – What Next? Compliant XML files from databases using different software may be combined into a common database by either: – – Manually processing files Using an automated processing system One such tool for automated processing is in development – U.S. Department of Commerce Technology Opportunities Program grant (#31-60-I03013) to develop a technological tool that will prepare data and create a new consolidated data set 14 University of Nebraska Public Policy Center / University of Nebraska Department of Computer Science & Engineering Other tools may use different approaches to process XML files Database Transformation – Manual Process Database from DB owner Database tables Verify if the XML file is compliant with XSD XML file Transform to XML file Reformat to ACCESS DB Feedback to DB owner for corrections Vendor implements these steps with utility • No mailing address or incomplete address (must have a zip code) • Wrong format of email, web-link, phone, etc. Typical errors Compare taxonomy tables (1) Check user login (2) Upload XML file (3) Validate XML file (4) Check addresses for validity Communicate with DB owners regarding duplicated records Create a file Read the file and insert records into the database on laptop; update tables Check for record duplication and send feedback to DB owner; update invalid (i.e., missing information) records in comprehensive database Test DB on laptop Upload approved DB to server Updated DB on the server Tool will implement these steps Processing System Tool - Features Database Partners: – – Comprehensive Database Administrator: – – – 16 Login and view previously uploaded data Login and upload an AIRS XSD compliant XML data export Run specialized data processing routines Browse partner uploaded data Promote data to live web applications Automated Tool - Partner Advantages Database Partners: – – – 17 Receive immediate feedback regarding compliance of the structure of the data export with the AIRS XSD Manage their own content delivery schedules Access audit trails of previous content deliveries Automated Tool – Administrator Advantages Comprehensive Database Administrator: – – – 18 Notified of content delivery only after successful validation of content Single point of access to work with data sets for testing and management Accurate data when content deliverables are promoted to live websites Custom Processing In addition to basic validation against a schema, the system allows extended processing to be plugged in, such as: – – 19 De-duplication of records from multiple providers Reclassification of taxonomy codes based on partner recognized (customized) taxonomies All processing is automated Flexible – allows manual adjustment of data before promotion to the website Data Owner Work Flow • Automated Processing 20 • • After validation, all processing is done automatically - no user input is required Audit trails are generated Any automated processing can be manually adjusted after the database is updated What will the web interface look like? 21 Login screen Welcome screen Upload file screen Browse files Successfully uploaded the file Successfully parsed the file 22 23 24 25 26 27 Benefits of Processing System 28 Decrease the amount of interaction between comprehensive database administrator and database partners currently required to update website data sets Provide greater control over partner database and comprehensive database content management processes Web infrastructure facilitates management and sharing of I&R data Some general things to think about when sharing data… Partners – – – AIRS Taxonomy – – – 29 Confidence in partner’s database Skill in coding with the AIRS Taxonomy Frequency with which partner data is checked and updated Levels to observe (customization) Updating customized taxonomy All partners must use the same version of the Taxonomy Things to think about (continued)… Geographical Area Served – – Search on this? Required by XSD, but not defined Differences in terminology used by partners – e.g., Program = Site = Provider Urban / rural partners may have different interests – – 30 Define as County, City, Zip Code, other? Level of indexing Distance to travel for services Things to think about (continued)… Resources needed to maintain a comprehensive database – – – AIRS XSD version Compliant XML file doesn’t guarantee you will have the data you want for your database – – 31 Time Money Technical expertise Required / optional fields in the AIRS XSD Work with software vendor regarding data to be included in the XML file Things to think about when sharing data with partners using different software… – Different vendors may have different XML coding styles – – Terms added to the XML by vendor may be coded in proprietary format Unique / non-unique keys – More than one way to use XML to code information All partners may not send information with same detail 32 Unique key requirement not necessarily consistent among vendors Information may be coded at different locations within a field – Number of levels of agencies Use of virtual agencies e.g., coding of target groups/modalities e.g., coding all Age, Gender, and Family exclusions vs. coding them as Eligibility under Service Things to think about even if sharing data with partners using the same software… – – – Different taxonomy versions Different XSD versions Vendor may allow custom fields to be created by software user – Differences in data standards – e.g., Hours information may be stored at Site or Service level What needs to be identical to allow data sharing? 33 e.g., names in upper case vs. mixed case Data stored in different locations – Information in new fields not transferred to another user e.g., county codes may not be identical between users of same software Comprehensive Databases Examples of 2-1-1 on-line databases – Los Angeles County – Nebraska – www.ne211.org Texas 34 http://www.healthycity.org/index.php?c=fs www.211texas.org/211/index.jsp Resources AIRS XML Schema Definition (XSD) – – Interoperability and Information and Referral – – – – On-line version of taxonomy at http://www.211taxonomy.org/ Bruni, M.G. (2000). Indexing with the AIRS/INFO LINE Taxonomy of Human Services. Information and Referral, 22, 83-109. Sales, G. (2003). An orientation to the structure and contents of the AIRS/INFO LINE Taxonomy. Information and Referral, 26, 1-26. XML – 35 Galleon, Marianne. (2006, January). AIRS working on way for I&Rs to share information. Alliance of Information and Referral Systems News, Volume XXVI No. 1, 1. Shank, N. (2004). Database interoperability: Technology and process for sharing resource information. Information & Referral, 26, 45-54. AIRS/INFO LINE Taxonomy of Human Services – http://www.airs.org/news/news_news.asp Current XML Schema is at http://www.ne211.nebraska.edu/vendorinformation.htm, take the “Current XSD” link XML - A good resource about XML is the XML Schema Primer at www.w3c.org/TR/xmlschema-0/ Contact Information Marianne Galleon – 211 LA County – Teri Perkins – University of Nebraska Public Policy Center – 36 (626) 350-1841 x2122, marianne@211LA.ORG 402-472-5620, tperkins@nebraska.edu