Modifying the HL7 Continuity of Care Document (CCD) to Improve Reporting of Results from Functional Assessment Instruments ***DRAFT*** Tom White1, Jennie Harvell2, Mark Tuttle3, John Carter3 1 New York State Office of Mental Health and Columbia University Assistant to the Secretary of Planning and Evaluation, Department of Health and Human Services 3 Apelon, Inc. 2 The opinions and views expressed in this document are those of the authors. They do not necessarily reflect the views of the Department of Health and Human Services or New York State. Abstract A standardization process recently endorsed by NCVHS has bearing on the Functional Status section of the current CCD Ballot. On 11/06 NCVHS endorsed a “LOINC-ification” process that supports the standard coding and messaging of patient assessment instruments that includes disability assessment and functional status content. The LOINC-ification process represents the content of the patient assessment instruments in LOINC and links LOINC-coded assessment questions and answers with usefully related semantic matches. That is, it represents both the instrument itself – the questions and possible answers, and the result of applying the instrument to a given patient. The process makes use of the UMLS (Unified Medical Language System) to represent mappings – links - between particular questions and answers, and codes from terminologies such as SNOMED, and it makes use of HL7v2 to transmit instrument results. The NCVHS endorsed the use of HL7 v. 2.4 (and higher) and CDA (Clinical Document Architecture) for the exchange of patient assessments and other standardized functioning and disability content. This work was advanced through and endorsed by the Consolidated Health Informatics (CHI) Initiative. The current CCD ballot proposal for Functional Status appears to be in conflict with existing psychometric theory, and it does not take advantage of the recent NCVHS recommendations. This paper will illustrate these conflicts, describe the relevance of the NCVHS endorsed LOINC-ification process, propose how the NCVHS proposal can be utilized within current CCD modeling, and propose a small modification to the CCD ballot draft to support use of the NCVHS recommendation. At the end of this document, we summarize the favorable results of presenting this proposal to the HL7 CCD working group. The Challenge of Creating a Canonical Value Set for all Possible Functional Status Problems An implied goal of the Functional Status section of the CCD is to enumerate all possible functional status deficits in a Value Set. There are many standard psychometric assessments for each of the dozens of functional status domains. There are also many federally mandated forms which use measurements of functional status for payment and quality assurance purposes. Although it might be tempting to create a unique list of functional status values within each of these domains, psychometric and survey theory caution against such an effort. Both theories dictate that even minor changes to the wording of questions or the allowable response options can significantly change the meaning of the deficits being Modifying CCD Representation of Assessment Instrument Results Page 1 of 12 measured. Moreover, federally funded efforts to find unique codes from existing standard terminologies which adequately capture the intent of questions on existing federally mandated reporting forms, like the Centers for Medicaid and Medicare Services (CMS) mandated Nursing Home Minimum Data Set (MDS) Version 2 form, found less than 50% “exact” matches1. However, this same effort found over 90% “usefully related” matches (such as broader or related matches); in this context an “exact” match can be substituted for the original matching element, and “usefully related” matches are exactly that – links between instrument elements and codes that a human or computer should find useful.* Further, consolidating across different instruments the different answer sets for measurements of related functional deficits often requires expensive research to properly place the answer choices along a continuum – a continuum being one way to represent diverse assessment values. For example, NIH has spent millions of dollars and several years on the PROMIS network2, as one of the NIH Roadmap initiatives. PROMIS, the Patient Reported Outcomes Measurement Information System, is meant to create single scales for the assessment and representation of “pain”; the goals of scale are to span from 0 to “maximal pain”, apply to all age groups, and be easy to assess (using computerized adaptive testing techniques). This has required significant statistical effort using Item Response Theory to both place the various different pain scale measurements along a single continuum, and also create computerized systems for accurate patient self-report of pain along that continuum. Were HL7 to try to create canonical lists of functional status types and values, a task considerably broader in scope than the PROMIS initiative, HL7 should expect to incur similar expenses, time delays and risk. Fortunately, recent modeling – formalization† - efforts for assessment instruments have avoided this problem by supporting the creation of unique codes for all elements of each standard assessment instrument, while at the same time facilitating the creation and use of semantic links – mappings between instrument concepts and external terminology standards. This obviates the need for creating single master lists of functional status descriptors, and instead empowers users to report exactly the results they found using their preferred functional status assessment instruments in their natural form. This also empowers government and researchers to study the reliability and validity of those measurements over time, using a common framework. This supports efforts to create new, improved instruments building upon the best existing content. It also supports enhanced policy decisions (such as payment and quality policies) through the use of the best subset of available instruments and/or improved instruments. Again, both original instrument uses may be preserved, and new uses, including more productive completion of old uses, are supported by the LOINC codes, semantic matching, and standard messaging. Modeling Content within Assessment Instruments: “Competency” as an Example LOINC personnel have invested significant resources over the past several years to represent survey instruments3,4,5 in LOINC. Since Psychometric6 and Survey Theory7 show that even minor changes to the wording of questions and their response sets (value lists) can change the perceived meaning of the * Preliminary work on early portions of MDSv3 suggests that similar results may be obtained with the next version of MDS. In this context, “modeling” and “formalization” refers to a process by which instruments are made “explicit” – especially explicit in a way that supports processing, comparability and re-use by computers. For example, a first – formalization - step is to reproducibly parse the text of the instrument into “tokens,” e.g., ‘questions,” and “answers.” A second step, is to identify (and make explicit) any redundancy – a process of defining whether pairs of tokens are the same, or equivalent, or related. † Modifying CCD Representation of Assessment Instrument Results Page 2 of 12 deficit being measured, LOINC stores the full text of survey questions and all allowable answers, when available. Moreover, LOINC has modeled both entire instruments, and the sections within them, so as to support re-use of the individual questions and answer sets which may be shared across instruments. A discussion of instrument-level modeling is beyond the scope of this current discussion, but may become relevant to future versions of the CCD. This process of LOINC-ifying assessment instruments and linking them to usefully related terms has been endorsed by NCVHS8. Starting in 2005, the Department of Health and Human Services funded, under the leadership of the Office of the Assistant Secretary for Planning and Evaluation (ASPE), an effort to standardize CMS’s mandated assessment instruments for long term care, including the Nursing Home Minimum Data Set (MDS). This work resulted in a white paper recommending the LOINCification strategy for encoding the content of assessment instruments, ensuring that content can be messaged via HL7, and facilitating efforts to identify, store, and use semantic mappings to the concepts measured by the MDS in standard terminologies.9 The white paper also noted that, in principle, this LOINC-ification/semantic matching process is applicable to any survey content, regardless of content domain. Thus the process is one strategy for overcoming the many gaps in data standards. One gap is the normalization shortfall – unidentified redundancy across instruments; another gap is the presence of instrument elements that lack a representation in standard terminologies. After meetings with relevant federal agencies, HL7 workgroups, standards development organizations, and representatives of SNOMED, the proposal was unanimously endorsed through the Consolidated Health Informatics (CHI) Initiative10 and presented to NCVHS in October, 2006. NCVHS endorsed this proposal, conditioned upon the resolution of a few issues. The conditions applicable to the exchange of patient assessment content have been largely resolved. This LOINC-ification/semantic matching process has been approved by CHI and NCVHS, applied to several large instruments used to collect functional status information, and it has been shown to be compatible with HL7 RIM11 and CCD modeling. Therefore, since it can apply to assessment instruments in general, we recommend that HL7 adopt this approach to representing and messaging functional status content instead of trying to create canonical value lists. The following example shows how the LOINC-ification process models, encodes, messages, and supports semantic interoperability for one of the many measurements of functional status within the MDSv2. Figure 1 is a screenshot of question B4 from the MDS, which assesses “competency.” B4 Figure 1. Question B4 from the paper MDSv2 form. Modifying CCD Representation of Assessment Instrument Results Page 3 of 12 LOINC/RELMA Coding of B4 12 12 Figure 2. LOINC 2.17 (RELMA 3.17) representation of MDSv2 question B4. As shown13 in Figure 2, LOINC encodes all of the content and contextual information from question B4. LOINC creates a code for the entire instrument (45981-8), and all relevant sections within the instrument (e.g. 45987-5 – Cognitive Patterns Section). It also creates unique codes for each unique question within the instrument (e.g. 45490-0 refers to the MDS Question B4). Moreover, as seen in the bottom of the Figure, LOINC stores information about all allowable answers to question B4, the order (seq#) in which they appear in the answer list, the value which the source system associates with each answer (Code), and optional spaces for Global IDs (e.g. concept codes for those answers, if relevant). In addition, LOINC stores contextual information determined to be relevant to the meaning of the items, such as consistency checks, relevance equations, and other “help” content. The ASPE-led MDS work found that there can be zero to many semantic matches between instrument “answers” and elements of existing terminologies. For example, Figure 3 shows that there are three SNOMED codes which are usefully related to the answer 2 (MODERATELY IMPAIRED) for question B4. Modifying CCD Representation of Assessment Instrument Results Page 4 of 12 Figure 3. There are three SNOMED codes usefully related to answer 2 (Moderately Impaired) for Question B4.14 1. MSH|^~\&| * 2. PID| * 3. OBR||||45981-8^MDS FULL ASSESSMENT FORM^LN| * 4. OBX||CE|45490-0^MAKES DECISIONS REGARDING TASKS OF DAILY LIFE^LN^B4^Ability to make decisions regarding daily life^MDS||2^MODERATELY IMPAIRED-decisions poor, cues/supervision required^MDS^F-90157^Difficulty using decision-making strategies (finding)^SNM| * 5. [additional OBX segments – one per question / measurement] Figure 4. Sample HL7 message to transmit answer 2 to MDSv2 question B4. Further work with LOINC and HL7 developers concluded that this content could be messaged within HL7 version 2.5 within the OBX-3 and OBX-5 message segments, as shown in the Figure 4. In Figure 4, Line 3 is the optional OBR message segment which names the instrument that is the source of these questions. Line 4 shows the OBX segment which indicates that answer code 2, Moderately Impaired, was selected for MDS question B4. The blue section is OBX-3, which uniquely names the question as 45490-0, and also indicates that the associated name within MDS is B4. The purple section is OBX-5, which transmits the value for this observation, indicating that the value is 2, per MDS’s coding system, and that the best usefully related semantic match for that value is SNOMED code F-90157. Additional name/value pairs for questions would be transmitted in subsequent OBX segments. The ASPE project also found a way to use the UMLS to overcome limitations in HL7’s ability to transmit a repertoire of semantic relationships. For example, one limitation of the HL7 2.5 messaging is Modifying CCD Representation of Assessment Instrument Results Page 5 of 12 that only a single associated code can be transmitted in the OBX-3 and OBX-5 segments. For OBX-3, that associated code must be the variable name of the source system (e.g. B4) so that when HL7 messages are received, the source system knows how and where to store the values sent in OBX-5. OBX-5, on the other hand, transmits the value expected by the source system, plus one other usefully related code. As Figure 3 illustrated, there can be many usefully related codes. The UMLS Metathesaurus15 table MRSMAP16 supports the representation of many-to-many relationships among concepts and coding systems. Figure 5 shows how the UMLS MRSMAP table formally represents the assertion that SNOMED codes F-90120, F-90156, and F-90157 (see Figure 3 for their Text values) are similar to answer 2 to MDS question B4. One barrier, which will be eliminated by the end of 2006, is the lack of a coding system to uniquely name instrument “answers.” LOINC will be enhanced to provide LOINC_ANSWER_PART codes, called LA codes, to uniquely represent caseinsensitive answer strings. These LA codes will not represent concepts. Rather, the combination of an LA code for an answer and the LOINC code for the question represent mappable concepts. Thus, the concept implied by answer 2 to MDS question B4, which is roughly equivalent to “moderately impaired ability to make decisions about daily life”, is similar to SNOMED concepts like “difficulty using decision making strategies (finding)”. The FROMEXPR uses a Boolean expression to indicate that the “from” concept is the conjunction of the LOINC code for question B4, and the LOINC_ANSWER_PART (LA) code for the string “MODERATELY IMPAIRED-decisions poor; cues/supervision required”. The TOEXPR is the “usefully related” SNOMED code. Figure 5. Fragment of planned UMLS MRSMAP submission to codify mapping between particular answers to MDS questions and usefully related SNOMED codes. More complex mappings are expected, and can be supported by the UMLS’ MRSMAP schema. For example, if one looks at Figure 3 more closely, one will see that SNOMED codes F-90120 and F-90157 are also usefully related to answers 1 (Modified Independence) and 3 (Severely Impaired), respectively. None of the single SNOMED codes listed captures the full intent of the question and the concepts an answer of 2 (Moderately Impaired) represent. Rather, the most exact mappings are likely to be to a Boolean combination of post-coordinated SNOMED terms. Work on identifying such best matches is underway. In addition, broader and partial matches have already been identified, and the CHI Disability Workgroup expressed interest in having those partial matches represented within UMLS so that all findings related to particular domains, like mental status and capacity, could be easily extracted and publicly accessible. Storing such semantic relationships within UMLS enhances the potential for reuse of survey and administrative data – both the instruments themselves and the results of applying those instruments to Modifying CCD Representation of Assessment Instrument Results Page 6 of 12 patients. Nursing homes are required to complete MDS forms for all patients at least quarterly, and those data are used by CMS for payment decisions and quality reporting. There is considerable clinical, safety, and functional status content within the MDS, that could be usefully included in a Continuity of Care Document. By LOINC-ifying the MDS and representing semantic mappings of MDS answers to SNOMED, MDS results could be re-used for multiple purposes including exchanging standardized functional status summary information as patients transition across health care settings and providers, as well as clinical, quality, safety, and other management purposes. Use Cases for Using CCD to Support Reuse of MDS Content Completed MDS forms are an excellent source of data for CCD documents, and for the functional status section in particular. Several federal agencies including CMS, SSA (Social Security Administration), and VA (Veterans Affairs) mandate the collection of administrative data which include considerable amounts of functional status and other clinical data. CMS, in particular, requires three different assessment forms depending upon the type of setting in which the patient is receiving services (i.e. different assessment forms for nursing homes, home health agencies, and in-patient rehabilitation facilities). Currently, systems that support the use of these instruments do not interoperate. The results of the HHS project are expected to enable the beginnings of such interoperability via the identified semantic mappings found among similar content across those different instruments, plus enable reuse of standardized assessment questions and answers as future forms are created. Further, if the CCD can represent the coded functional assessment content and/or complete assessment forms, then a patient’s historical MDS data could be transmitted and used to auto-populate his or her Personal Health Record.17 Currently, most nursing homes do not have a robust EHR. However, almost all have HIT applications that support the electronic production of the MDS for administrative purposes including claims submission and compliance. Some NHs use CMS’s free “MDS” software18 for this task. Now that the MDS has been modeled in LOINC, any MDS data can be represented within CCD as HL7 Clinical Statements (Observations). Under this scenario, future systems could generate and send a CCD that includes (i) functional status results for a particular patient based in that patient’s MDS assessment and (ii) the usefully related SNOMED codes that have been linked with the assessment questions and answers and describe assessment results for the patient. HIT vendors for the sending system would have obtained (either directly or from a third party terminology services provider) from the UMLS and integrate into their HIT system the terms that have been linked with the assessment content. The CCD exported from the sending system could include all usefully related SNOMED matches or only those usefully related semantic matches that the sending system has linked to that patient’s assessment results. The CCD would transmit both coded and human-readable assessment content. The scenario described above offers a near and longer term solutions for the exchange of much needed functional status information as patients transition from one provider or care setting to another. The mapping of the nursing home MDS to LOINC and usefully related vocabulary terms permits nursing home HIT vendors to create more HIT standards compliant products. The more general policy push for the acquisition of standards compliant EHRs (e.g., the CCHIT efforts to certify physician and hospital EHRs and recently announced expansion of the CCHIT certification efforts to include additional health care settings) is expected to increase the acquisition and use of standardized EHR products in physician, hospital and other health care settings. If standardized EHRs/PHRs receive a CCD that includes standardized functional status assessment content, the receiving systems will, upon receipt of the Modifying CCD Representation of Assessment Instrument Results Page 7 of 12 standardized assessment content, be able to reuse the assessment results in a variety of applications (e.g., applications for patient management, trend analyses, resource management, etc.). Modeling Survey Instrument Content Using HL7 Version 3.0 Clinical Statements The most practical way to transmit content, i.e., “results,” from survey instruments is within Observations portion of HL7 Clinical Statements. As Figure 6 illustrates, the included Observation.code names the type of content being transmitted within the observation, and the Observation.value encodes the results of the observation. Within the HL7 2.5 messaging (Figure 4), OBX-3 named the observation (question) using a LOINC code, and the alternate coding system within OBX-3 communicated the source vocabulary’s name for the variable. Figure 6. RIM Model for Observations, a type of Act within Clinical Statements. The Observation.code syntax uses the CD data type, which supports “Translations to allow the representation of the concept in the coding scheme in which it originated as well as translated into other preferred terminologies.” Thus, the Observation.code could codify both the LOINC code for the survey question and also (via the translation syntax), the name of the variable as expected by the source system. Similarly, since the Observation.value is of data type ANY, the CD data type could be used to transmit the LA code uniquely identifying the answer string, a translation to encode the value expected by the source system, and additional translations to represent usefully related SNOMED codes. The translation data type may be sufficiently similar to the notion of “usefully related matches” to support use of the translation syntax to transmit SNOMED codes which are near or related matches to MDS or other survey content. Translations are a set of other CD data types which are “quasi-synonyms” of the top-level concept. However, it may not be possible for the translation syntax to represent the type of semantic match, such as a broader vs. related match. Translation CDs can be qualified with Qualifier codes to increase their own specificity (e.g. to create code phrases and post-coordination), but it appears that those qualified translations would still be expected to be “quasi-synonyms” for the original concept. Nevertheless, these translations may be adequate for current Use Cases, and the availability of richer semantic modeling from the UMLS can be retrieved and utilized using standard tools. Modifying CCD Representation of Assessment Instrument Results Page 8 of 12 Figure 7. Potential syntax for messaging MDS B4 example within CCD Functional Status section. The syntax for <translation> may not be quite correct, since it is a type of <set>. Figure 7 shows how the MDS B4 example might be “messaged” within a CCD document. The Functional Status portion of the Clinical Statement (e.g. whether the problem is Active) was deleted for Modifying CCD Representation of Assessment Instrument Results Page 9 of 12 the sake of space, but should remain in any final example. There are three differences from the November 26 SampleCCDDocument.xml. (1) Functional Condition was divided into Observation Type and Observation Value. (2) Observation Type is represented within the observation.code (LOINC code 45490-0), instead of “Assertion”. (3) The Observation Value uses the LA code as the primary code for the result, plus several translations including the coded value expected by MDSv2, and the usefully related SNOMED codes. Note, the translation data type is a set, so multiple translations (and even nested ones) can be encoded, but the syntax in Figure 7 may not be incomplete. Proposed Modification to HL7 CCD Ballot In section 3.5.2, CONF-135 states: CONF-135: The value for “Observation / code” in a problem observation or result observation in the functional status section MAY be selected from ValueSet ??? FunctionalStatusTypeCode STATIC 20061017. This presumes the ability to create canonical lists (Value Sets) of all Functional Status Types, which as described above, may be at best unproductive and at worst unrealistic. Instead, we suggested that following changes (with word-smithing as appropriate)be included to the HL7/ASTM CCD Implementation Guide that will be balloted beginning December 6, 2006: CONF-135: If the functional status was collected using a standardized assessment instrument, then the value for “Observation / code” in a problem observation or result observation in the functional status section SHOULD be the LOINC code from 2.16.840.1.113883.6.1 LOINC DYNAMIC which uniquely identifies the question within that instrument used to collect the problem observation or result observation. CONF-136: If the functional status was collected using a standardized assessment instrument, and the answer is part of an enumerated value list, then the value for “Observation / value” in a problem observation or result observation in the functional status section SHOULD be the LOINC_ANSWER_PART code from 2.16.840.1.113883.6.1 LOINC DYNAMIC which uniquely identifies the answer string selected for that result. CONF-137: If the functional status was collected using a standardized assessment instrument, and the answer is part of an enumerated value list, and there are available associated values for that answer (e.g. SNOMED codes or codes expected by the sources system), then there MAY be a “Observation / value / translation” set containing the pertinent code translations, and conforming to CD datatype syntax. Conclusions A generic process for encoding, transmitting, and representing semantic relationships for assessment instruments has been endorsed by NCVHS. This process is sufficiently complete and compatible with HL7 RIM and the CCD that it could be proposed as SHOULD-level Clinical Statement Conformance criteria within the Functional Status section. Adoption of such criteria may facilitate efforts to re-use administrative instruments and data on functional status for clinical, quality, and information exchange improvement purposes. Results of Presenting This Proposal to HL7 CCD Working Group The above proposal was presented to the HL7 Structured Document Technical Committee (SDTC) on 11/30/2006. Members of the SDTC approved the incorporation of these recommendations into the 12/6/06 CCD Ballot, with some modifications. These modifications support the goals of this proposal, including supporting the use of SNOMED to name and encode results from assessment instruments. For Modifying CCD Representation of Assessment Instrument Results Page 10 of 12 purposes of internal consistency, the conformance criteria indicate that the codes and values for instruments should both be derived from the same standard, but also supports messaging of translation of these concepts from other terminologies. The ballot is available on the HL7 ballot website19. The authors of this document were made aware of other work on assessment instruments being done within HL7 and SNOMED and are beginning to compare and contrast these efforts. SNOMED’s Nursing Working Group has done considerable work on modeling results from nursing assessment instruments. The HL7 TermInfo Workgroup is providing guidelines on use of SNOMED within the CDA to message results from assessment instruments. The HL7 Patient Care Workgroup is the primary committee responsible for refining the Structure Document domain model to support the needs of assessment scales and instruments. Our current understanding is the SNOMED and LOINC modeling efforts are synergistic and complimentary, with few overlaps or conflicts. LOINC models the contents of instruments, including the full text of the questions and their allowable answers, their surrounding context, plus any numerical values and variable names used by source systems for those questions and answers. LOINC also models whole instruments, complete with branching logic, and efficient re-use of the questions and answers across instruments. However, LOINC does not code the granular concepts embedded in many assessment questions and answers nor does it currently address the semantic relationships among instruments or their contents. Rather, a parallel effort with NLM is creating an efficient process to map LOINC-modeled concepts within assessment instruments to terminologies like SNOMED which have many (although sometimes incomplete) granular terms and robust hierarchy and semantic structures. Synergistically, SNOMED has provided names of over 700 assessment scales, and is starting to classify questions and answers within the Observable Entities and Findings hierarchies, respectively. Thus, SNOMED is using its hierarchy structure to represent the apparent intent of questions and answers (items) to as deep a granularity as possible; and SNOMED is also creating a hierarchy of broader matches to classify the constructs being measured by those items. However, SNOMED does not appear to intend to store or create codes for the exact questions or answers used by the source systems, nor model instrument contents to the level being done by LOINC. Thus, it appears that SNOMED and LOINC are focusing on separable, but complementary aspects of modeling assessment instruments; and that both approaches will be needed to fully model and maximally re-use assessment instruments and their contents. As next steps, our group is reaching out to the above-mentioned SNOMED and HL7 committees and workgroups to raise awareness of our related efforts, and to discuss ways to harmonize or coordinate efforts. It also appears that sufficient modeling work and content creation work has been done to support vendor and academic efforts to create working prototypes. 1 Making the "Minimum Data Set" Compliant with Health Information Technology Standards, http://aspe.hhs.gov/daltcp/reports/2006/MDS-HIT.htm. 2 http://www.nihpromis.org/ 3 Bakken, S, Cimino, JJ, Haskell, R, et al. Evaluation of the clinical LOINC (Logical Observation Identifiers, Names, and Codes) semantic structure as a terminology model for standardized assessment measures. J Am Med Inform Assoc. 2000; 7(6):529-538 4 White, TM. Extending the LOINC Conceptual Schema to Support Standardized Assessment Instruments, J Am Med Inform Assoc. 2002; 9(6):586-599. Modifying CCD Representation of Assessment Instrument Results Page 11 of 12 5 Choi, J, Jenkins, ML, White, TM, Cimino, JJ, Bakken, S. Toward Semantic Interoperability in Home Health Care: Formally Representing OASIS Items for Integration into a Concept-Oriented Terminology. Journal of the Medical Informatics Association 2005; 12(4):410-417. 6 Nunnally, JC, Bernstein, IH. Psychometric Theory, 3rd ed. New York, NY: McGraw-Hill, 1994. 7 Aday, LA. Designing and Conducting Health Surveys: A Comprehensive Guide, 2nd ed. San Francisco, CA: Jossey-Bass, 1996. 8 http://www.ncvhs.hhs.gov/061011tr.htm 9 See ref 1 10 http://www.ncvhs.hhs.gov/061011p2b.pdf 11 http://en.wikipedia.org/wiki/HL7#Reference_Information_Model_.28RIM.29 and http://www.hl7.org/library/datamodel/RIM/modelpage_mem.htm 12 Logical Observation Identifiers Names and Codes Version 2.17. Regenstrief Institute, Indianapolis, IN. June 2005. http://www.regenstrief.org/loinc 13 http://www.regenstrief.org/medinformatics/loinc/relma 14 See ref 1. 15 http://www.nlm.nih.gov/pubs/factsheets/umlsmeta.html 16 http://www.nlm.nih.gov/research/umls/meta2.html 17 http://www.hhs.gov/healthit/ahic/ce_main.html 18 http://www.cms.hhs.gov/MinimumDataSets20/07_RAVENSoftware.asp 19 http://www.hl7.org/ctl.cfm?action=ballots.home and http://www.hl7.org/documentcenter/ballots/2007JAN/downloads/CDAR2_IMPL_CCD_I2_2007JAN.zip Modifying CCD Representation of Assessment Instrument Results Page 12 of 12