LRI Validation Suite
LRI Validation Suite Meeting
Rob Snelick—NIST
April 24th, 2012
Agenda
• Tool status and plans
• LRI DSTU Version—demonstration of use
– LIS validation is focus today
• LRI DSTU Errata
• Schedule -- Tuesdays, 1:00 PM EST
– Next meeting: May 22nd (focus is EHR validation)
– Additional meetings to be added if necessary
– HL7V2_LRI_Testing Google Group will be used for
notification
Tool Status and Plans
• Validation Tool for LRI (Release 1 for trial use)
LRI IG DSTU Version – lri-dstu
– http://lri.sipilotdevelopment.org/lri-dstu/
Notes:
– Addresses LRI IG DSTU version submitted for ballot April 9th,
2012 (Plus Errata)
– All technical requirements for LIS validation incorporated
– More testing needed and some odds and ends need
addressed
– Next target is May 22nd—EHR functionality and Test Plan
updated
– Start to move into the phase of adding test cases
• However, the framework is in place
– Changes to the tool will still be made; notice of such
changes to be announced on the LRI Testing Google Group
Resources
• Tool Documentation Tab
– Test plans, messages, profiles, and other testing
artifacts
• Wiki Page—Validation Suite Resources
– Latest version of test plans, messages, meeting
minutes, etc
• Google Group
– Discussion forum for LRI IG, Pilots, Validation Tool,
Tool and Resources Updates, Release Notes, etc
– http://groups.google.com/group/hl7v2_lri_testing
Plans
• Update EHR Validation
– Test Plan, Juror Documents, Work flow, Communication
• Testing the validation tool
– We need community feedback!
– Get a head start on MU-2 certification
• More Test Cases (Test Coverage Matrix)
–
–
–
–
Lab Test Scope (to cover vocabulary recommendations)
Scenario test cases (e.g., P  F  C)
Technical requirements (e.g., Snapshot processing)
“Negative” testing
• User’s Guide and other documentation
• Exploring/Considering LOI and Compendium Validation
– Depending on NIST resources/time
LRI Validation Tool Highlights
•
•
•
•
•
•
•
•
Web Application
LIS and EHR Validation
Context-free and context-based testing
Test Plans and Test Cases for LIS and EHR
Example/Test messages
Conformance Profile Browser (on-line requirements)
Vocabulary Browser (Searchable)
Supports IG vocabulary
– HL7, LONIC, UCUM, SNOMED, & external tables
• Support IG Profile Variations
– Globally and Non-globally Unique Identifiers Profiles
– Order Placer/Filler Number Unique and Order
Placer/Number non-unique Profiles
Phase 1 Core Test Cases/Messages
Supports both NG and GU versions:
• Erythrocyte sedimentation rate
–
–
–
–
–
–
•
30341-2 OBR.4 LN code
Valid Minimally Populated Message – Final Results
Valid Typically Populated Message – Final Results
Valid Typically Populated Message – Corrected Results
Valid Maximally Populated Message – Final Results
Valid Specimen Reject Message
CBC W Auto Differential panel in Blood
– 57021-8 OBR.4 LN code
– Valid Typically Populated Message – Final Results
– 27 Results (OBXs)
•
Lipid 1996 panel in Serum or Plasma
– 24331-1 OBR.4 LN code
– Valid Typically Populated Message – Final Results
– 4 Results (OBXs)
•
Culture Results Reporting (Relating OBX segments) – Final review needed
– 625-4 OBR.4 LN code
•
Culture and Susceptibility Results Reporting (Parent/Child Relationships) – Final
Review Needed
– 625-4 OBR.4 LN code
– RU and RN Profile Versions
Tool Demonstration
• Big Picture
– Validation Options
– Profile Viewer
– Vocabulary Browser
– Documentation
• LIS Context-free Message Validation
– Supported Profiles
• LIS Context-based Validation
– Test Plan/Test Cases Overview
– Data Sheet
– Expected Validation Results
LIS Context-Free Testing
1.
2.
3.
4.
5.
Test any LRI message created by the LIS
Select “Message Validation” Tab
Select profile version (e.g.,
Enter test messages into validation tool
Validation results appear
LIS (or
Proxy)
 Load
 Cut/paste
 Send
Process:
1. An LRI message is created by the vendor’s LIS
2. The message is sent, pasted, or loaded in the
test tool
3. A validation is performed
HL7 V2 Lab
Results
Message
LIS Test Tool
Validation
Report
Context Free Testing:
1. Context free testing will validate a LRI message
created by the LIS
2. The context (e.g., the type and results of the
lab test) is unknown to the validation tool
3. Therefore not all conformance requirements
of the LRI implementation guide can be
assessed
4. However, the validation provides a simple and
convenient method for testing message
structure and most vocabulary
LIS Context-based Testing
• A lab test is ordered for a patient
• The specimen is collected, and is received and
processed in the lab
• The lab result is produced and stored in the LIS
database
• The lab result message is created
•The lab result is transmitted to an ambulatory EHR
•The lab result is viewed in the ambulatory EHR
EHR transmits Hemoglobin blood test order to the LIS
for John Doe along with pertinent demographic and
order request data
Lab Results Data Sheet
John Doe
DOB: 05/23/1959
Gender: M
Result: 13.7 g/dL
Range 13.2 – 16.2
Status: Final
and more…
Process:
1. A technician enters lab results data into LIS
based off the data sheet provided
2. The message is sent, pasted, or loaded in the
test tool
3. A validation is performed
Use Case
Test Case
Test Data
Sheet
 Load
 Cut/paste
 Send
Context-based Testing:
1. Context-based testing will validate a LRI message
created by the LIS
2. The context (e.g., the type and results of the lab test)
is known to the validation tool
3. Therefore all conformance requirements of the LRI
implementation guide can be assessed
4. Supports various scenarios (FC, rejected specimen)
Manual entry
of test data
LIS
HL7 V2 Lab
Results
Message
LIS Test Tool
select test case
Validation
Report
LRI EHR Testing – Test Harness
Use Case
1.
2.
3.
4.
LRI EHR Test Harness
Test Case
Test Data
Inspection Testing can be performed by:
On-site inspection
Over a webex like technology
Screen-scraper or screen-capture (include clock)
Printed Reports
Inspection Testing Techniques
1. EHR display screens
2. Database access
3. Configuration files
Juror
Document
Lab Message
LRI Test
Message
EHR
Communication
ACK
ACK
Validation
Validation
Report
Automated Testing
Acknowledgement Message
Limited Utility
LRI Validation Suite WG Charter Overview
NIST LRI Test Tool
ONC S&I Framework Test Data
LIS LRI
Validation
Tool
EHR LRI
Test
Harness
Ambulatory EHR
LRI ORU R01
S & I Framework
EHR Pilots
LRI ACK R01
Vendor EHR Products
NIST LRI Test Tool
Ambulatory EHR Systems
1. Suite scope limited to the requirements specified in the LRI IG 1. EHR Pilots
2. Develop test data
1. Draft standard trial use (implementation)
1. provided & verified by the S&I Framework community
2. Verifies LRI IG can be implemented—provide feedback
3. LIS LRI Validation Tool
3. Use LRI test harness to perform conformance testing
1. Used to validate vendor LIS systems & Test Harness
4. Function of the S&I Framework LRI Pilots WG
2. Develop a test plan
2. Vendor Product EHRs
3. Capture and validate LIS LRI messages
1. Can use LRI test harness to perform conformance
4. EHR LRI Test Harness
testing
1. Used to validate EHR systems
2. LRI Test Harness expected to be used for MU
2. Develop a test plan
certification
3. Simulation of an LIS system
4. Manage, Send, Receive LRI Messages
5. Inherently provides CLIA Testing
6. Targets
1. S & I Framework EHR Pilots
2. Vendor EHR Products
3. MU Certification
Part of validation suite deliverables
Anticipated users of validation suite products
EHR transmits Hemoglobin blood test order to the LIS
for John Doe along with pertinent demographic and
order request data
Lab Results Data Sheet
John Doe
DOB: 05/23/1959
Gender: M
Result: 13.7 g/dL
Range 13.2 – 16.2
Status: Final
Use Case
Test Case
1. Local code
2. No code—text
3. LOINC code
Test Data
Sheet
No LOINC given for results
For Discussion:
1. No suggested method given (Should we?)
2. With no method given we should expect one
from a set of valid LOINC codes for this order
(Lab dependent?)
3. If a specific method is given (is this
typical/possible?) then should we look for a
specific LOINC code in the sent message?
 Load
 Cut/paste
 Send
For Discussion:
1. Does the LAB expect a local code for the order in
a electronic order request?
2. Use Cases to consider: agreement/no agreement
between LIS and EHR for local codes
Manual entry
of test data
Data Entry
• A lab test is ordered for a patient
• The specimen is collected, and is received and
processed in the lab
• The lab result is produced and stored in the LIS
database
•The lab result is transmitted to an ambulatory EHR
•The lab result is viewed in the ambulatory EHR
DB
LIS
Message
It is assumed Lab has
capabilities for manual
entry
• Test tool inspects OBX.3
for a specific LOINC code
• Inspect for 718-7 only or
inspect for one of 718-7,
20509-6, 30313-1, 30352-9,
30351-1, and 30350-3?
HL7 V2 Lab
Results
Message
LIS Test Tool
select test case
Validation
Report
For Discussion:
1. What are the requirements on the EHR with regards to LOINC—
given the LRI IG, CLIA, and meaningful use?
2. What are the EHR display (GUI) requirements for the received lab
results?
3. What are the EHR storage requirements for the received lab
results?
4. Given that the LOINC code is likely to be translated into the EHR
internal representation, what lab test names need to be
displayed in exact form and what lab test names can be displayed
in an equivalence text (same concept)?
5. What qualifies as “equivalence—same concept”? How is that
determined? Can it be? Expert inspector only? Should displaying
of the LOINC text (only) be the requirement—i.e., no mapping?
6. What is the impact of sending only the LOINC code (i.e., no local
code) on testing?
Use Case
LRI EHR Test Harness
Test Case
Juror
Document
Test Data
718-7 (LN)
LRI Test
Message
718-7 (LN)
Lab Message
ACK
ACK
Validation
EHR
Communication
Validation
Report
718-7 (LN)
S
Name
Value
REF Range
F
Hemoglobin
13.7
13.2 – 16.2 g/dL
Displayed as internal
display text representation
Translated into
internal
representation