Encounter Data Validation:
Review and Project Update
Presenters:
Thomas Miller, MA
Executive Director, Research and Analysis Team
Amy Kearney, BA
Associate Director, Research and Analysis Team
1
Welcome
 About the presenters
 Rules for engagement
 Presentation overview
• The importance of encounter data
• CMS protocols
• Florida EDV study, including best
practices for medical record
procurement
2
Objectives
1. Learn why Encounter Data Validation
studies are important.
2. Identify the core evaluation components
outlined in CMS’ protocols for validating the
quality of encounter data.
3. Review status of Florida Medicaid’s Year
One encounter data validation study.
Discuss best practices for medical record
procurement.
3
Importance of Encounter Data
 Accurate and complete data are critical
to success of managed care programs
 Essential for overall management and oversight of
Florida’s Medicaid program
– Ability to monitor and improve quality
of care
– Establish performance measures
– Generate accurate and reliable reports
– Obtain utilization and cost information
4
Importance of Encounter Data
 Used by MCOs and the State for many purposes
–
–
–
–
–
–
Performance measure development and calculation
Performance improvement measurement
Focused studies/quality activities
Rate-setting
Compliance monitoring
Provider practice patterns
5
Key Trends
 Importance of Federal and State monitoring
– Development of core measurement sets
• Medicare versus Medicaid
• Health care reform
• Holding health care accountable
 Data, not anecdotes
6
Objectives
1. Learn why Encounter Data Validation
studies are important.
2. Identify the core evaluation components
outlined in CMS’ protocols for validating the
quality of encounter data.
3. Review status of Florida Medicaid’s Year
One encounter data validation study.
Discuss best practices for medical record
procurement.
7
8
Objectives
1. Learn why Encounter Data Validation
studies are important.
2. Identify the core evaluation components
outlined in CMS’ protocols for validating the
quality of encounter data.
3. Review status of Florida Medicaid’s Year
One encounter data validation study.
Discuss best practices for medical record
procurement.
9
EQR Protocol
 Developed and refined with the evolution of the
External Quality Review program
10
EQR Protocol
 Guidelines for External
Quality Review
Organizations (EQRO) to
use when assessing
completeness and accuracy
of encounter data.
 Data submitted by Managed
Care Organizations (MCO)
to the State
11
EQR Protocol
 State establishes standards for
encounter data
 State must establish the
following standards:
– Definition of “encounter”
– Types of encounters
– Data accuracy and
completeness
– Objective standards for data
comparison
12
EQR Protocol
 Five key activities
1. Review state
requirements
2. Review MCO’s
capability
3. Analyze electronic
encounter data
4. Review of medical
records
5. Submission of findings
and recommendations
13
EQR Protocol
 Attachment A: Encounter Data Tables
Table 2: Data Element Validity Requirements
14
EQR Protocol
 Five key activities
1. Review state requirements
• Develop understanding of State-specific policies
and procedures for collecting and submitting
encounter data
• Identify data exchange protocols and layouts
• Evaluate encounter data system interchange
flows, including system edits and submission
timelines
• Review existing encounter data quality activities,
requirements, and performance standards
15
EQR Protocol
 Five key activities, continued
2. Review MCO’s capability
• Develop, conduct, and review MCO’s
Information System Capabilities Assessment
–
•
Identification of IS vulnerabilities
Key informant interviews
16
EQR Protocol
 Five key activities, continued
3. Analyze electronic encounter data
• STEP 1 - Develop data quality test plan to
determine:
– Magnitude and type of
missing encounter data
– Overall data quality issues
– MCO data submission issues
17
EQR Protocol
 Five key activities, continued
3. Analyze electronic encounter data
• STEP 2 - Verify integrity of encounter data
– Macro-level analysis
– Encounter file completeness and
reasonableness
» Volume and utilization by encounter type and
service setting
» Internal field consistency
» General field completeness and validity
18
EQR Protocol
 Five key activities, continued
3. Analyze electronic encounter data
• STEP 3 – Generate and Review Analytic
Reports
– Micro-level analysis
– Encounter record completeness and
reasonableness
19
EQR Protocol
 Five key activities, continued
3. Analyze electronic encounter data
• STEP 4 – Compare findings to state-identified
standards
– Identification of appropriate benchmark
population
20
EQR Protocol
 Five key activities, continued
4. Review of medical records
•
•
•
Verification of the accuracy of coding
Protocol assumptions
STEP 1 – Determine sampling for medical record
review
– Identify valid sample size
– Encounter- vs. recipient-based samples
21
EQR Protocol
 Five key activities, continued
4. Review of medical records
• STEP 2 – Obtain and review medical records and
document findings
– Procurement efficiencies
– Abstraction staff and training
– Categorization of errors by level, type, and
source
– Procurement tracking and abstraction tools
22
EQR Protocol
 Five key activities, continued
5. Submission of findings
• Narrative report summarizing findings from
Activities 1-4
• Actionable recommendations for overall encounter
data quality improvement
23
Questions?
Whatcha talkin’ about?
24
Objectives
1. Learn why Encounter Data Validation
studies are important.
2. Identify the core evaluation components
outlined in CMS’ protocols for validating the
quality of encounter data.
3. Review status of Florida Medicaid’s Year
One encounter data validation study.
Discuss best practices for medical record
procurement.
25
26
Objectives
1. Learn why Encounter Data Validation
studies are important.
2. Identify the core evaluation components
outlined in CMS’ protocols for validating the
quality of encounter data.
3. Review status of Florida Medicaid’s Year
One encounter data validation study.
Discuss best practices for medical record
procurement.
27
SFY 2013-2014 Encounter Data
Validation (EDV) Study
Agency for Health Care Administration
VALIDATION OF ENCOUNTER
DATA
28
Year One Encounter Data
Validation (EDV) Study
 Four key steps for conducting successful evaluations
– Project implementation
– Study design
– Data collection &
analysis
– Reporting &
recommendations
29
Year One Encounter Data
Validation (EDV) Study
 Study design
– Prepared and finalized methodology which included:
•
•
•
•
•
Study objectives and research questions
Data source and collection procedures
Measurement methodology
Analytic methods
Timeline
30
Year One Encounter Data
Validation (EDV) Study
 Data collection and analysis
– Information systems review
• Questionnaire for AHCA
– Assessment of AHCA’s policies and procedures
for data exchange, its capacity and ability to
acquire and process data, and its staff responsible
for executing data processing
• Questionnaire for MCOs
– Assessment of MCOs’ claims processing systems
and processes, and its capability to submit
encounter data
31
Year One Encounter Data
Validation (EDV) Study
 Data collection and analysis, continued
– Information systems review
• MCO questionnaire divided into five sections:
1. Submitting Encounter Data to AHCA
2. Handling Submission Information from AHCA
3. Encounter Data Submission from Capitated Providers
4. Processing and Submission of Medicare Crossover
and other Third Party Claims
5. Policies and Procedures in Processing Payment
Information
32
Year One Encounter Data
Validation (EDV) Study
 Data collection and analysis, continued
– Information systems review
• AHCA questionnaire divided into three sections:
1. Data Exchange Policies and Procedures
2. Data Submission Processing Procedures and
Personnel
3. Encounter Data Processing within the Florida
MMIS
33
Year One Encounter Data
Validation (EDV) Study
 Data collection and analysis, continued
– Information systems review
• Documentation will be used to assess encounter data quality
• Questions target how data moves through AHCA’s data
systems and how the MCOs prepare data files for
submission
34
Year One Encounter Data
Validation (EDV) Study
 Data collection and analysis, continued
– Information systems review
What has been completed?
• Questionnaires were approved by AHCA and
distributed to AHCA and the MCOs
• Received completed questionnaires from AHCA and
MCOs
What needs to be completed?
• Currently reviewing responses from AHCA and MCOs
• May conduct additional follow-up for clarification
35
Questions?
Year One Encounter Data
Validation (EDV) Study
 Data collection and analysis, continued
– Comparative data analysis of AHCA and MCO
encounter data
• Evaluates the extent to which encounters submitted by
MCOs to AHCA are accurate, complete, and reasonable
– Included all claim/service types—i.e.,
inpatient/outpatient, physician visits, dental, and
pharmaceutical
37
Year One Encounter Data
Validation (EDV) Study
 Data collection and analysis, continued
– Comparative data analysis of State and MCO
encounter data
• Indicators to measure degree of completeness and
accuracy for each encounter type
– Overall record matching—percentage of state
encounters present in MCO files
– Field-level matching—percentage of state
encounters with exact value match in MCO file for
each select data element
38
Year One Encounter Data
Validation (EDV) Study
 Data collection and analysis, continued
– Comparative data analysis
What has been completed?
• Distributed data submission requirements documents to
AHCA and MCOs
• Conducted technical assistance sessions with MCOs on
9/16 & 9/17/2013
• Received, processed, and loaded encounter data
39
Year One Encounter Data
Validation (EDV) Study
 Data collection and analysis, continued
– Comparative data analysis of State and MCO
encounter data
What needs to be completed?
• Conducting preliminary file review
– Ensuring files are sufficient for processing
– Completing basic checks
• Generate comparative analysis tables and figures for
final report
40
Phew… Questions?
41
Year One Encounter Data
Validation (EDV) Study
 Data collection and analysis, continued
– Medical record review
• Represents the “gold standard”
• Evaluation of service level accuracy
and completeness
• Methodology developed:
– Only includes MCOs operational as of January 2013
– Year One – SFY 2016: review one-third of plans each
year as selected by AHCA
– Minimum 50 cases reviewed per plan
– Target professional, dental, and inpatient/outpatient
encounters
42
Year One Encounter Data
Validation (EDV) Study
 Data collection and analysis, continued
– Medical record review
• Sample selection methodology
1. To generate list of randomly selected encounters for
medical review, HSAG will use AHCA’s data files from
comparative analyses
2. Two-stage stratified sampling design used to ensure:
» Member’s record is selected only once
» Number of encounters included in final sample covers
all encounter types and proportional to total
distribution of encounters
43
Year One Encounter Data
Validation (EDV) Study
 Data collection and analysis, continued
– Medical record review
• Sample selection methodology
– HSAG will evaluate the key data elements below:
Key Data Elements for Medical Records Review
Key Data Fields
Date of Service
Dental
√
Diagnosis Code
CPT/CDT/HCPCS
Code/ Surgical
Procedure Code
√
Inpatient/
Outpatien Physician
t
√
√
√
√
√
√
44
Year One Encounter Data
Validation (EDV) Study
 Data collection and analysis, continued
– Medical record review
• Procurement and abstraction process
– Based on established policies and procedures
– Continually monitored to ensure validity and
accuracy
» Inter-rater reliability testing & Rater-tostandard testing
» All reviewers must achieve 95% accuracy rate
» Variety of reports will be generated, i.e.,
medical record compliance rates
45
Year One Encounter Data
Validation (EDV) Study
 Data collection and analysis, continued
– Medical record review – analysis of cases
• Verify the service(s) provided on selected data of service
and one additional date of service
• Each enrollee listed on sample has corresponding selected
date of service
• Validate services conducted by provider on date of service
as compared with encounter data
• Reviewers to validate services for additional date of
service.
46
Year One Encounter Data
Validation (EDV) Study
 Data collection and analysis, continued
– Medical record review – analysis of cases
• Analyze record completeness and the accuracy of coding
• Four primary indicators for data completeness and
accuracy
1. Medical Record Agreement
2. Medical Record Omission (surplus)
3. Encounter Record Omission (missing)
4. Erroneous
47
Year One Encounter Data
Validation (EDV) Study
 Data collection and analysis, continued
– Medical record review
What has been completed?
– Introductory letter sent to MCOs on 10/1/13
– Conducted technical assistance calls with all
participating MCOs on 10/16 & 10/18/13
48
Year One Encounter Data
Validation (EDV) Study
 Data collection and analysis, continued
– Medical record review
What needs to be completed?
– Pull samples and send lists of study cases
– Provide letter to send to its providers with sample
– MCOs will procure records from provider and
accommodate various submission methods
– MCOs to submit identified medical records to HSAG
for review
49
Year One Encounter Data
Validation (EDV) Study
 Reporting and recommendations
– Prepare aggregate EDV report of findings from:
• Information system review
• Comparative Analysis
• Medical Record Review
– Preparation of supplemental findings
for future evaluation by MCOs
– Present statewide and MCO-specific results
– Actionable recommendations for improvement
50
Objectives
1. Learn why Encounter Data Validation
studies are important.
2. Identify the core evaluation components
outlined in CMS’ protocols for validating the
quality of encounter data.
3. Review status of Florida Medicaid’s Year
One encounter data validation study.
Discuss best practices for medical record
procurement.
51
Questions