The Nuts and Bolts of
Enrollment Design,
Implementation and Evaluation
Families USA Health Action 2015
Jodi A. Ray, MA
PI/Project Director
Florida Covering Kids & Families
USF Health
The Lawton & Rhea Chiles Center
Goal & Objectives of Evaluation
GOAL: to assess the effectiveness of the activities through a
meaningful, targeted evaluation of strategies and a
summative assessment of overall project success.
Determine if you
Develop strategies are reaching your
Understand why is
and tactics
goals, and which
it necessary to
strategies are
designed to
establish
achieve those
successful in
measurable goals
helping you meet
goals
them
Why Evaluate My Efforts?
Evaluation removes the guesswork
and creates a system where
evidence guides your outreach &
enrollment strategies.
Proper evaluation allows you to:
•
•
•
•
Learn from your mistakes;
Expand your successes; and
Continuously improve your efforts
Communicate success in way that stakeholders
can understand and see
3
Program Evaluation Steps
Engage
Stakeholder
s
Ensure Use
and Share
Lessons
Learned
Justify
Conclusions
Standards:
Utility
Feasibility
Propriety
Accuracy
Gather
Credible
Evidence
Source: Monitoring PPOR Efforts. CityMatch. 2005
Describe the
Program
Focus the
Evaluation
Design
Objectives
 Understand expectations for data collecting and
reporting
 Increase data accuracy and decrease human
error from data reporting
 Compile and analyze information collected for
Events & Outreach
 Compile and analyze information collected
for Navigator Enrollment Assistance
 Develop data management and internal
reporting process
• 12 Regional Navigator
projects, covering 67 counties
• Each comprised of an existing local
formal consortium of partners providing
enrollment assistance within designated
counties
• Includes over 150 trained Navigators
• Utilizes existing informal partnership
networks within each regional area to
create referral sites to serve targeted
populations
Program evaluation first steps
• Each Consortium partner developed a strategic workplan utilizing a template provided by
USF.
• Each navigator is required to input enrollment and outreach data daily to ensure
information is always current
• All Consortium Leads completed data reporting training developed by the USF Evaluation
Team
• Data is submitted via a HIPPA compliant, secure online data form
• USF completed and adheres to all IRB requirements for collecting consumer data
• All results compiled are reported to CCIIO as required
• Conduct monthly TA calls with all Consortium Leads to ensure that outcomes are
reviewed and they have up-to-date information for proper program management
• Site visits are conducted to provided one-on-one TA to each region, providing additional
qualitative data supplementing information already collected via online reporting.
• Quarterly/monthly reports on workplan progress are submitted and analyzed to
determine ongoing program plan compliance.
Events & Outreach
Google docs
Entering data related to events, outreach,
advertising and marketing
Under type of activity, select only 1.
This selection will determine the next questions to appear.
Phone Calls
Pay close attention to the
format of the response for
each item!
10/31/2014
20
The following questions are intended to capture information about E nrollment Event and Outreach Efforts
designed to inform consumers .about signing up tor health care coverage through the Federal Health Insurance
Marketplace.
Information entered into this survey should be tor Project Performance Period September 16, 2014 through
September 15, 2015. Any outreach or enrollment event information prior to this period should be entered in the
Year 1
linK.
Please select your Consortium Partner Lead.
Consortium Partner Lead
•
Please indicate the type of Outreach/Enrollment activity tor which you are entering information.
O
I0
O
O
Daily Phone Calls
Education and/or Enrollment Ev ent
I
Additional Outreach/Education E fforts (Example: Flyers, Mail-outs, New sletters, Email Blasts, W ebinar, Presentation)
Advertising and Marketing Activ ities (Example: New spaper/Print Media, Ra dio, TV, Social Media)
>>
Events
Get Covered Enrollment Event with USF College of Public Health
10/31/2014
Partnered with USF College of Public Health to provide education and distribute information
to students about the upcoming open enrollment period.
3
200
170
75
How did
advertise this event?
;zi Flyers
;zi Facebook
O
Tw itter
0
L inkedln
O
Other forms of social networking
;zi Organizational w ebsite
O
Mail out
0
New sletter
:£1Email blast
O
Promo Board
O
Other (please describe):
<<
>>
The following questions are intended to capture information about E nrollment Event and Outreach Efforts
designed to inform consumers .about signing up tor health care coverage through the Federal Health Insurance
Marketplace.
Information entered into this survey should be tor Project Performance Period September 16, 2014 through
September 15, 2015. Any outreach or enrollment event information prior to this period should be entered in the
Year 1
linK.
Please select your Consortium Partner Lead.
Consortium Partner Lead
•
Please indicate the type of Outreach/Enrollment activity tor which you are entering information.
O
O
O
O
Daily Phone Calls
Education and/or Enrollment Ev ent
Additional Outreach/Education E fforts (Ex ample: Flyers, Mail-outs, New sletters, Email Blasts, W ebinar, Presentation)
Advertising and Marketing Activ ities (Ex ample: New spaper/Print Media, Ra dio, TV, Social Media)
>>
Additional Outreach/Education
Select the month
of the outreach
effort
Indicate the type
of effort and the
estimated reach
or date
Some selections have prompts for additional
information. Be as detailed as possible as this
information is used in both internal and external
reporting !
The following questions are intended to capture information about E nrollment Event and Outreach Efforts
designed to inform consumers .about signing up tor health care coverage through the Federal Health Insurance
Marketplace.
Information entered into this survey should be tor Project Performance Period September 16, 2014 through
September 15, 2015. Any outreach or enrollment event information prior to this period should be entered in the
Year 1
linK.
Please select your Consortium Partner Lead.
Consortium Partner Lead
•
Please indicate the type of Outreach/Enrollment activity tor which you are entering information.
O
O
O
O
Daily Phone Calls
Education and/or Enrollment Ev ent
Additional Outreach/Education E fforts (Ex ample: Flyers, Mail-outs, New sletters, Email Blasts, W ebinar, Presentation)
Advertising and Marketing Activ ities (Ex ample: New spaper/Print Media, Ra dio, TV, Social Media)
>>
Advertising and Marketing
Select the month
of the
advertising or
marketing
activity
Indicate the type
of effort and the
estimated reach
Advertising and Marketing
When prompted for additional information, be as
detailed as possible
as this information is used in both internal and external
reporting !
Verification & eSignature
Must complete
this section for
data to count.
Click the
Affirmation
Box
Type your full
name
Navigator Enrollment
Appointment Survey
Qualtrics
Entering information related to Navigator
appointments
Enter your Partner Name
Enter your Navigator ID
in this format
•
•
Select only one:
New/Existing lead to same set of
questions
SHOP leads to different questions
Enter zip code of the consumer being assisted
Enter size of entire household (all members)
Enter only the number of consumers in
household who are being assisted
Enter the primary language of the consumer.
Only check the box above if the language is
other than English.
This should match the number in the household
who are receiving Navigator services
Reminder:
This should match the number in the household
who are receiving Navigator services
Assistance provided during appointment
Reminder:
Check all that apply!
Outcome of appointment
If there is a text box below an item,
be sure to enter a number!
Reminder:
Check all that apply!
Enrollment information
Reminder:
Consider these questions for the
outcome of this individual
enrollment appointment, not
previous appointments.
Please be as specific as possible when
entering the name of the health care
plan and medal tier selected.
ONLY record plan if enrolled in this
visit.
•
•
•
•
When entering additional notes:
Do not type INALL CAPS
Be as detailed as possible
Okay to type in English or Spanish
Share positive stories as well as
challenges and barriers
Looking Beyond the Contract
Now that you’ve met your
contractual obligations how
do you critique the
effectiveness of your
project?
35
Heat Map
Data is transferred from Qualtrics
Jodi Ray
Project Director
(813) 974-3143
jray@health.usf.edu