Paths to QUALIITY
Using Evaluation Data for Program and Policy
Decision Making: Indiana’s Approach to
Implementing a Quality Rating and
Improvement System
Presenters
★ Melanie Brizzi
Child Care Administrator, Indiana Bureau of Child Care,
Division of Family Resources
★ James Elicker, Ph.D.
Associate Professor, Director, Paths to QUALITY
Evaluation, Department of Human Development &
Family Studies, Purdue University
★ Pattie Ryan
Deputy Director, Indiana Association for Child Care
Resource & Referral
Objectives
★ Paths to QUALITY™ 101
★ Examination of Evaluation Results
★ Parents; Providers; Children
★ Data
★ Use of results for implementation on state and local level to
strengthen QRIS
★ Share future considerations and actions
★ Questions and Answers
History of Paths to
QUALITY
★ Homegrown Model
★ Two local CCR&R agencies
★2000: Early Childhood Alliance (NE Indiana)
★2005: 4 Cs Southern Indiana (SW Indiana)
★ Statewide Planning Process
★ Rolled Out Statewide in Four Waves
★ January, 2008 Wave 1
★ January, 2009 Last Wave, 4
A True Collaboration
★ Indiana Family and Social Services Administration
(FSSA), Bureau of Child Care (BCC)
★ Indiana Association for Child Care Resource and
Referral (IACCRR)
★ Indiana Association for the Education of Young Children
(IAEYC)
★ The Consultants Consortium (TCC)
★ Purdue University Human Development and Family
Studies
★ Head Start Collaboration Office
★ The Indiana Department of Education
Additional Partners
★
★
★
★
★
★
★
★
★
★
United Way
Success by Six
Ball Brothers Foundation
Lilly Endowment
Child and Adult Care Food Program
Indiana Department of Environmental Management
Lowes
Hagerman Construction
Halstead Architects
Working Dog Marketing and Public Relations
Ultimate
outcomes
Longer-term
outcomes
Short term/
intermediate
outcomes
Inputs
Positive child cognitive, language, and social emotional outcomes.
More children receive
high quality
developmentally
appropriate child care.
Low quality child care programs
either improve their quality of care
or close.
Parents select high quality
over low quality child care.
Good and high quality programs
follow Paths to QUALITY
requirements, maintaining quality
child care.
Parents’ awareness and
utilization of Paths to
QUALITY rating system.
Programs continue to join Paths
to QUALITY and utilize technical
assistance support provided by
Paths to QUALITY rating
system.
Paths to QUALITY quality rating and improvement system
implementation.
Paths to QUALITY
Primary Goals
1. Provide an easily recognizable and understandable tool to
help families with their child care choice
CCR&Rs support families for the purpose of increasing access to high
quality child care and providing information about the child care
options available to them.
2. Expand the quantity and access to high quality child care
CCR&Rs support providers for the purpose of increasing access to
high quality child care for all Indiana’s children.
3. Provide a system of resources and supports to help
providers increase quality
CCR&Rs support providers to enhance the quality of child care they are
offering, through the provision of professional development, technical
assistance, and access to resources.
Committee Structure:
Paths to QUALITY Steering Committee
Paths to
QUALITY
Marketing and
Outreach
Policy and
Procedure
Data
Evaluation
Software
Steering
Standards
Work Group
Provider
Resources
The Process
Understanding the
Levels
Validity Report
★ Preliminary assessment of Paths to QUALITY standards
validity by Purdue University
★ Reviewed evaluation results of regional Paths to
QUALITY programs
★ Are the quality indicators in the proposed statewide
Paths to QUALITY standards supported by the scientific
literature?
★ Conclusion: “If implemented with diligence…the Paths to
QUALITY program has the potential to increase child
care quality…[and] we can expect Indiana’s young
children to benefit..”
Rating System
★ Independent team of raters
★ Providers receive annual visits and visits at level
increases
★ Raters utilize the same checklist
★ Alignment Committee to ensure that everyone measures
the items in a reliable way
★ Inter-rater reliability
2128 Participating
Programs
★ Licensed Homes
★ Goal 55% of licensed homes
★Actual Y3 enrollment 56% or 1568 homes
★ Licensed Child Care Centers
★ Goal 85% of licensed centers
★Actual Y3 enrollment 86% or 495 centers
★ Registered Ministries that have met Voluntary
Certification Standards
★ Goal 5% registered ministries
★Actual Y3 enrollment10% or 65 ministries
Total # of Providers as of
10/1/2009
7%
5%
13%
75%
Level 1
Level 2
Level 3
Level 4
Total # of Providers as of
10/1/2011
14%
12%
Total # of Providers as of
10/1/2010
8%
12%
68%
Level 1
Level 2
Level 3
Level 4
10%
52%
24%
Level 1
Level 2
Level 3
Level 4
A Closer Look at
Paths to QUALITY Evaluation Research
Overview of Results
Evaluation Questions
★ Do the Paths to QUALITY levels standards and ratings
discriminate real differences in child care quality?
★ What are the experiences of child care providers in
Paths to QUALITY?
★ How do parents perceive Paths to QUALITY?
★ How are children doing in Paths to QUALITY?
Does
Paths to QUALITY
actually improve child
care quality?
★ We found strong evidence that the Paths to QUALITY
rating system distinguishes levels of quality.
★ Level 4 providers are clearly higher quality than Level 1
providers.
★ There is room for improvement in quality at Paths to
QUALITY Level 4.
★ Quality we observed at each Paths to QUALITY Level is
quite variable.
ERS Results
Quality Indicator
Study:
★ Areas of Quality Strength-★ Provisions, supervision, communications with parents and
staff, professional growth
★ Supervision of play, discipline
★ Daily schedule, free play, group play
★ Areas for Potential Quality Improvement-★ Sanitation in infant/toddler care, preschool
★ Active physical play
★ Learning curriculum and interactions: blocks, nature,
science activities, support for language and thinking
Improving Quality:
Recommendations
★ Continue to refine Paths to QUALITY rating system:
★ Reassess rating instruments to make sure the criteria are
rigorous
★ Regularly monitor the precision and consistency of Paths
to QUALITY ratings statewide
★ Consider Paths to QUALITY standards
revisions/refinements that would elevate overall quality
Statewide Response
★ Celebrating Successes
★ System approach to improving quality
★ Professional Development
★ Technical Assistance
★ Licensing
★ Stakeholder Committees
Local Response
★ Technical Assistance
★ More than 30,000 hours since Paths to QUALITY rolled
out
★ Fidelity to the process
★ Case Management System
★ Support for TA Specialists and their supervisors
Why do Providers
Enroll?
★ To improve quality, gain recognition, increase business
★ Mentoring and public recognition were seen as most
important benefits of participation in Paths to QUALITY
Incentives and
Awards
Levels 1-3
Level 4
2008
2009
2010
2011
119,600
237,150
249,400
270,000
58,500
104,400
159,200
127,000
★ Total
876,150
449,100
★
★
★
★
★ Grand Total
$1,325,250
State and local
response
★ Grassroots marketing
★ Providing local and state level recognition
★ Addressing “myths” in a systematic way
Yard Signs
Banners
Specifically….
★ Local newspaper ads and other press recognizing
participating programs and level advancement
★ Low cost recognition: ribbons for conferences; CCR&R
newsletters; social media
★ Marketing toolkit
NACCRRAware
★ Highest rated appear first on referrals
★ Explanation of each rating
★ Change in profile language
★ Coming soon! Child Care Online profiles for programs
Provider Challenges to
Paths to QUALITY
Advancement
★ Time
★ Training
★ Funding
★ Documentation
★ Time Between Rating Visits
State and Local
Response
★
★
★
★
★
On Line Trainings
Webinars
Professional Development Network
Blended T/TA Model for training purposes
Training Summary Form- move to wireless webforms
Parent Awareness
★ General awareness of Paths to QUALITY among parents
is about 14% statewide
★ 35% Early Childhood Alliance
★ 45% 4 C Southern Indiana
★ Child care providers are parents’ most common source
of information about Paths to QUALITY
★ Up to 71% of families said Paths to QUALITY would
influence their future child care decisions
★ More than 50% are willing to pay more for higher-rated
providers
CCR&R Findings
Families surveyed after receiving consumer education and
referral services
★ 90% of families state they know more about how to use
Paths to QUALITY in their child care search
★ 85% of families view Paths to QUALITY rating as an
important or extremely important factor when searching
for child care
★ 72% of families report they have or intend to look for a
Paths to QUALITY program
★ 55% of families report selecting a Paths to QUALITY
provider
Branding