Kathryn Tout
Presentation at the NACCRRA National Child Care Policy Symposium
March 8, 2012




Parent Aware Evaluation Team
◦
◦
◦
◦
◦
◦
Rebecca Starr
Tabitha Isner
Jennifer Cleveland
Meg Soli
Katie Quinn
Ladia Albertson-Junkans
Minnesota Early Learning Foundation
Greater Twin Cities United Way
Office of Planning, Research and Evaluation in the
Administration for Children and Families, U.S. Department
of Health and Human Services
◦ INQUIRE – Quality Initiatives Research and Evaluation Consortium




QRIS Compendium – information from 26 QRIS
collected in late 2009 from QRIS administrators and
publicly available documents
The majority of QRIS provide TRAINING that is aligned
with QRIS standards and ON-SITE TECHNICAL
ASSISTANCE to support programs
These supports tend to focus primarily on “navigating”
the QRIS and improving environment ratings
Training and TA address interactions with children to a
lesser extent (see new report by NCCP)

Majority of QRIS in the Compendium reported that
the frequency, length, and duration of onsite
assistance varied by the needs of the program

Work usually is guided by a QI plan. Intensity and
content of TA will vary also on the stage of the rating
cycle (pre-entry or preparation phase; entry to the
QRIS; re-rating and improvement phase) (see Isner et al,
2011)

Review findings on quality
improvement from one QRIS –
Minnesota’s Parent Aware – to
examine how well QI strategies
are working

Analyze who improves and how
they improve

Discuss implications of the
findings for improving QI in QRIS


Voluntary QRIS
Two rating pathways
◦ Full rating: 1 to 4 stars
◦ Accelerated 4-star rating for
accredited programs, Head Start
and school-based pre-kindergarten
programs

Points are earned in four quality
dimensions
◦
◦
◦
◦
Family Partnerships
Teaching Materials and Strategies
Tracking Learning
Teacher Training and Qualifications




Provider Resource
Specialist – facilitates
and oversees the
process for programs
ERS Consultant
CLASS Coach
Financial assistance for
QI expenditures
Re-Ratings in Parent Aware
1 star,
0%
4 stars,
10%
1 star,
14%
3 stars,
34%
Initial Rating
2 stars,
15%
4 stars,
43%
2 stars,
42%
3 stars,
42%
Second Rating
N=97

Improvements by at least one star level at second
rating:
◦ 60% of centers
◦ 70% of family child care providers


One-third of family child care providers increased by at
least two star levels
11% of centers decreased their rating by one star
compared to 2% of centers
Average change in points at re-rating
8
6
4
2
0
-2
Decreased by one Stayed the same
or more stars (N=5) star rating (N=28)
-4
Improved by one Improved by more
star (N=38)
than one star
(N=26)
Family Partnerships
Teaching Materials and Strategies
Tracking Learning
Teacher Training and Education




Points are earned in the Teaching Materials and
Strategies category for ERS and CLASS scores
Programs gain about 1/3rd of a point on ERS scores.
Programs gain about 1/3rd of a point on Classroom
Organization and nearly half a point on Emotional
Support.
However, no gains are made on measures of
Instructional Support.
Quality improvement supports
in Parent Aware


Seven PRSs reported on services to 83 programs (34
centers, 49 family child care)
Hours of direct contact
◦ Range: 3 to 30 hours
◦ Average: 8.2 hours


Average number of site visits per program – 3.6
PRSs spend twice as many hours of direct contact with
providers who are English Language Learners than
English-speaking providers
◦ 12.7 hours compared to 6.2 hours

PRSs spend more time with family child care providers
than center-based programs
◦ 9.9 hours compared to 6.3 hours

PRSs report that their most common activities with
programs are:
◦ Assembling the QRIS documentation packet
◦ Preparing programs for their ERS observations
◦ Picking out new materials or equipment for the program
Percent of Rating Cycles
Family child care
100
90
80
70
60
50
40
30
20
10
0
Center-based
Level of Need Average
Contact
Hours
Average
Number of
Site Visits
% that are
“very
engaged and
open to
support”
Average
Star Rating
Few needs
5.6 hours
2.7 visits
57%
3.5
Average
needs
8.8 hours
3.7 visits
90%
2.9
More needs
11.2 hours
4.8 visits
85%
2.5


Five ERS consultants reported on services to 77 programs
(29 centers, 48 family child care)
Hours of consultation
◦ Range: 2 to 40 hours
◦ Average: 13.75 hours


Average number of site visits per program – 6.4
ERS consultants spend more hours of consultation with
family child care providers than with center-based
programs
◦ 15 hours compared to 12 hours

In centers, ERS consultants worked with 1, 2, or 3
classrooms

Hours of consultation differed for family child care
providers who received different star levels
◦ 2- or 3- stars – 16.9 hours
◦ 4 stars – 10.6 hours


No difference in hours of consultation for centers that
received different star levels
Most common activities that providers report doing
with their ERS consultant
◦
◦
◦
◦
Understanding the ERS scoring system
Rearranging the physical space
Purchasing new materials
Learning hand washing and other sanitation procedures






ERS Consultants reported that programs were “very
engaged and open to consultation” in 79% of the rating
cycles
Family child care providers were slightly more engaged
(85%) compared to center-based providers (68%)
Hours spent with “very engaged” programs – 15.6 hours
Hours spent with “not engaged” programs – 9.5 hours
No correlation between hours of consultation and ERS
score for centers
Hours of consultation was negatively correlated with
FCCERS-R scores for family child care providers


Two CLASS coaches reported on services to 13 centerbased programs
Hours of coaching
◦ Range: 4 to 67 hours
◦ Average: 23.2 hours


Average number of site visits per program – 8.8
CLASS coaches worked with 1 or 2 classrooms per
center


Providers reported on the activities they did with their
CLASS Coach
The most common activities were:
◦ Observing me teach and giving me feedback – 76%
◦ Helping me organize my classroom processes to aid children’s
learning – 59%
◦ Helping me understand the CLASS scales – 41%
◦ Helping me understand the CLASS scoring – 35%

CLASS Coaches report than only 38% of providers are
“very engaged”; 23% were “somewhat disengaged or
hesitant”



The average total expenditure per rating through June,
2011 was $2,791
Average expenditure was significantly greater for
family child care programs ($2,923) compared to
centers ($2,604)
Expenditures per rating were classified into categories.
◦ The largest portion of the expenditure was made on materials
for the learning environment – books, toys, games (59%)
◦ The second largest portion was on equipment for the learning
environment – shelves, playground equipment (33%)
◦ Training and consultation, teacher resource materials and
assessment materials made up the remaining 8% of
expenditures








Free training – 65%
Free curriculum materials -65%
Free assessment materials – 61%
Financial assistance – 61%
Provider Resource Specialist – 53%
ERS Consultant – 34%
Feedback reports from ERS – 36%
Publicity materials – 24%





Programs make significant improvements in their star level
when they are re-rated
Programs make small but significant improvements on
ECERS-R, ITERS-R and FCCERS-R
Programs make small improvements on the CLASS but not
on the Instructional Support scale
Providers who are English Language Learners and family
child care providers receive more hours of support from
PRSs and ERS consultants
PRSs and ERS Consultants spend the majority of their time
with providers on activities that do not involve interactions
with children – paperwork, learning about the ERS, buying
supplies and materials



Developing models to work with providers who are not
engaged – what works best?
Continuing to develop supports for high priority
providers such as English Language Learners and family
child care providers
Focusing site visits and consultation on activities that
relate more directly to interactions with children
◦ May require a change in the Quality Standards

Ensuring that financial supports are adequate to make
“real” changes
 Please
contact Kathryn Tout
ktout@childtrends.org