The Preschool Curriculum Evaluation
Research (PCER) Program
The 2005 OSEP National Early Childhood
Conference
February 8, 2005
Caroline Ebanks
James Griffin
Institute of Education
Sciences
Acknowledgements
PCER 2002 and 2003 Researchers
RTI
PCER 2002 National Evaluation Coordinator
Ina Wallace
and Holly Rhodes
Mathematica Policy Research
PCER 2003 National Evaluation Coordinator
John Love
Institute of Education Sciences
(IES)
IES Statutory Mission
• Condition and progress of education in the
United States
• Education practices that improve academic
achievement & access to education
opportunities
• The effectiveness of Federal and other
education programs
Institute of Education Sciences
Office of the Director
National Board for Education Sciences
National Center for Education Statistics
National Center for Education Research
National Center for Education Evaluation
National Center for Special Education Research
Preschool Curriculum
Evaluation Research
• Immediate Program Goals
– Provide practitioners with evidence for selecting
preschool curricula
– Support rigorous evaluations of preschool curricula using
random assignment
• Additional Goals
– Support complementary research studies to supplement
the evaluation
– Build capacity for collaboration between researchers and
practitioners
PCER: General Framework
• Three year longitudinal studies across the transition
from preschool to kindergarten and first grade
• Grantees were funded to implement curricula and conduct
complementary research studies at each project site
• Focus on supporting school readiness, especially for
disadvantaged children
• Common core of evaluation data collected by an outside
contractor
Basic Design of Evaluation
• Two groups of grantees (PCER 2002 and PCER
2003) implementing a variety of preschool
curricula
• Randomly assigned either schools, classrooms,
or children to the treatment condition
• Baseline and three follow-up assessments—end
of preschool, end of kindergarten, and end of
first grade
Primary Research Questions
• What outcomes do specific curricula
produce?
• Are outcomes sustained over time?
Secondary Research Questions
• What is the relationship between level of
implementation and participation and
curriculum effectiveness?
• How are experiences in kindergarten and
first grade related to child outcomes?
PCER Project Cohorts
IES
Cohort 1:
PCER 2002
National
Evaluator:
RTI
7
Grantees
Cohort 2:
PCER 2003
National
Evaluator:
MPR
5
Grantees
Basic Design Features
• 14 Experimental Curricula
• 12 Grantees in 13 geographic locations
-California
-Florida
-Georgia
-Kansas
-Missouri
-New Hampshire
-New Jersey
-New York
-North Carolina
-Tennessee
-Texas
-Virginia
-Wisconsin
PCER Curricula
• PCER 2002 Curricula • PCER 2003 Curricula
– Bright Beginnings (TN)
– Creative Curriculum (NH, NC,GA,
TN)
– Doors to Discovery (TX)
– Early Literacy and Learning Model
(ELLM) (FL)
– Ladders to Literacy (NH)
– Let’s Begin with the Letter People
(TX)
– Pre-K Mathematics with DLM
Express Math (CA/NY)
– Project Approach (WI)
– Curiosity Corner (FL, NJ, KS)
– The Language Focused Curriculum
(VA)
– Literacy Express (FL-FSU)
– Open Court with DLM Early
Childhood (FL-FSU)
– Project Construct (MO)
– Ready, Set, Leap! (Newark, NJ)
PCER 2002 Cohort
Grantee
Research Site(s)
Dale Farran
Vanderbilt University
Tennessee
Cheryl Fountain
University of North Florida
Florida (Bay, Jacksonville,
and Miami)
Richard Lambert
UNC Charlotte
Georgia and
North Carolina
Susan Landry
University of Texas Health Science Center, Houston
Houston, Texas
Doug Powell
Purdue University
Milwaukee, Wisconsin
Jeff Priest
University of New Hampshire
New Hampshire
Prentice Starkey
University of California, Berkeley
California and New York
PCER 2002 Sites and Curricula
Research Site
Curriculum
Tennessee
Creative Curriculum vs.
Bright Beginnings vs. Comparison
Early Literacy and Learning Model vs.
Comparison
Creative Curriculum vs. Comparison
Doors to Discovery vs. Let’s Begin
with the Letter People vs. Comparison
Project Approach vs. Comparison
Creative Curriculum with Ladders to
Literacy vs. Comparison
Pre-K Mathematics vs. Comparison
Florida
North Carolina/Georgia
Texas
Wisconsin
New Hampshire
California/New York
PCER 2003 Cohort
Grantee
Research Site(s)
Bette Chambers
Success for All Foundation
Florida, Kansas,
New Jersey
Anne Cunningham
University of California, Berkeley
Laura Justice
University of Virginia
Newark, New Jersey
Christopher Lonigan
Florida State University
Florida (Taylor and
Leon counties)
Kathy Thornburg
University of Missouri
Missouri
Virginia (Culpeper
and Wise counties)
PCER 2003 Sites and Curricula
Research Site
Curriculum
Florida, New Jersey,
Kansas
Curiosity Corner vs. Comparison
Newark, New Jersey
Ready, Set, Leap! vs. Comparison
Virginia
The Language Focused Curriculum
vs. Comparison
Florida
Literacy Express vs. Open Court/DLM
vs. Comparison
Missouri
Project Construct vs. Comparison
Grantees’ Complementary
Research
• FSU (Chris Lonigan): Implementation of Open
Court/DLM and Literacy Express
• NC/GA (Richard Lambert): Implementation of
Creative Curriculum, 4th edition
• UNH (Jeff Priest): Implementation of Ladders
to Literacy
Cross-Site Study Sample Size
• Total sample size
– 2913 children and 317 classrooms
• PCER 2002:
– 1,686 Children and 180 Classrooms
• PCER 2003:
– 1,227 Children and 137 Classrooms
Types of Measures
in the Cross-Site Study
•
•
•
•
•
Child Assessments
Classroom Observations
Teacher interviews
Teacher reports of child behavior
Parent interviews
Overview
•
•
•
•
•
Preschool program type
Demographics
Academic Outcomes
Social Outcomes
Teacher Characteristics
Types of Preschool Programs in
the PCER 2002 Cohort
6%
44%
50%
Head Start
Public School
Private Pre-K
Types of Preschool Programs in
the PCER 2003 Cohort
23%
12%
Head Start
Public School
Private Pre-K
65%
Demographics
From the Parent Interview
Children’s Age (in years)
5
4.5
Treatment
Control
4
2002 Cohort
2003 Cohort
Children’s Gender (% Male)
80
70
60
50
40
30
Treatment
Control
20
10
2002 Cohort
2003 Cohort
Race/Ethnicity
100%
80%
60%
40%
20%
0%
2002
White (non-Hispanic)
2003
African American
Hispanic/Latino
Other
Disability Status (% Yes)
(Parent report)
35
30
25
20
15
Treatment
Control
10
5
2002 Cohort
2003 Cohort
Maternal Employment Status
100%
80%
60%
40%
20%
0%
2002
2003
Other
Not Working
Part-Time
Full-Time
Maternal Educational Level
60%
50%
40%
30%
PCER 2002
PCER 2003
20%
10%
0%
< HS
HS
Some post
HS
BA or
higher
Sample Comparisons:
Poverty Indicators
60%
50%
40%
PCER 2002
PCER 2003
FACES
ECLS-K
30%
20%
10%
0%
Medicaid (Child)
AFDC/TANF
PCER
Sample Comparisons-Child
PCER 2002
PCER 2003
FACES
53%
48%
51%
50%
-White (non Hispanic)
30%
36%
31%
57%
-African American (nonHispanic)
44%
52%
26%
14%
-Asian/Pacific Islander
2%
1%
1%
7%
-Hispanic/Latino
17%
5%
30%
18%
0.4%
1%
2%
2%
6%
6%
18%
3%
15%
19%
9%
Gender (% Male)
ECLS-K
Race/Ethnicity
-American
Indian/Eskimo
-Other
Disability Status
--
Teacher Education
100%
80%
60%
40%
BA/BS or >
< BA/BS
HS or less
20%
0%
2002
Treatment
2003
Control
2003
Treatment
2003
Control
Academic Outcomes
From the Child Assessment
-Baseline Data-
Child Math Assessment (CMA)
Composite
0.5
0.45
0.4
0.35
0.3
0.25
2002 Cohort
2003 Cohort
Child Math Assessment (CMA)
Composite score,
by treatment status
0.5
0.45
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
Treatment
Control
PCER 2002
PCER2003
Woodcock Johnson-III:
Applied Problems
100
95
90
85
80
2002 Cohort
2003 Cohort
Woodcock Johnson-III: Applied
Problems, by treatment status
100
95
Treatment
Control
90
85
80
PCER 2002
PCER 2003
Test of Early Reading Ability
(TERA): Reading Quotient
100
95
90
85
80
2002 Cohort
2003 Cohort
Test of Early Reading Ability
(TERA): Reading Quotient,
by treatment status
100
95
Treatment
Control
90
85
80
PCER 2002
PCER 2003
Woodcock-Johnson-III:
Letter Word Identification
105
100
95
90
2002 Cohort
2003 Cohort
Woodcock-Johnson-III:
Letter Word Identification,
by treatment status
102
100
98
Treatment
Control
96
94
92
90
PCER 2002
PCER 2003
Woodcock-Johnson-III:
Spelling
100
95
90
85
2002 Cohort
2003 Cohort
Woodcock-Johnson-III: Spelling,
by treatment status
100
95
Treatment
Control
90
85
80
PCER 2002
PCER 2003
Peabody Picture Vocabulary
Test (PPVT) III
95
90
85
80
2002 Cohort
2003 Cohort
Peabody Picture Vocabulary
Test (PPVT) III,
by treatment status
100
95
Treatment
Control
90
85
80
PCER 2002
PCER 2003
Test of Language Development
(TOLD): Grammatical
Understanding
10
9
8
7
6
5
2002 Cohort
2003 Cohort
TOLD: Grammatical
Understanding,
by treatment status
10
9
8
Treatment
Control
7
6
5
PCER 2002
PCER 2003
Social Outcomes
From Teacher Report Form
-Baseline Data-
Social Skills Rating System
(SSRS): Social Skills
105
100
95
90
2002 Cohort
2003 Cohort
Social Skills Rating System
(SSRS): Social Skills,
by treatment status
105
100
Treatment
Control
95
90
PCER 2002
PCER 2003
SSRS: Problem Behaviors
105
100
95
90
2002 Cohort
2003 Cohort
SSRS: Problem Behaviors,
by treatment status
105
100
Treatment
Control
95
90
PCER 2002
PCER 2003
Preschool Learning Behaviors
Scale (PLBS)
55
53
51
49
47
45
2002 Cohort
2003 Cohort
Preschool Learning Behaviors
Scale (PLBS),
by treatment status
55
53
51
Treatment
Control
49
47
45
PCER 2002
PCER 2003
What we can learn from PCER
• Which curricula appear to better support
school readiness
• The type of support teachers need in order
to effectively implement curricula
Additional Benefits of PCER
• Support collaboration between practitioners
and researchers which can produce:
– Research that is more relevant to practice
– Researchers and practitioners who understand
how to be good collaborating partners
– Model for other education research programs
PCER: Contact Information
• Program Officers:
– Caroline Ebanks, PhD
E-mail: Caroline.Ebanks@ed.gov
– James Griffin, PhD
E-mail: James.Griffin@ed.gov
• PCER Websites:
– PCER 2002: http://pcer.rti.org/
– PCER 2003: www.pcer-mpr.info