Child Outcomes for Early
Intervention and Preschool Special
Education: The National Picture
Kathy Hebbeler
Lynne Kahn
ECO at SRI
ECO at FPG
Presented at
OSEP Project Directors Meeting
Washington, DC
July, 2012
Overview
• National numbers and the quality of the state
data
• The findings for 2010-11
• Findings for the last 3 years
• State variation and data quality
Early Childhood Outcomes Center
2
OSEP Reporting Requirements:
Child Outcomes
Early Childhood Outcomes Center
• Positive social emotional
skills (including positive
social relationships)
• Acquisition and use of
knowledge and skills
(including early language/
communication [and early
literacy])
• Use of appropriate behaviors
to meet their needs
3
If you have not seen it….
Early Childhood Outcomes Center
4
State Approaches to Outcomes Data, 2010-11
Approach
Part C
Preschool
(56 states/jurisdictions)
(59 states/jurisdictions)
COS* 7 pt. scale
43/56 (77%)
36/59 (61%)
One tool statewide
7/56 (13%)
9/59 (15%)
Publishers’ online
analysis
3/56 (5%)
6/59 (10%)
Other
5/56 (9%)
8/59 (14%)
*Child Outcomes Summary Rating
Child Outcomes: What we know
• States are in various stages with regard to
building their child outcomes
measurement systems
• Some phrased in
• Some changed approaches
Early Childhood Outcomes Center
6
3 Methods Methods for
Calculating National Estimates
1. States with the highest quality data
weighted by child count to represent all
states*
2. All states weighted by child count
3. All states averaged (each state weighted
as 1)
*The data we will be presenting for the national picture
Early Childhood Outcomes Center
7
Identifying the States with Highly
Questionable Data
Criteria for eliminating states:
• Not reporting data on enough children
• Odd patterns in the data
• Review of method reported suggested the
data were of questionable quality
Early Childhood Outcomes Center
8
Continued increase in the number
of states included
Number of States that Met Criteria
for Inclusion in the National Analysis
08-09 09-10
10-11
Pt. C
19
29
39
Part B Preschool
15
33
36
Early Childhood Outcomes Center
9
• And now the data….
And now the
data….
Early Childhood Outcomes Center
10
OSEP Reporting Categories
Percentage of children who:
a. Did not improve functioning
b. Improved functioning, but not sufficient to move nearer
to functioning comparable to same-aged peers
c. Improved functioning to a level nearer to same-aged
peers but did not reach it
d. Improved functioning to reach a level comparable to
same-aged peers
e. Maintained functioning at a level comparable to sameaged peers
3 outcomes x 5 “measures” = 15 numbers
Early Childhood Outcomes Center
11
Illustration of 5 Possible Paths
70
60
Score
50
40
30
20
10
0
1
6
11
16
21
26
31
36
41
46
51
56
Age in Months
Maintained functioning comparable to age peers
Achieved functioning comparable to age peers
Moved nearer functioning comparable to age peers
Made progress; no change in trajectory
Did not make progress
*Based on 39 states with highest quality data
13
*Based on 36 States with the highest quality data.
14
The Summary Statements
1. Of those children who entered the program below
age expectations in each outcome, the percent who
substantially increased their rate of growth by the
time they turned 3 [6] years of age or exited the
program.
2. The percent of children who were functioning within
age expectations in each outcome by the time they
turned 3 [6] years of age or exited the program.
Early Childhood Outcomes Center
15
*Based on 39 states with highest quality data
16
*Based on 36 States with the highest quality data.
17
Part C: Trends over Time
Early Childhood Outcomes Center
18
Part C: Trends over Time
Early Childhood Outcomes Center
19
Part B Preschool: Trends over Time
Early Childhood Outcomes Center
20
Part B Preschool: Trends over Time
Early Childhood Outcomes Center
21
Looking for Patterns
Early Childhood Outcomes Center
22
Early Childhood Outcomes Center
23
%
Early Childhood Outcomes Center
24
Early Childhood Outcomes Center
25
Building the national
capacity to report and use
child outcomes data is a
long term endeavor.
Early Childhood Outcomes Center
26
Part C: States continue to report on
more children
Total Number of Children States Included in the Child Outcomes Data
Number of
children
reported
Number of States and Jurisdictions
2007-08
(N=56)
2008-09
(N=56)
2009-10
(N=54)
2010-11
(N=54)
99 or less
13
6
5
4
100-499
25
16
9
10
500-999
6
13
10
7
1,000-1,999
9
11
10
13
2,000+
3
10
20
22
Early Childhood Outcomes Center
27
Early Childhood Outcomes Center
28
Part B Preschool: States continue
to report on more children
Total Number of Children Included in Progress Data
Number of States and Jurisdictions
Number of
children
reported
FFY 2007
(N=58)
FFY 2008
(N=58)
FFY 2009
(N=58)
FFY 2010
(N=59)
10 or fewer
1
1
1
1
10-99
11
7
6
7
100–499
14
6
4
2
500-999
10
12
10
8
1000–1999
8
7
9
5
2000–2999
5
10
11
12
3000–4999
5
11
9
13
5000-8999
3
3
7
9
9000+
1
1
1
2
Early Childhood Outcomes Center
29
Early Childhood Outcomes Center
30
%
Early Childhood Outcomes Center
31
Early Childhood Outcomes Center
32
Conclusions
• The data show good outcomes for many
children who attend Part C and Part B,
Preschool.
• The data quality continues to get better.
• The data will be a powerful tool for
program improvement when
– The data are consistently good at the local level
– States and local programs consistently use it.
Early Childhood Outcomes Center
33
For more information….
www.the-eco-center.org
Early Childhood Outcomes Center
34
Early Childhood Outcomes Center
35
Identifying States with Quality Data
Early Childhood Outcomes Center
36
Calculating Missing Data for Part C
Proxy for missing data =
Number with data for C3/
Exiting Data (618)
• Do not expect this number to be
100%
• ..but we don’t expect it to be 10%
either
Early Childhood Outcomes Center
37
Percentage of Exiters Reported by
Part C Continues to Increase
Early Childhood Outcomes Center
38
Calculating Missing Data for 619
Proxy for missing data =
Number with data for B7/
Child count
• Do not expect this number to be
100%
• ..but we don’t expect it to be 10%
either
Early Childhood Outcomes Center
39
Percentage of Child Count Reported by
Part B Preschool Continues to Increase
Pt. B Preschool: % of Child Count Reported in Child
Outcomes Data
18
Number of States
16
<10%
14
10-20%
12
10
21-30%
8
31-40%
6
41-50%
4
>50%
2
0
08-09
Early Childhood Outcomes Center
09-10
10-11
40
Many states reporting a low percentage
for “a” but still have a few outliers.
Number of States by % Reported in “a,” 2010-11
<2%
2 to <5%
5 to <7%
Part C
32
11
5
Part B
Preschool
30
11
3
7 to <13%
0
5
28 to 49%
3
0
Early Childhood Outcomes Center
41
Number
excluded
Percent
(n=51)
State is sampling
No outcomes data reported
Reported outcomes data on less than 28% of
reported exiters
Had at least one outcome with category a
greater than 10% or category e greater than
65%
3
0
3
6
0
6
4
8
Reported outcomes data on less than 28% of
reported exiters
1
2
1
2
39
76
Reason Part C state was excluded
AND
Had at least one outcome with category a
greater than 10% or category e greater than
65%
Questionable data quality based on review of
SPP/APR and knowledge gained through TA
States included in the analysis
Early Childhood Outcomes Center
42
Number
excluded
Percent
(n=51)
4
1
8
2
2
4
4
8
Reported outcomes data on less than 12% of
child count
AND
Had at least one outcome with category a
greater than 10% or category e greater than 65%
0
0
Questionable data quality based on review of
SPP/APR and knowledge gained through TA
2
4
1
2
1
2
36
71
Reason Part B state was excluded
State is sampling
No outcomes data reported
Reported outcomes data on less than 12% of
child count
Had at least one outcome with category a
greater than 10% or category e greater than 65%
No child count data available
Reported Summary statements but no progress
categories
States included in the analysis
Early Childhood Outcomes Center
43
Producing National Estimates Using the
States with the Highest Quality Data
1. Divide the states into those with large, medium,
and small child counts.
2. Identify the states that appear to have the best
data.
3. Weight by child count and calculate the overall
OSEP percentages using the “best data” states
in each category.
4. Use the OSEP percents and total child count
for each category to get the national estimates.
Early Childhood Outcomes Center
44