Welcome to North Euphoria
The Early Childhood Outcomes Center
September 2011
Data Workshop: Activity 1
Background: Statewide Child Outcomes Data for North Euphoria
Statewide Child Outcomes Data 2010 - 11 (N = 4751)
60%
50%
40%
30%
Outcome A
20%
Outcome B
10%
0%
Outcome C
%a
%b
%c
%d
%e
Outcome A
3.2%
7.8%
13.5%
24.0%
51.5%
Outcome B
3.3%
11.2%
18.9%
23.5%
43.0%
Outcome C
2.5%
6.7%
12.4%
23.5%
54.9%
Statewide Child Outcomes Data
2010 - 11
100%
80%
60%
40%
SS1
20%
SS2
0%
Outcome A (10-11)
Outcome B(10-11)
Outcome C(10-11)
SS1
77.3%
74.5%
79.5%
SS2
75.5%
66.6%
78.4%
SS1 Outcome A N = 2309; Outcome B N = 2701; Outcome C N = 2144
SS2 N = 4751
What do you think about North Euphoria’s data?
The Early Childhood Outcomes Center
September 2011
What comparisons would help you decide whether these are “good” results?
National Child Outcomes Data
45
40
35
30
25
20
Outcome A
15
Outcome B
10
Outcome C
5
0
%a
%b
%c
%d
%e
Outcome A
4.1
17.7
13.5
25.2
39.5
Outcome B
4.3
20.1
19.8
32.1
23.7
Outcome C
4.4
17.9
16.2
31.3
30.2
80.0%
70.0%
60.0%
50.0%
40.0%
SS1
30.0%
SS2
20.0%
10.0%
0.0%
Outcome A
Outcome B
Outcome C
SS1
64.0%
68.0%
68.1%
SS2
64.7%
55.8%
61.5%
How does North Euphoria compare to the national data?
The Early Childhood Outcomes Center
September 2011
Child Outcomes Data for States with Similar Population Size
50%
40%
30%
Outcome A
20%
Outcome B
Outcome C
10%
0%
%a
%b
%c
%d
%e
Outcome A
3%
16%
13%
28%
39%
Outcome B
3%
16%
22%
36%
23%
Outcome C
3%
15%
17%
35%
31%
80%
70%
60%
50%
40%
SS1
30%
SS2
20%
10%
0%
Outcome A
Outcome B
Outcome C
SS1
64%
75%
75%
SS2
67%
59%
66%
How does North Euphoria compare to other states of similar size?
The Early Childhood Outcomes Center
September 2011
Child Outcomes Data for States with Similar Percent Served
50%
45%
40%
35%
30%
25%
20%
15%
10%
5%
0%
Outcome A
Outcome B
Outcome C
%a
%b
%c
%d
%e
Outcome A
3%
19%
11%
23%
44%
Outcome B
5%
19%
16%
32%
28%
Outcome C
4%
18%
14%
35%
29%
80%
60%
40%
SS1
SS2
20%
0%
Outcome A
Outcome B
Outcome C
SS1
62%
67%
69%
SS2
68%
61%
64%
How does North Euphoria compare to other states with a similar percentage of children
served?
What might be reasons (hypotheses) why North Euphoria has different patterns than
similar states?
The Early Childhood Outcomes Center
September 2011
Data Workshop: Activity 2
Hypothesis checking- Do we have enough data to trust the findings? Is the data stable?
North Euphoria began statewide, census data collection in June 2006. All children
exiting each year with at least 6 months of service should be in their system. Data for all
children who exited the program in the past 4 years between July 1– June 30 are
reported below.
Year
Total number of
children with
outcomes data
reported
2007 - 08
2008 - 09
2009 - 10
2010 - 11
900
3200
4000
4741
Total number of
children exiting
6000
6400
6061
6773
Number of
children exiting
with less than 6
months of
service
500
588
900
700
Percent of
missing data
83.6
44.9
22.5
21.9
N’s for Summary Statement 1 by reporting year and outcome
Year
2007 - 08
2008 - 09
2009 - 10
2010 - 09
Outcome B
225
1024
1960
2701
N’s for Summary Statement 2 by reporting year
Year
2007 - 08
2008 - 09
2009 - 10
2010 - 11
N
900
3200
4000
4741
How would you describe the trends in N’s in the state?
The Early Childhood Outcomes Center
September 2011
North Euphoria progress category trends from 2007 – 2011 for Outcome B
80%
70%
60%
50%
40%
2007 - 08
30%
20%
2008 - 09
10%
2009 - 10
0%
OC-B_% a OC-B_% b OC-B_% c OC-B_% d OC-B_% e
2007 - 08
11%
8%
4%
2%
75%
2008 - 09
9%
6%
7%
10%
68%
2009 - 10
1%
9%
19%
20%
51%
2010 - 11
3%
11%
19%
24%
43%
2010 - 11
Summary Statement trends from 2007 – 2011 for Outcome B
90%
80%
70%
60%
50%
40%
OC-B_SS1
30%
OC-B_SS2
20%
10%
0%
2007 - 08
2008 - 09
2009 - 10
2010 - 11
OC-B_SS1
24%
53%
80%
75%
OC-B_SS2
77%
78%
71%
67%
How would you describe the changes in the data trends over the last four years?
Do the data seem stable enough to trust?
The Early Childhood Outcomes Center
September 2011
Data Workshop: Activity 3
Hypotheses checking- Do particular programs have challenges with data quality or
program quality that are influencing the state data?
Progress categories for Programs in North Euphoria on Outcome B
80%
70%
60%
50%
OC-B_% a
40%
OC-B_% b
30%
OC-B_% c
OC-B_% d
20%
OC-B_% e
10%
0%
90%
80%
70%
60%
50%
40%
SS1
30%
SS2
20%
10%
0%
Which programs have unusual data patterns?
What might be reasons for the unusual patterns?
The Early Childhood Outcomes Center
September 2011
Percent Gains in Outcome B from Entry to Exit by Program
70%
60%
50%
40%
30%
20%
10%
0%
P1
P2
P3
P4
P5
P6
P7
P8
P9
(238) (1425) (475) (257) (950) (380) (143) (285) (333)
P10
(48)
P11
(109)
P12
(48)
P13
(62)
State
Are there programs where children seem to make greater gains than in other programs?
What might be reasons (hypotheses) for these differences?
The Early Childhood Outcomes Center
September 2011
Hypotheses Checking- Do some programs show patterns that suggest problems with
data quality?
Outcome B- Program values for category “a”
25%
20%
15%
10%
5%
0%
P1
P2
P3
P4
P5
P6
P7
P8
P9
(238) (1425) (475) (257) (950) (380) (143) (285) (333)
OC-B_% a
2%
2%
4%
1%
1%
10%
2%
2%
1%
P10
(48)
P11
(109)
P12
(48)
P13
(62)
State
3%
4%
22%
2%
3%
P10
(48)
P11
(109)
P12
(48)
P13
(62)
State
16%
42%
22%
68%
43%
Outcome B- Program values for category “e”
80%
70%
60%
50%
40%
30%
20%
10%
0%
P1
P2
P3
P4
P5
P6
P7
P8
P9
(238) (1425) (475) (257) (950) (380) (143) (285) (333)
OC-B_% e 35%
52%
42%
35%
40%
21%
42%
38%
65%
What programs stand out as having unusual patterns?
The Early Childhood Outcomes Center
September 2011
Hypotheses Checking- Do some programs have problems with missing data?
Missing data by program for 2010-11
Program
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
State
Total
number
of exiters
with child
outcomes
data
238
1425
475
257
950
380
143
285
333
48
109
48
62
4751
Total exiters
with more than
6 months of
service
Percent of missing data
242
1520
507
328
1515
486
160
300
425
150
110
50
280
6073
2%
6%
6%
22%
37%
22%
11%
5%
22%
68%
1%
5%
78%
22%
Are there any programs that may have problems with missing data?
Which programs have the most influence on the state data?
The Early Childhood Outcomes Center
September 2011
Hypotheses Checking- Are some programs’ data more stable over time than others?
Three years of trend data for SS1 for Outcome B
Program
P1 (238)
P2 (1425)
P3 (475)
P4 (257)
P5 (950)
P6 (380)
P7 (143)
P8 (285)
P9 (333)
P10 (48)
P11 (109)
P12 (48)
P13 (62)
SS1 (2008- SS1 (2009- SS1 (201009)
10)
11)
61%
88%
83%
54%
81%
83%
56%
83%
78%
54%
81%
76%
12%
39%
34%
11%
38%
33%
57%
84%
79%
53%
80%
75%
46%
73%
68%
31%
35%
79%
58%
85%
80%
37%
64%
59%
87%
48%
83%
Does the data for any of these programs look particularly unstable?
The Early Childhood Outcomes Center
September 2011
Overall for Activity 3
What potential red flags have you identified in local programs’ data?
What might be reasons (hypotheses) about why some programs have unique patterns
compared to other programs in the state?
The Early Childhood Outcomes Center
September 2011
Data Workshop Activity 4
Drilling down- Are differences related to data quality or program characteristics?
Hypotheses Checking- Do some programs serve a different population of children than
other programs?
Percent of Children in each program who entered:
 At age expectation in all 3 outcome areas
 At age expectation in 2 outcome areas
 At age expectation in 1 outcome areas
 At age expectation in 0 outcome areas
100%
90%
80%
70%
60%
%_3
50%
%_2
40%
%_1
%_0
30%
20%
10%
0%
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10 P11 P12
(238) (1425) (475) (257) (950) (380) (143) (285) (333) (48) (109) (48)
P13
(62)
Are there differences in the programs related to the entry scores on the outcomes?
The Early Childhood Outcomes Center
September 2011
Percent of children with outcomes data by primary language
1
0.9
0.8
0.7
0.6
0.5
Other
0.4
Spanish
English
0.3
0.2
0.1
0
Are there differences in the programs related to the primary language of the children
they serve?
Which programs appear to serve a different population of children than is typical in the
state?
The Early Childhood Outcomes Center
September 2011
Hypothesis checking- Is primary language related to child outcomes?
Progress Categories by Child’s Primary Language
60%
50%
40%
OC-B_% a
OC-B_% b
30%
OC-B_% c
OC-B_% d
20%
OC-B_% e
10%
0%
State (4751
English (3426)
Spanish (957)
Other (368)
Do there seem to be differences in child outcomes for children who do not have English
as a primary language?
The Early Childhood Outcomes Center
September 2011
Overall Discussion Questions
Inferences:
Are there programs in North Euphoria that seem to have trustworthy data?
What Programs in North Euphoria need assistance with improving the quality of their
data?
Are there programs in North Euphoria that seem to have quality services leading to
good child outcomes?
Are there programs that need assistance in improving their services?
Actions:
What improvement activities would you plan for North Euphoria related to data quality?
What improvement activities would you plan for North Euphoria to improve the quality of
services?
The Early Childhood Outcomes Center
September 2011
Notes on Hypotheses
Program #
Unusual pattern
The Early Childhood Outcomes Center
Possible reason(s) for pattern
September 2011