Uncovering indicators of effective
school management in South Africa
using the National School
Effectiveness Study
Stephen Taylor
Department of Economics, Stellenbosch University
PSPPD Project – April 2011
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
Motivation (the problem)
• Low quality education a poverty trap to many
children in historically disadvantaged schools
• Question: Poverty itself or the characteristics of
schools in poor communities?
• SACMEQ II and III:
Poor South African children performing worse than equally
poor children in other African countries
• This despite substantial resource shifts to correct for
apartheid inequalities
• Historically disadvantaged schools have been largely
unresponsive to additional resources
• Consequence: Perpetuation of a “2 systems” system
• How does the literature explain this?
2
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
Motivation (the literature)
• Resources do not necessarily make a difference:
• the ability of schools to convert resources into outcomes is
the crucial factor (Van der Berg, 2008)
• Socio-economic status (SES) has a dominant impact on
the distribution of achievement
• Studies based on large sample surveys have typically
struggled to identify specific aspects of effective
management and teaching practice that explain
performance.
• Crouch and Mabogoane (1998): 50% of variance explained by
“management efficiency”
• Van der Berg and Burger (2002): 2/3 variance explained by SES,
racial composition & school resources; remainder probably due
to unobserved “management efficiency”.
• Largely due to data limitations most large surveys
3
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
Data
• National School Effectiveness Study (NSES)
• JET Education Services & RNE
• Literacy and numeracy testing:
• Grade 3 (2007)
• Grade 4 (2008)
• Grade 5 (2009)
same individuals
• Principal questionnaires (2007, 2008, 2009)
• Teacher instruments (2008, 2009)
• Teacher comprehension and maths test
• Extensive review of learner workbooks
• Greater potential to uncover indicators of
effective management an teaching
4
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
Results – overall scores
5
Literacy
Numeracy
2007 (grade 3)
20.15
29.38
2008 (grade 4)
29.59
35.50
2009 (grade 5)
37.73
47.04
Gain 2007 - 2008
9.43
6.12
Gain 2008 - 2009
8.14
11.54
2-year gain
17.57
17.66
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
Numeracy 2007
6
Numeracy 2009
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
WESTERN CAPE
NORTHERN CAPE
NORTH WEST
MPUMALANGA
LIMPOPO
KWAZULU-NATAL
FREE STATE
EASTERN CAPE
0
20
40
60
80
100
Results: Numeracy scores by province
0
.01
.02
.03
.04
Results: Literacy achievement by SES
0
20
40
60
Literacy score 2009 (grade 5)
Quintile 1
Quintile 3
Quintile 5
7
80
Quintile 2
Quintile 4
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
100
10
20
30
40
50
60
Results: Literacy achievement by SES
0
1
2
SES (min = 0, std dev = 1)
3
Literacy 2007 (grade 3)
Literacy 2008 (grade 4)
Literacy 2009 (grade 5)
8
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
4
0
.01
.02
.03
.04
.05
Results: Numeracy achievement by
ex-department
0
20
40
60
Numeracy score (%)
Numeracy grade 3 (DET)
Numeracy grade 4 (DET)
Numeracy grade 5 (DET)
9
80
100
Numeracy grade 3 (HOA)
Numeracy grade 4 (HOA)
Numeracy grade 5 (HOA)
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
.015
.01
0
.005
Density
.02
.025
Results: Numeracy achievement of African
language students by ex-department
0
20
60
40
Numeracy score 2008
Ex-DET/Homelands schools
10
80
Historically white schools
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
100
Results: Indicators of effective
management and teaching
Percentage of students in schools where more than
25 maths topics were covered
Ex-department
DET (B)
HOR (C)
HOD (I)
HOA (W)
Total
11
Percentage > 25 topics
26%
25%
38%
75%
29%
Number of students
6306
849
86
591
7832
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
Results: Indicators of effective
management and teaching
Mean number of literacy exercises found in the
“best” learner’s book
ex-department
DET (B)
HOR (C)
HOD (I)
HOA (W)
Total
12
Mean number of exercises
33.43
62.40
72.44
75.21
39.58
Number of students
6478
837
102
580
7997
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
Results: Extended writing
100
90
88
85
Number of English classes
80
70
70
60
50
40
30
23
19
20
10
0
No exercises 1 or 2 exercises 3 to 9 exercises
with paragraphs with paragraphs with paragraphs
13
More than 10
exercises with
paragraphs
Unspecified
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
Results: Maths teacher knowledge
10 days 75 hours can be written as .... days .... hours
Number of
students
%
Cumulative %
Mean Numeracy
2008
0
210
2.12
2.12
37.27
1
2130
21.52
23.64
33.04
2
2774
28.02
51.66
33.50
3
2168
21.9
73.56
34.14
4
1408
14.22
87.79
34.77
5
1209
12.21
100
46.92
Total
9899
100
100
35.44
Teacher score
14
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
Results: Multivariate analysis
• Are teachers with better subject knowledge located in
more affluent schools?
• And is it this affluence driving the association of student
achievement with teacher knowledge?
• The need for multivariate analysis to disentangle this.
• After accounting for the influence of SES, what school
and teacher characteristics are associated with student
achievement?
• What distinguishes better and worse-performing
schools within poor communities?
15
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
Results: Multivariate analysis
• 4 multivariate regression models estimated in the
education production function tradition:
• OLS regression predicting Literacy achievement in grade
4
• OLS regression predicting Numeracy achievement in
grade 4
• 2 more sophisticated techniques to model gain scores
16
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
Results: Multivariate analysis
Explanatory variables
Student characteristics
Student SES
Male
Young
Old
Household size: large
Read 1 to 3 times a week
Read more than 3 times
Books at home: 1 to 10
Books at home > 10
Home language English
Speak English 1-3 times
Speak English 4+
English on TV 1-3 times
English on TV 4+
School characteristics
Mean School SES
Mean School SES squared
Pupil-teacher ratio
Teacher absenteeism zero
LTSM Inventory good
Problems with students index
Curriculum planned using year schedule
Teacher characteristics
Full year learning programme
Constant
R-squared statistic
N
17
0.39*
-2.48***
-0.40
-2.84***
-1.89***
1.37**
2.39***
0.60
1.17*
8.42***
1.75***
1.86**
0.85*
3.35***
(0.18)
(0.26)
(0.46)
(0.33)
(0.37)
(0.44)
(0.62)
(0.39)
(0.48)
(1.52)
(0.38)
(0.68)
(0.39)
(0.44)
-9.13***
3.35***
-0.18**
1.93*
1.66*
-0.96*
1.46~
(1.77)
(0.45)
(0.07)
(0.81)
(0.80)
(0.43)
(0.81)
1.55~
(0.87)
29.69***
0.4591
10 860
(3.45)
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
Results: Multivariate analysis
• Literacy grade 4 (2008)
• Estimated effects of change in characteristics on the literacy
national average (Original sample mean = 26.57%)
18
Teacher absenteeism zero
LTSM Inventory good
Curriculum planned using year schedule
Full year learning programme
Predicted new mean
27.84
27.36
27.18
27.18
Gain
1.27
0.79
0.61
0.61
Combined effect of improved characteristics
29.85
3.29
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
Results: Multivariate analysis
• Numeracy grade 4 (2008)
• Estimated effects of change in characteristics on the numeracy
national average (Original sample mean = 34.21%)
19
Assessment record keeping
No timetable available
Teacher absenteeism zero
Maths teacher test score: 100%
Maths topics covered: 25 plus
Predicted new mean
35.08
34.45
36.01
36.38
37.20
Gain
0.87
0.24
1.80
2.17
3.00
Combined effect of improved characteristics
42.29
8.08
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
Results: Multivariate analysis:
Modelling the literacy gain scores (Historically black schools only)
Explanatory variables
[A] Pooled gains step 2
[B] 2-year literacy gains
Mean School SES
0.39
0.14~
1.37*
0.27~
2.16*
-2.72
-4.03***
3.03***
3.76***
-4.12***
2.35*
1.03
2.64
3.83*
Facilities index (2008)
(0.35)
(0.08)
Monitoring through class visits
No timetable available (2008)
Principal absent
Teacher punctuality good
More than 2 English mark records
Paragraph writing: none
Literacy exercises: more than 27
-1.67**
0.94~
1.44*
-1.72**
1.34*
(0.65)
(0.53)
(0.64)
(0.57)
(0.55)
Years teaching: 4 to 9
Years teaching: 10 to 19
Years teaching: 20 plus
st
Time dummy (1 year)
0.40
Constant
-5.33***
0.1214
390
R-squared
N
(0.63)
(0.15)
(0.90)
(1.93)
(1.13)
(0.91)
(1.13)
(1.01)
(0.96)
(1.87)
(1.61)
(1.67)
(0.51)
6.10**
0.3976
195
~ p<0.10, * p<0.05, ** p<0.01, *** p<0.001
Standard errors in parentheses
20
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
Conclusions and Policy Implications
• Resource variables were not amongst the most important
factors predicting achievement
• Several indicators of effective school management and
teacher practice that are associated with student
achievement have been identified
• even within the large historically disadvantaged section of the
school system.
• This is an advance on earlier analyses
• An organised learning environment:
• curriculum planning for the full year, a functional timetable, goodquality inventories for LTSM, low teacher absenteeism and up-todate assessment records
• Extensive coverage of curriculum and exercises
21
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union
Conclusions and Policy Implications
• Policies should empower teachers to cover
curriculum and administer exercises:
• At the top: clearly communicated curriculum
requirements
• Also, textbooks and workbooks that make worked
examples easier for both teachers and students to
implement.
• Command and control measures to enforce
adherence to best practices?
• Probably not…
• Explore ways to attract, train and support better
principals, and to replace those at the head of
dysfunctional schools.
22
Programme to Support Pro-Poor Policy Development
A partnership between the Presidency, Republic of South Africa and the European Union