Understanding ‘Learning’ in Pakistan:
Building a Comprehensive Picture of
School Functioning in Pakistan
Monazza Aslam, Centre for the Study of
African Economies (CSAE), Dept. Of
Economics, University of Oxford, UK and
Idara-Taleem-o-Aagahi, Lahore.
Education is Key...
• “Education is central to development and a key to
attaining the Millennium Development Goals. It is
one of the most powerful instruments for
reducing poverty and inequality….” (The World
Bank);
• Access to education is high on the development
agenda;
• Desire to achieve MDGs has galvanised
policymakers worldwide into action.
• In many countries progress in school enrolment is
phenomenal.
‘Learning’ is critical!
• It is now a well known fact – it is not years of schooling
that matter to important outcomes but what is actually
learnt that matters.
• Hanushek and WoBmann (2007) show that differences
in learning achievements matter more in explaining
cross-country differences in productivity growth than
differences in the average number of years of schooling
or in enrollment rates.
• An effective strategy should focus not solely on sending
more children to school but also on enhancing the
quality of schooling.
• Hanushek and WoBmann (2007) show using PISA
surveys that disparities in secondary education
between developing countries and OECD
countries are even larger when one considers not
only access but also learning achievements.
• Things are not much better at the primary level.
In recent surveys in Ghana and Zambia, fewer
than 60 percent of young women who complete
six years of primary school could read a sentence
in their own language.
‘Learning’ is critical for:
• growth and for pro-poor growth;
•
•
•
•
•
•
•
For poverty reduction;
For reduced inequality;
Fertility, population control;
For access to more lucrative occupations;
Higher earnings during the lifetime;
Women’s empowerment;
Children’s health outcomes and hence future life
outcomes.
Schooling and ‘Learning’ in
Pakistan
• While Pakistan has made substantial headway recently in
achieving some dimensions of the ‘Education for All’
Millennium Development Goal (MDG2), it still has a long way
to go.
• although the net primary enrolment rate increased from 45%
in 2002 to 54% in 2006 (World Bank), it is pitiably short of
universality.
• More importantly, the MDG doesn’t focus enough on
‘learning’ outcomes.
But...
• Enrolment is the very first rung of the ‘access’ ladder.
• Meaningful access requires – in addition to school enrolment
–good attendance rates, timely progression through grades,
and mastery of basic cognitive skills.
• Not all enrolled children attend school regularly
• not all who attend regularly complete primary schooling;
• not all who complete primary schooling acquire adequate
learning to be literate and numerate.
• Stable and regular attendance of both students and teachers
is necessary for learning.
What ‘needs’ doing according to
common knowledge
• Improve curriculum;
• Improve teaching;
• Improve facilities for ‘learning’;
But research on most aspects is limited.
What do we know so far?
(will talk about work I am associated with)
PREVIOUS RESEARCH IN PAKISTAN SHOWS:
1. We know that students in private schools may achieve more and that girls may be
worse off in terms of access to private schools;
2. We also know potentially how student A may be targeted through improving weak
teaching;
3. ASER (2008) – builds a picture of basic learning and tells us that 65% 5-16 year olds
in 11 districts in Pakistan cannot read standard 2 story level text and more than 70
% cannot do two digit division (again standard 2 curriculum).
4. We also know that there are striking gender inequalities in occupational choice and
in earnings;
5. Finally, we also know that the labour market rewards women’s ‘skills’ substantially.
If today's children are to become tomorrows earning adults, we need to know how
they are learning today to ensure they are able to achieve their full potential
tomorrow.
The private schooling and learning
story
• Aslam (2009), ‘The Relative Effectiveness of Government and Private
Schools in Pakistan: Are Girls Worse Off?”, Education Economics, Vol 17 (3),
pp. 329-353.
• If you look at raw PIHS data, girls are found more likely to be enrolled in
private schools.
• But when we condition on household observables/unobs. Picture
changes! BOYS are more likely to be enrolled in private schools.
• A major channel of lower educational expenditure on girls schooling than
boys is because boys are more likely to be sent to private school;
• Although average cost of private schools low, relatively more expensive
than government school.
• Are girls condemned to poor ‘quality’ schooling?
Achievement Data used to answer the question: Are
girls condemned to poor quality schooling in Pakistan?
• Data – one district (Lahore);
• School-based sample survey of 65 schools (40
pvt, 25 govt);
• Data collected in 2002-2003;
• Detailed info on 1800+ students, 300+
teachers – 8th grade students;
• Tests administered – literacy, mathematics,
ravens (discuss);
Raw and standardised Achievement, Reading and
Maths scores by school type
• Estimates based on School survey conducted by author in 65
schools (40 private and 25 government) in Lahore district in
2002-2003.
• A random student with average characteristics would score
9.63 points in maths if in private school compared to 8.10
points in government school (19 % higher)
• Predicted reading score of student in private school 8 %
higher;
Hence girls lose out in terms of the quantity and
quality of education acquired! Achievement
data helps us identify a major inequality.
What is it about teachers that
matters for student achievement?
• Aslam and Kingdon (2008, RECOUP Working
Paper 19, under review). ‘What can teachers
do to raise pupil achievement?’
• This study probes the ‘black box’ of teaching
to explore the teacher characteristics that
impact student learning;
• How important are the teaching practices
adopted by teachers in determining student
achievement?
What the study does…
• This study takes the direct approach linking teacher characteristics
to student outcomes in an achievement production function, but
with two innovations.
• Firstly, we use a methodology that overcomes the biases associated
with the non-random matching of students to schools and, within
schools, to teachers.
• Secondly, we also test the importance of classroom practices and
teaching techniques rather than confining attention only to
teachers’ resumè characteristics i.e. qualifications, experience and
training.
Teacher characteristics data
• Standard teacher chars such as age, gender, certification,
training, tenure, experience etc.
• ‘Teaching methods and techniques’ information gathered by
asking each individual teacher to answer a series of questions
such as record minutes/week spent on average in giving
surprise tests/quizzes, maintaining discipline and order,
writing material on the black board etc.
• Teachers also given series of questions on use of lesson plans,
whether they read out loud from books, how well they
engaged students when teaching etc.
Methodology
• Achievement production function – ‘direct approach’
• Estimate:
Aik = α + βXik + δSk + μi + ηk
(1)
•
•
•
•
•
Where i = individual and k = school
X = individual and family characteristics
S = school and teacher characteristics
μi = individual-level unobservables
ηk = school-level unobservables
Summary of results
• Resume’ – i.e. teacher qualification, training, etc. - characteristics of
teachers don’t matter to student learning – highly controversial
claim but supported by studies from India, USA etc.;
• Girls benefit from being taught by female teachers; Why? 1)
Preferences hypothesis...same-sex teachers prefer same-sex
students , 2) stereotypes influence teachers evaluations; We gave
tests and marked them so can rule out the ‘evaluation’ stemming
from these 2 but can’t rule out possibility that knowledge
transmission not affected by either route.
• Teaching ‘process’ – lesson-planning, quizzing and involving them in
classroom discussions clearly matters to achievement.
G and P school differences
• We know from recent literature that private schools more ‘effective’
than govt. schools (World Bank researchers, Alderman’s work etc.
on primary and my work on Middle school students).
• Why more effective?
- Private school student body superior quality?
- Infrastructure superior?
- Teacher quality superior?
- In terms of apparent observed charachtersitics, G school teachers
more certified, experienced, better paid.
- Know nothing about ‘unobserved’ motivation etc. – indeed contract
stuctures in G schools generate perverse incentives.
- We know nothing about ‘process’ differences or even how the
production functions differ across school types.
Conclusions of ‘Teachers’ study
• Delved into the ‘black box’ representing teaching;
• Identification strategy was unique;
• Importance of class room practices;
• Separate estimates for govt. and private schools;
• Teacher unobservables still remain as source of endogeneity
undermining confidence in attributing causality to observed teacher
variables.
Conclusions continued…
• Standard teacher resume’ characteristics have no bearing on
student’s standardised mark;
• Teacher pay schedules are ‘inefficient’;
• Girls in 8th grade benefit being taught by female teachers;
• Teaching classroom practices (lesson planning, involving
students by asking questions, quizzing etc.) substantially
benefit student learning.
• Private schools teachers more ‘efficient’ and use the more
innovative teaching methods in imparting learning.
What does ASER (2008) tell us?
• Annual Status of Education Report (ASER).
• The purpose of the ASER rapid assessment survey and its planned annual
rounds in rural areas is twofold:


To get reliable estimates of the status of children’s schooling and basic
learning (reading and arithmetic) up to grade II at the district level
To measure change in these basic learning and school statistics over
time- annually
• In ASER Pakistan 2008 , 16737 children and 6520 households were
surveyed in the rural areas in 11 districts
How ASER is conducted?
• ASER is a citizens-led initiative conducted in a
campaign mode. This means that ASER is not
conducted by any single organization. Instead, ASER is
done in collaboration with a variety of civil society
organizations,
universities,
research
institutions
government organizations and citizens’ groups.
Collaborators voluntary supporters, teachers and
students devote their time and resources to conduct
ASER in their region/districts.
• Pratham and ASER India, provide technical support at
various stages of the survey to the ASER Pakistan.
Scale & Scope of Survey
Coverage : In all five provinces i.e. in Sindh, Balochistan, Punjab,
Pakhton Khawa, Gilgit Baltistan, FATA and AJK.
Only rural areas will be sampled.
Phase I : Year I 2010 – 30 districts across Pakistan
Phase II: Year II 2011 – 70 districts across Pakistan
Phase III : Years III, IV , V all districts across Pakistan (138 districts)
Sample: 600 households per district. Two-stage sample, stratified in the
first stage. Obtained by selecting 30 villages per district & 20
households per village.
Villages will be selected randomly using the village directory of
the latest Census. The Probability Proportional to Size
Sampling (PPS) technique will be adopted as an appropriate
one when the sampling units are of different sizes.
Focus on nuances between public and private . In each village
profile of 1 govt. and 1 private school will be collected .
ASER Pakistan Assessment Tools
ASER 2008 in Pakistan
ASER Pakistan 2008 only completed in 11 districts of 2
provinces: Punjab, Sindh and the Islamabad Capital Territory.
Unfortunately, ASER Pakistan 2008 could not be undertaken in
Baluchistan and NWFP due to the Oct. 2008 earthquake/floods
& rapidly deteriorating law and order situation /conflict ..local
collaborators got engaged in multiple emergency responses
Sr #
Province
District
1
Lahore
2
Sheikhupura
3
Multan
4
Jhang
5
Punjab
Rawalpindi
6
Rahim Yar Khan
7
Mianwali
8
Faisalabad
9
Sub total
8
ICT
Islamabad
Sub total
1
10
Khair Pur
11
Dadu
Sub total
2
TOTAL
11
Districts’ Summary ASER 2008
Sr #
Province
District
Punjab
Lahore
Sheikhupura
Multan
Jhang
Rawalpindi
6
7
8
1
2
3
4
5
9
Sub total
ICT
Sub total
10
11
Sindh
Sub total
TOTAL
Villages
HH
Children
Surveyed
Schools
30
30
30
30
30
600
600
600
600
600
1,367
1,199
1,430
1,823
1,419
30
30
30
30
30
Rahim Yar Khan
30
600
1,565
30
Mianwali
Faisalabad
8
Islamabad
1
Khair Pur
Dadu
2
11
30
30
240
30
30
600
600
4,800
600
600
28
560
28
560
56
1120
326
6,520
1,631
1,631
12,065
1,232
1,232
1,539
1,888
3,427
16,737
30
30
240
0
0
28
19
47
287
In ASER Pakistan 2008 , 16737 children and 6520 households were surveyed
in the rural areas during the survey in 11 districts
Enrolment is still a problem –
children aged 5-16
-More girls are out of school – no surprise here;
-Enrolment is the very first rung of the access ladder – we
stumble on the very first one.
Enrolment by School Type
•Apparently a larger
proportion of girls enrolled
in private schools (46%
versus 41% boys).
•This finding is NOT
different from what the
PIHS data tells you when
you do not condition on
household observables.
•If we were to condition on
observables etc., I suspect
ASER 2008 finding would
be identical to Aslam
(2009) – girls lose out in
the quality of schooling
available to them!
What about ‘meaningful access’?
• Good Attendance rates?
• Timely progression through grade?
• Mastery of basic cognitive skills?
Attendance
School with
Std 1-5 Std 1-8 Others
Children attendance 79%
72%
72%
Teachers
75%
82%
76%
Attendance
PTR
49
39
29
•
•
Teachers Attendance is over all 78%
Primary level 75%;
Elementary : 82%
Children’s Attendance overall : 74%
Primary level 79%;
Elementary level 72%
Average
74%
78%
39%
Think about it!
• 79% children attendance in grades 1-5 means that in an
average class of 49 children, 10 children were not attending
school on the day visited.
• If there are 15 teachers in a primary school, 4 out of the 15
teachers are not available on the day visited.
• Aslam (2003), ‘Determinants of Student Achievement in
Government and Private Schools in Pakistan’, The Pakistan
Development Review, 42:4, Part II (Winter), pages 841-876:
finds that student absenteeism has a detrimental affect on
student learning.
School Facilities: Provision & Use
Schools with
Std 1-5 Std 1-8 Others
Number of Schools Visited
220
44
23
Average No. of Rooms Available for Classes
Average No. of Rooms Used for Classes
Water Facility (working) %
Water Facility (not working / not Available) %
Toilet Facility (Working) %
Toilet Facility (not working/ not available) %
4
3
69%
31%
58%
42%
6
5
77%
23%
73%
27%
9
7
74%
26%
71%
29%
Primary Schools
• 31% schools at primary level are without water facility,
• 42% schools are without toilet facility
Elementary Schools
• 23% schools at Elementary level are without water facility,
• 27% schools are without toilet facility
What about mastery of basic
cognitive skills?
• Standard 2/3 instruments designed and all
children between 5-16 years tested.
• Children found to be performing way below
curricular standard.
Standard 2 Reading ability of 3-16 year olds
Of the 16, 737 children tested, only about 30% are able
to read a story with fluency in Urdu or local language.
Standard 3 Mathematics ability of 3-16 year olds
Of the more than 16,000 children tested, only about a
quarter can do double digit division.
Learning Levels (Reading)
Reading (%)
Ages
Beginning /
Nothing
Letter
Word
Para
05-09
25
24
23
15
(Class 1-5)
10-12
7
9
15
21
(Class 6-7)
13-14
7
3
8
15
(Class 8-9)
15-16
8
2
6
11
(Class 10-11)
05-16
16
15
17
17
Learning Ability of the Age Group 5-9 Years:
• 27% of the children read para or level - I text
• 12% of the children in this age group read story text level-II
Story
Total
12
100
49
100
66
100
73
100
35
100
Learning Ability of the Age Group 10-12 Years:
• 70% of the children in this age group can read para-or level 1 text
• 49% of the children in this age group can read story or level II
Learning Ability of the Age Group 15-16 Years:
• 84% of the children read para or level - I text
• 73% of the children in this age group read story text level-II
36
Learning Levels (Arithmetic)
Arithmetic (%)
Age Group
5-09
(Class 1-5)
10-12
(Class 6-7)
13-14
(Class 8-9)
15-16
(Class 1011)
05-16
Beginning /
Nothing
Numbers
1-9
Numbers
11-99
Subtraction
Division
Total
26
25
29
14
6
100
13
8
21
26
32
100
8
3
12
18
58
100
9
3
9
15
64
100
18
14
22
19
27
100
Learning Ability of the Age Group 05-09 Years:
• 20% do subtraction or Level-I(arithmetic)
• 6% of the children in this age group solve division (level-II)
Learning Ability of the Age Group 10-12 Years:
• 58% do subtraction or Level-I(arithmetic)
• 26% of the children in this age group solve division (level-II)
Learning Ability of the Age Group 15-16 Years:
• 79% do subtraction or Level-I(arithmetic)
• 64% of the children in this age group solve division (level-II)
37
Learning - National
Age Group
Who Cannot Read (%)
Who cannot do Arithmetic (%)
Paragraph
Story
Subtraction
Division
5-9
72
88
80
94
10-12
30
51
42
68
13-14
19
34
23
42
15-16
16
27
21
36
5-16
49
65
54
73
38
Going back to the ‘WHY’ in
Pakistan
• Research further shows that even the most basic
literacy and numeracy skills among adults in Pakistan:
1. Improve women’s access to more lucrative
occupations;
2. Increase earnings for both men and women once in
the labour market;
3. Reduce fertility;
4. Improve mortality outcomes;
5. Increase agricultural productivity.
6. Etc.
Measure ‘basic’ skills
• Focus on ‘Public/Private’ /other school
types differences;
• Rural/Urban divides;
• Gender differences;
• Socio-economic groups;
• School/village location;
• Individual level data – allows us to calculate
achievement differences b/w different
groups;
We also need to go BEYOND existing
efforts
We come back full circle:
• Many children still not enrolled in school;
• Not all enrolled children attend school regularly;
• Not all who attend regularly complete primary schooling
• finally not all who complete primary schooling acquire
adequate learning to be literate and numerate.
• Stable and regular attendance of both students and teachers
is necessary for learning (among other factors)
Pakistan needs more research
• In a new project I am proposing to carry out (with ITA
collaboration) two (linked) unique surveys to enable a better
understanding of the constraints on (a) children’s schooling
participation and (b) student learning, in Pakistan.
• Apart from testing children at two points in time in order to
measure gain in achievement, the survey will measure factors
that might be behind poor child learning but which have
hardly received research or policy attention thus far, e.g.:
• (i) teacher effort (attendance rate, time-on-task);
• (ii) teacher competence to teach material in the primary texts
(teacher’s knowledge, ability to explain and ability to spot the
mistakes that children make); and
• (iii) aspects of school functioning such as existence of/
adherence to a school/class time-table; stability/flux in multigrade class configurations in the school over the year;
schooling resources and teacher characteristics; and the
extent of community involvement in the school, inter alia.
• The proposed school survey will be unique in Pakistan
because it will visit each sample school four times in the
school year. This will permit observing the dynamic
functioning of schools, in contrast to a snapshot picture.
Then what?
• Planned interventions can be undertaken once
we know the areas that need targeting;
• Informed national policy decisions;
• Align classroom curricula with actual
instruction in class;
• Motivate the masses – increase public
awareness, mobilize communities, parents,
educators, policymakers into ACTION!
Importance of having this
disaggregated data
• Identify the schools/students/areas/groups in
most need of educational policy targeting;
• Guided decisions on how to allocate
resources/mobilize resources;
• How to tailor curricula and teaching to specific
needs of different groups;