How many students are paying attention?
(Senegal)
The revelation of the secret and mysterious
causality chains
Helen Abadzi
Senior Education Specialist
FTI Secretariat
c/o World Bank
May 12, 2010
Habadzi@worldbank.org
APEIE workshop
Data collection from the “black box” ?? !!
Inside inputs get converted into information
Project inputs
Buildings
Textbooks
Salaries
Training
School grants
School plans
Community
management
Impact evidence
classroom
activities
Information
processing
Enrollments
Dropout
Repetition
Skills
Learning outcomes
Useful for you personally
As well as for school surveys
How well was a treatment implemented?
Find out
• Physical inputs
– Building, maintenance, furniture, consumables
– Radios, lab equipment, maps?
– Textbooks
• Training inputs
– For teachers, principals, administrators
•
•
•
•
Instructional delivery to students
Management committees
School budgets
School planning
Building condition may affect implementation
of other components
Poor mixture of cement
and sand
Structural problems
Lack of maintenance
Structural defects in a school
Computers and labs
How often do students use them?
With what software?
Teacher and management training courses
Ask:
 Did teachers and principals receive the training
that had been financed?
 How many days?
 What did they do?
 Lectures, activities?
 What can they remember from it?
 Please ask for three items they recall
 Do they use the training information in classroom
or in school?
 Ask: How? Where?
School feeding
Nutritional supplements
• Do students in fact get fed
what the government thinks?
• Do the schools get enough?
– What kinds?
• How often are they
delivered?
• When were they last
delivered?
• Take a look at them, see
expiration date
Management improvement components
Find out:
 School grants & budgets
 What did the schools buy? Did not buy?
 In what ways (if any) did the grant purchases increase the amount of
learning?
 Community management
 How often do they meet? What % come?
 What is the added value of their presence?

Buildings, money, monitoring?
 Do they just approve what principal decides?
 School planning
 What did school decide was missing?
 What did they do about it?
 Do you see deficiencies that the school staff did not find?
Do school management committees
perform the expected functions?
Where are the textbooks?
Look for them
 Nowhere?
 In the students’ hands?
 In the classroom?
 In the library?
 How many students vs. textbooks in a class?
If textbooks are insufficient,
they stay on the shelves
Class time spent in blackboard
copying
How much actual teaching do you see?
Scheduling visits to schools
 Sampling schools – important issues
 Omitting certain schools from surveys alters the
sample
 If more than 5% of the schools are changed
 Visit unannounced if possible !
 Perhaps not reveal to local authorities when schools will
be visited
Classroom observation experience
for you
 Take a look inside classes if possible without being
seen (e.g. from window)
 What were they doing before you went in?
 Sit down for 5-10 minutes
 Take video or audio evidence
 Teachers may try to teach better in front of you but
may not know how!
 After a while everyone may forget about you.
Which variables are worth measuring from
classrooms events?
Use as framework the rules of how people
learn
Look at:
Is time spent for instruction?
 Are all students engaged in learning tasks?
 Does teacher know the subject?
 Is teacher using methods that help students
remember?
How many interact with the teacher
Look around the class and think:
Can the students learn the expected subjects in this class?
 How many of the enrolled students present?
 How many interact with the teacher, participate?
 How is the class time spent?
 Activities carried out?
 Are students kept busy all the time?
 Are students processing information or just waiting?
Problems identified in many classrooms
 Instruction may not be going on
 They may be unable to see the blackboard
 May spend much time copying
 May repeat without understanding
 They may receive no feedback
 Be unoccupied most of the time
 Grade 1-2 textbooks may not really teach the poor how
to read
How much of the time that governments pay
for is used for instruction?
Class: When financing is converted to
information
In a class hour, students must retain as much info
as possible

and recall it effortlessly when needed
Time use has been measured many times

Stallings classroom snapshot
Only a small fraction of the time that a government budgets is
converted into information
Class time as allotted by a government (e.g., 200 days, 1000 teaching hours)
Remaining after school closures (strikes, weather, teacher training, extra holidays)
Remaining after teacher absenteeism and tardiness
Remaining after student absenteeism
Class time devoted to
any learning task
Learning
time
relevant to
curriculum
Teacher off task (India)
Legal teacher absenteeism:
Teacher’s book from rural Honduras
How many days did this class study?
Students unoccupied, waiting for the bell to ring
(most of them illiterate)
Brazil – group work?
Brazil – group work?
Teacher absenteeism and time use
in Maputo: Schools constantly in recess
To remember and use information, students
must:
 Receive it
 Practice it, contemplate it
 Have prior knowledge on which to fit it
 Read fluently to learn from books
 Know the official language well!
 do math fluently to solve complex problems
High-quality schools:
Offer activities that create complex cognitive networks
 Students recite +
 read long texts+
 manipulate +
 collect real-world samples +
 answer questions connecting various items +
 derive new conclusions from data +
 solve problems +
 practice for fluency +
 generalize into various circumstances
 Spend time in “active learning”
Mexico – efforts to reclassify
Signs that the students will probably
remember the information
 Group or even individual work with students
concentrated
 Activities of students reading, discussing
 not merely verbally repeating
 Writing material that is not simply copying
 Teacher uses aids like flash cards
 Teacher monitors individual student work
 Gives feedback
 Teacher going towards the back,
 addressing individual students who did not volunteer to
answer a question
Good: Group work, engaged students
but time must be kept strictly
•
Poor-quality schools:
may teach items in series
with few connections
Students may just recite or listen…
The heroes of the revolution are…
The books of the bible are….
2x2=4, 2x3=6, 2x4=8, 2x5=10….
The principles of constitutional law
are…
How many are paying attention?
One student recites, rest unoccupied
Low-quality schools
High quality schools
Recife, Brazil:
seatwork – teacher not monitoring
Determine whether students will learn
Check:
How many of the enrolled students are
present?
 We all hear about large classes
 If you go towards the end of the school year will you find
them?
 Students may enroll but attend rarely, particularly
when they are illiterate or do not know the
material.
 Is there space in a classroom for everyone enrolled?
 There may only be enough space for the regular students
 What excuses do you get?
Students falling behind are often absent
• Bank documents say that classes are large…
• 19 of 45 students in grade 4, outside Nampula,
Mozambique
Bangladesh – student absenteeism
Look at students’ notebooks
• How much is written?
Too much:
no textbooks
Just 2-3
pages?
No classes or
frequent
absences
Check for “hidden dropout”:
How many of the students are actually involved in a
class?
 Ask the teacher: is s/he up to date with the curriculum
that must be taught?
 If so, observe how many students actually follow
teachers’ instructions, answer questions, seem
involved
 The teacher may say that s/he is up to date, but may be
only teaching 3-4 students
 Rest may be illiterate and uninvolved
“Hidden dropout?”:
Teachers in Nepal interacting
mainly with the front of the class
Do teachers know sufficient
material to teach?
 See material on the blackboard
 Does it make sense?
 Example:
 Students who don’t know letter values cannot learn them
from others’ fast reading
 Rural Mozambique
rura
What impresses you most about this scene?
Do these students
discriminate among letters
of the fuzzy blackboard
from this distance?
Lack of textbooks translates into loss of
time at all levels
• Due to a lack of knowledge and materials,
teachers do very few activities
– But they can follow a textbook
• Teacher boredom:
How many of us would spend 20 years in
blackboard transcription?
Dictation in Burkina Faso
It takes 30 seconds to read and 7+ minutes to dictate this
phrase:
“Les industries Francaises ont connu un développement
considerable mais rencontrent des difficultés dans les
ressources.. »
10tth grade math– private school
Reading often not taught specifically
Methods are often very inefficient
Informal way to find out how many
are literate
 Take book, go 3-4 pages ahead
 Ask a student to read
 Mark the start of 1 minute by your watch,
 start a paragraph, mark end in one minute
 Take photo or count words on the spot
 Ask 2-3 simple questions about what they read
 What is their approximate reading speed?
Can students read their notes to you?
Illiterate students writing “art”
Short-term memory
Crucial for reading comprehension
About 7 items
4 pictures
12 seconds at most
Long-term memory
35 words per minute
Quelle est la langue d’instruction?
Les langues Africaines ont orthographie régulière
Peuvent être automatisées dans quelques mois
La fluidité transfert aux autres langues qui ont la même écriture
Grade 1 textbooks – critical for
subsequent years
How should efficient textbooks instruct in reading?
Burkina Faso grade 1 book
Are there dictionaries in local
languages?
Activity at entry in class
Brazil 2002
State
Ceará
Rio Grande
de Norte
Alagoas
6
40%
1
7%
6
33%
3
17%
2
22%
3
33%
Reading lesson in
class
No activity
2
13%
1
6%
1
11%
1
2%
1
7%
5 (2 waiting for
bell)
28%
1
11%
Teacher busy with
2-3 students only
Teacher teaching
whole class
Group practice
1
7%
8
17% (some
teachers absent)
4
8%
Copying
Text production
Goias
São Paulo
4
28%
8 (2 secondary)
17%
2
4%
1
11%
2
13%
1
11%
6
43%
10
22%
1
7%
2
11%
3
21%
4
8%
Art and Play
1(scheduled
activity)
7%
1 (play with letters)
6%
1 (play with
letters)
11%
No. classes
15
18
9
14
45
8
6
7
5
8
34%
45%
48%
92%
40%
No schools
Apparently
instructional
activities (some
suboptimal)
1 (drawing) 8 (during subject
7%
matter)
17%
4. Read fluently
with appropriate
expression
3. Read about a
word per sec.,
some stumbling
1. Could hardly
read letters,
words, or not at
all
2.Barely decoded,
flat voice, no
comprehension
What was the class doing at the moment of mission entry?
(How likely is this activity to engage all students in the task?)
Teacher estimate
of number or %
Approx % of
students off task
Other teaching
activity: art, play
singing, exam,
multiples
Teacher chatting
or disciplining
students
No clear
instructional
activity
Class copying
from black
Board
Students working
in groups
Students doing
seatwork alone
1-2 students at
blackboard
Student & teacher
attendance
Individual or
choral answers
no. responding
Teacher teaching
entire class
No. of textbooks
in sight
No. students
present
Grade
No. students
enrolled
School
Teacher present?
Sample form to record
instructional data from school
visits
How many sampled students can read?
How well?
Comments
How to measure and quantify classroom
variables?
 Need for summative scales, “low-inference”
instruments
 Many observation instruments exist
 On the minute, Flanders, Virgilio, CLASS
 Classroom observation instruments
 Difficult
 complex
 Imperfect
Stallings modified classroom
snapshot
 Low inference
 Easier to communicate to less educated staff
 For lower-level grades, deprived environments

Where time gets wasted
 Stallings 5-minute interaction – to be updated?
Snapshot
10
I: Individual T: Teacher
L: Large Gp E: Entire
S: Small Gp
Gp
Number of Adults Present
Number of Students Present
MATERIALS
None
Books
Notebook
Chalk
board
Manipul
ative
Visual/C
omputati
onal Aids
Cooperative
T
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
I
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
T
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
I
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
T
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
I
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
T
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
I
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
T
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
I
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
T
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
I
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
T
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
I
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
T
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
I
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
T
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
I
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
1SLE
T
1SLE
I
1SLE
Student Uninvolved
I
1SLE
Being Disciplined
T
1SLE
Classroom Management
T
1SLE
I
1SLE
ACTIVITY
Reading Aloud
Instruction/Demonstration/Lecture
Discussion
Practice/Drill
Kinesthetic/Projects
Written Assignments/Seatwork
Copying
Assignments
Other
Social Interaction
Adult Social Interaction/Adult Uninvolved
T
Adult Management
T
Teacher out of Room
T
Training and future use
 Classroom snapshot takes 5 days training
 Trainees learn how to spot time and wastage
Instructional time loss as a
monitoring indicator
 What % of the time bought by governments for
students is used in learning activities?




Ghana 39%
Pernambuco (Brazil) 63%
Dominican Republic 65%
Tunisia 79%
 What % of the curriculum covered each year?
 What % of objectives do students master?
 What test scores should we expect?
 What % of students in a class interacted with a teacher?
Instructional Time Use in the participating countries
School Year (Days)
School Closures (Days)
Pernambuco
(Brazil)
200
4.79
Ghana
Morocco
Tunisia
197
3.17
204
1.38
190
5.15
Days after Closures
195.21
193.83
202.62
184.85
Teacher Absence (Days)
Teacher Delays
Early Class Dismissals
12.76
5.50
2.31
43.01
39.75
2.43
13.36
6.94
6.68
11.55
1.27
1.22
School Year (Days left after losses)
% Year available for teaching
174.65
87.3%
108.6
55.1%
175.6
86.1%
170.8
89.9%
Engagement Rate in Interactive
or Passive Classroom Tasks
72.1%
70.2%
82.6%
86.7%
School Days Devoted to Learning
School Year % Spent Engaged
in Learning Tasks
125.9
76.3
145.1
148.1
63.0%
38.7%
71.1%
77.9%
Student off Task Rate
19.3%
21.1%
9.2%
9.9%
7.82
5.64
9.04
10.61
4.30
5.187
3.35
2.63
Student Absence (Days)
Student Delays (Number of Times)
Percentage of Time Spent in Instructional Tasks
Instructional Strategy
% of time
U.S Classroom
Criteria
Brazil
Ghana
Morocco
Tunisia
Interactive instruction
50% or more
52.4
59.9
62.8
61.2
6.7
8.7
15.7
15.3
32.8
19.9
26.7
27.9
Discussion
6.3
24.1
6.6
6.2
Practice Drill
1.4
6.5
12.3
11.3
19.6
10.3
19.9
25.6
16.3
7.4
14.8
22.9
3.0
2.9
5.0
2.7
72.1
70.2
82.6
86.7
15% or less
27.9
28.0
17.8
13.3
6% or less
19.3
21.1
9.2
9.9
Oral Reading
Teaching, Explanation
Passive instruction
35% or less
Seatwork
Copy
Total Instructional Time
Organizing/Management
Student Off Task Rate
Now how black is the box?
Thank you for your attention
73