UNIVERSITY OF JYVÄSKYLÄ
Characteristics of low and top performers in
reading and mathematics. Exploratory
analysis of 4th grade PIRLS and TIMSS data
in Nordic countries
Sari Sulkunen, Department of Languages
Kari Nissinen, Finnish Institute for Educational
Research
Pekka Kupari, Finnish Institute for Educational
Research
University of Jyväskylä
UNIVERSITY OF JYVÄSKYLÄ
Introduction
 Consider three student groups:
– Top performers
– Low performers
– ’In-betweens’, intermediate performers
 Vast majority of students are neither low or top
performers
– ’In-between’, intermediate pupils
– Define these pupils as the reference group of analyses
UNIVERSITY OF JYVÄSKYLÄ
Introduction
 How to minimize the number of low performers and
maximize the number of top performers?
 Look for characteristics which distinguish the low
performers and top performers from the reference
group
– Student
– Home
– Teaching
– School
Are there key characteristics that could be managed by school
and educational system?
UNIVERSITY OF JYVÄSKYLÄ
Previous findings
 Analyses of earlier large-scale data sets (TIMSS,
PIRLS, PISA, national surveys etc.)
=> Certain characteristics repeatedly appear important
1. Socio-economic background
– Parents’ education and occupation
– Resources at home
– Appreciation of education and culture at home
– Often these explain school differences also
UNIVERSITY OF JYVÄSKYLÄ
Previous findings
2. Student’s attitudes and engagement
– Motivation to school work
– Free-time reading vs other activities
– Self-concept
3. Student’s basic skills
– E.g. reading accuracy and fluency
• Lack of basic skills makes all learning difficult
4. Student’s language background
– Command of instruction language highly important
• ’Everyday’ vocabulary not enough?
UNIVERSITY OF JYVÄSKYLÄ
Previous findings
 Role of gender?
– Girls outperform boys in reading
– Gender differences usually smaller in mathematics
– Gender itself not always significant in multivariate analyses
=> question of attitudes and engagement more than question of
sex?
UNIVERSITY OF JYVÄSKYLÄ
Previous findings
 School and class-level factors play minor role (in Nordic
countries)
– Student characteristics explain much of (already small) school
and class differences
Why is this? Are teachers’ and schools’ efforts relevant at
all?
– Overall level of teachers’ and schools’ efforts and resources
relatively standard (high)?
– Question of teacher-student interaction; student’s contribution
(engagement) crucial
– Difficult to catch with questionnaire data!
UNIVERSITY OF JYVÄSKYLÄ
Data
 We utilize Nordic PIRLS 2011(reading) and TIMSS
2011 (mathematics) data sets
– 4th grade pupils (except 5th grade for Norway)
– Several sources of data:
• student questionnaire
• parent questionnaire (PIRLS)
• teacher questionnaire
• school questionnaire
UNIVERSITY OF JYVÄSKYLÄ
Data
 Finland, Norway and Sweden implemented TIMSS
and PIRLS on the same schools and students
 In Denmark PIRLS and TIMSS had separate student
samples
 Norway assessed 5th grade students also (with
smaller sample!)
– Comparable with other countries’ 4th grade
 FinIand, Norway and Sweden implemented an
additional test of student’s basic reading skills
UNIVERSITY OF JYVÄSKYLÄ
Data
Number of
Students
schools
assessed in
participating PIRLS
in PIRLS
Denmark 4th
Number of
Students
schools
assessed in
participating TIMSS
in TIMSS
Norway 5th
232
145
53
4594
4640
1258
216
145
54
3987
4638
1270
Sweden 4th
152
4622
152
4482
Finland 4th
UNIVERSITY OF JYVÄSKYLÄ
Overall results
 In reading (PIRLS), all four Nordic countries performed
internationally relatively high
– Finland (568) and Denmark (554) in Top 10
– Norway 5th grade (549) and Sweden (542) in Top 20
• Scale midpoint = 500
 In mathematics (TIMSS), Norway 5th, Finland and
Denmark performed high while Sweden was mediocre
– Norway 5th grade (548) and Finland (545) in Top 10
– Denmark (537) in Top 20
– Sweden (504) close to scale midpoint (500)
UNIVERSITY OF JYVÄSKYLÄ
Performance groups
 Define low performers as students below 475 points
475 = intermediate international TIMSS and PIRLS benchmark
 Define top performers as students over 625 points
625 = advanced international TIMSS and PIRLS benchmark
UNIVERSITY OF JYVÄSKYLÄ
Group percentages / reading
% low
performers
in reading:
girls
Denmark 4th
% low
performers
in reading:
boys
% top
performers
in reading:
girls
% top
performers
in reading:
boys
Norway 5th
10
6
9
14
10
13
14
23
12
11
14
8
Sweden 4th
13
17
11
7
Finland 4th
UNIVERSITY OF JYVÄSKYLÄ
Group percentages / mathematics
% low
performers
in maths:
girls
Denmark 4th
% low
performers
in maths:
boys
% top
performers
in maths:
girls
% top
performers
in maths:
boys
Norway 5th
19
17
14
17
17
13
9
10
9
12
13
16
Sweden 4th
34
32
2
4
Finland 4th
UNIVERSITY OF JYVÄSKYLÄ
Statistical modelling
 Three-level logistic regression models for each
country
 A number of explanatory variables were introduced in
the multivariate regression
– Suggestions from previous research
– Interesting variables from the TIMSS and PIRLS background
questionnaires
UNIVERSITY OF JYVÄSKYLÄ
Statistical modelling
 After preliminary steps a reduced variable list was
used in the final models
– Variables appearing significant in at least one country
– Two-way interactions were originally considered but left out
from the final regression models
Recall: multivariate model => effects are adjusted for all
other variables in the model
UNIVERSITY OF JYVÄSKYLÄ
Results
 Student’s basic skills
– Important predictor of performance in every case
– Only minor variation between countries
– Strong relation between skills at school start and 4th grade
performance (stability)
• Early literacy activities with child predict top reading
performance
 Student’s attitudes
– Confidence <=> performance
– Motivation and engagement in learning, no show???
– Liking to read related to reading performance
• Not found with mathematics!
UNIVERSITY OF JYVÄSKYLÄ
Results
 Student’s activities
– Reading activities:
• No role in distinguishing low performers from intermediate
ones
• Instead, top performers stand out from the rest
– Free-time computer use negatively associated with
performance
 Home resources (SES)
– Important in every country, with minor variations
– Educational resources, books, parent’s education,…
– Wealth of neighborhood (DEN, SWE)
UNIVERSITY OF JYVÄSKYLÄ
Results
 Language at home / language difficulties in class
– Some role found in every country
 Parental and teacher support
– Highly significant in every country (in various forms)
– But: more support <=> lower performance
• Top performers’ need for support not recognized?
• Top performers do not express need for support?
UNIVERSITY OF JYVÄSKYLÄ
Results
 Male gender
– Low performance in reading: DEN
– Top performance in maths: FIN, NOR, SWE
 Female gender
– Low performance in maths: SWE
– Top performance in reading: FIN
UNIVERSITY OF JYVÄSKYLÄ
Results
 School-related variables largely non-significant
(given the student, home and teaching related variables)
– Discipline and safety at school
– School emphasis on academic success
– School size and location
– School resources
– Only occasional exceptions:
• Sweden: low performance associated with urban schools
• Denmark: top reading performance associated with urban
schools and good resources
• Denmark: low maths performance associated with small
schools
UNIVERSITY OF JYVÄSKYLÄ
Results
 Teaching-related variables completely non-
significant except for supporting low performers
(given the student, home and school related variables)
– Teacher’s experience and education
– Student ability grouping
– Activities in engaging students
– Teacher’s confidence in teaching maths
– Memorization or elaboration in teaching maths
– Calculator or computer use in class
– Amount of given homework in maths
– Characteristics of reading materials
UNIVERSITY OF JYVÄSKYLÄ
Conclusions
 Gap between low and top performers seems stable
– School cannot even out the impact of background and
differences in basic skills at school start
 support for struggling students still insufficient?
 In reading:
– Early help with problems in linguistic development
– Support in reading engagement, also for (disadvantaged)
families
• Bring school’s and student’s textual landscapes closer
(text materials)
• Family literacy programs
UNIVERSITY OF JYVÄSKYLÄ
Conclusions
 In reading:
– Immigrants: need for intensified support both in instruction
language and in mother tongue
– Development of reading and learning strategies
• Special benefit for struggling readers!
 In mathematics:
– Support in reading!
– Self-concept / confidence / positive attitudes crucial
• Influence of teacher actions and attitudes
• Appropriately tuned tasks and homework
• Bring school’s and student’s ’mathematical landscapes’
closer (tasks, learning materials)
UNIVERSITY OF JYVÄSKYLÄ
Conclusions
 What to do with top performers?
– Teachers and parents may overlook them since everything
goes fine => implications e.g. for school enjoyment?
– Mild interest from adults’ side and lack of challenges may
hinder reaching full potential
 Need for individualized pedagogical solutions, how
about resources?
 Need for continuing professional development of
teachers, how about resources?
UNIVERSITY OF JYVÄSKYLÄ
Tak! Takk! Tack!
Kiitos!