Peter Hendry: CEM Consultant
Monitoring Achievement and Progress in
Independent Schools
Running Your School on Data
January 2011
Peter.Hendry@cem.dur.ac.uk
Running your school on data
CEM data includes:
•
•
•
•
•
•
Baseline test acquired ability data (IPRs)
‘Predictive’ data including chances graphs
Value-added data
Attitudinal data
Curriculum assessments (Insight)
PARIS software programmes
CEM data is used for, e.g.
•
•
•
•
•
•
Teachers to help learners
Curriculum and staffing decisions
Target setting and monitoring pupil progress
Inspection evidence
Self-evaluation
Monitoring changes over time such as pupil
ability intake profiles and VAD
• Asking the question ‘can we do better?’
i.e. the data is used to aid and to support
professional judgement
with due consideration to:
•
•
•
•
Ethos and tradition of ‘my’ school
Accountability
Parental expectations
Staff training in use of data and ability to cope
with data (data overload)
• Integrating the data into school procedures,
storage, retrieval, distribution and access
(..........policy?)
i.e. doing our best to help every pupil to at least
achieve, if not exceed, their potential
So much data................................!
Some key questions:
1. Which data do I need AND which
data do I not need?
2. What does the data mean and what
doesn't the data mean?
3. Who is the data for? Who does not
need to have some of the data?
1. What data do I need? A key first step!
e.g. using MidYIS GCSE predictive data for target
setting
•
•
•
•
•
Based on whole MidYIS cohort?
Based on the Independent Sector?
Based on the school’s prior value-added?
Point and/or grade predictions? (e.g. 6.2/grade B)
The chances graphs?
subject predictions: all MidYIS cohort
9
8
7
6.4
6
5
6.2
5.7
4
3
Biology
2
Business studies
1
Mathematics
0
60
65
70
75
80
85
90
95
100
105
110
MidYIS baseline score:120
115
120
125
130
135
140
9
subject predictions: Independent Sector
8
7.1
7
6
7.0
6.4
5
4
Biology
Business studies
3
Mathematics
2
1
0
60
65
70
75
80
85
90
95
100
105
110
115
MidYIS baseline score: 104
120
125
130
135
140
Comparing MidYIS predictions
MidYIS Point Predictions
Biology
Business Studies
Mathematics
Midyis
Score:
120
6.4
5.7
6.2
Equivalent
Independent
Sector
Score:
104
7.1
6.4
7
40
Individual Chances Graph for ADAM BECKSMITH - GCSE English
MidYIS Score 121 MidYIS Band A
Teacher's Adjustment : 0 grades / levels / points
34
35
34
30
Percent
25
Prediction 6.3 A/B
20
17
15
12
10
3
5
0
0
0
0
U
G
F
E
0
D
Grade
C
B
A
A*
Individual Chances Graph for ADAM BECKSMITH - GCSE English
MidYIS Score 105 MidYIS Band B
55
50
45
45
Prediction 6.9 A
40
Percent
35
(Independent Sector)
30
25
25
25
20
15
10
5
5
0
0
0
0
0
U
G
F
E
D
0
Grade
C
B
A
A*
MidYIS Score 121 MidYIS Band A
Prior VA Adjustment : 1 grades / levels
50
45
41
39
40
Prediction 7.4 A*/A
35
Percent
30
25
20
16
15
10
4
5
0
0
0
0
0
U
G
F
E
D
0
Grade
C
B
A
A*
Discussion: setting the targets...........
• Which type of predictive data would you use to
set the targets, and why?
• Would your students be involved as part of the
target setting process?
• Would parents be informed about the process
and outcome?
• How would you ensure that HoDs were involved
to ensure overview the process?
2. What does the data mean?
e.g. value-added data:
• The difference between raw and standardised
residuals
• The use of confidence limits to distinguish between
average and statistically significant data, and to allow
for small entry subjects
• Can ‘zero’ or negative value-added be acceptable?
Raw residual Bar chart
(MIDYIS and YELLIS only)
0.0
-0.3
Voc Health & Social Care
0.2
SC Religious Studies
SC Physical Education
-1.0
SC ICT
Science: GCSE Additional
0.1 0.1
Science: GCSE
Science: Other
0.1
Welsh
-0.3
Religious Studies
0.2 0.2 0.2
Physics
-0.3
Physical Education
0.3
Music
0.1
Mathematics
-0.2 -0.2
ICT
0.3
Home Economics
0.0
History
0.1 0.1
German
1.5
Geography
1.0
French
English Literature
0.0
English
-2.0
Drama
-1.5
-0.1
Design & Technology
-0.2
Chemistry
0.5
Business Studies
-0.5
Biology
Art & Design
Average Standardised Residual
Standardised Residual Bar Chart
2.5
2.0
99.7% confidence limit
95% confidence limit
0.9
0.1
0.0
-0.2
-0.7
-0.9
ANY VALUE IN THE INNER SHADED AREA IS
WITHIN THE EXPECTED RANGE AND IS
THEREFORE CONSIDERED TO BE AVERAGE
-2.5
Raw residual Bar chart
(MIDYIS and YELLIS only)
0.0
-0.3
Voc Health & Social Care
0.2
SC Religious Studies
SC Physical Education
-1.0
SC ICT
Science: GCSE Additional
0.1 0.1
Science: GCSE
Science: Other
0.1
Welsh
-0.3
Religious Studies
0.2 0.2 0.2
Physics
-0.3
Physical Education
0.3
Music
0.1
Mathematics
-0.2 -0.2
ICT
0.3
Home Economics
0.0
History
0.1 0.1
German
1.5
Geography
1.0
French
English Literature
0.0
English
-2.0
Drama
-1.5
-0.1
Design & Technology
-0.2
Chemistry
0.5
Business Studies
-0.5
Biology
Art & Design
Average Standardised Residual
Standardised Residual Bar Chart
2.5
2.0
99.7% confidence limit
95% confidence limit
0.9
0.1
0.0
-0.2
-0.7
-0.9
ANY VALUE IN THE INNER SHADED AREA IS
CONSIDERED TO BE AVERAGE VALUE ADDED
-2.5
0.2
0.3
0.0 0.0
0.2
0.0
0.5 0.5
0.4 0.3
0.5
-0.1
-0.5
0.0 0.1
-0.9
-1
0.3 0.3
0.6 0.5 0.6
0.9
0.8
1
0
Average Standardised Residual
Standardised Residual Bar Chart
Average Standardised Residuals by Subject
MidYIS Year 7 2004/2005 to GCSE 2009
2
1.5
-1.5
-2
Statistics
Spanish
Science
Religious Studies
Physical Education
Music
Mathematics
Latin
Home Economics
History
German
Geography
French
English Literature
English
Drama
Design &
Technology
Business Studies
Bus. & Comm.
Systems
Art & Design
Additional Science
3. CEM data: who is it for? For example:
•
•
•
•
•
•
•
•
SMT/SLT:
HoDs:
Subject teachers:
Form/House Tutors:
Head of Learning/Year:
Parents:
Pupils:
Governors:
Case study: Alis value-added data
Four sets of VAD are available!
From average GCSE baseline:
• all Alis cohort
• type of Institution (Independent Sector)
• syllabus
From the baseline test
• all Alis cohort
School SPC Chart with confidence limits: Baseline Av. GCSE
All Alis Cohort:
Syllabus
Institution
Using PARIS software: CABT Baseline
Whole School
From your perspective, which set of VAD would you use for the
different user groups? (Governors, HoDs, Parents, SMT/SLT...)
1.2
Additional Science
1.4
Art & Design
0.8
Biology
1.2
Chemistry
A key question is
therefore.......
0.8
Classical Civilisation
2.0
Design & Technology
1.7
Drama
1.1
English
0.8
English Literature
1.6
French
‘Is it possible to
keep adding value
at each key
stage?’
1.4
Geography
1.5
German
1.3
History
0.5
Latin
1.0
Mathematics
0.8
Music
1.2
Physical Education
1.1
Physics
1.5
Religious Studies
1.3
Science
1.4
Spanish
-3.0
-2.5
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0
2.5
Average Raw Residual
Typical Av. GCSE Subject Raw Residuals
3.0
Average Standardised Residuals by Subject
MidYIS Year 9 2005/2006 to GCSE 2008
Typical Av. GCSE Subject Standardised Residuals
2
1.5
1.3
Average Standardised Residual
1.1
1.1
1.0
1.2
1.0
1
0.7
1.2
1.1
0.9
0.9
0.8
0.6
0.5
1.1
1.2
1.0
1.0
0.5
0.4
0.5
0.9
0
-0.5
-1
-1.5
A2 Intake Profiles: typical average GCSEs for students in each subject
-2
Spanish
Science
Religious Studies
Physics
Physical Education
Music
Mathematics
Latin
History
German
Geography
French
English Literature
English
Drama
Design &
Technology
Classical
Civilisation
Chemistry
Biology
Art & Design
Additional Science
2009 A2
2010 A2
2009
2010
•
USE ONE YEARS DATA WITH CAUTION!
•
Better to use three years data as patterns over time are
more significant.