Using Alis Predictive
Data
Dr Robert Clark
Alis Project Manager
Predictions
vs
Targets
What is an Alis ‘Prediction’ ?
 NOT a forecast of the grade the student will get
 An indication of the grade (points score) achieved on average by students
of similar ability in the previous year
Targets ?
•
Minimum Targets – Round Alis prediction down ?
•
Realistic Targets – Use nearest Alis grade ?
•
Challenging Targets – 75th percentile ?
Prior Value Added ?
Arbitrary grade fraction ?
75th Percentile Predictions
•
•
•
•
Excel spreadsheet - ‘Predictions – Spreadsheet (75th Percentile)’
If all students attain 75th percentile predictions, School VA will be at top 25%
Approx 1/5 grade per student per subject
Can also be generated in PARIS software
Prior Value Added
•
•
•
•
Only where prior VA is positive ?
1 year or 3 ?
Reasonable to use raw residual figures as this is an approximate measure
and raw residuals give grade fractions
Can be calculated using PARIS software
Data used to inform, not replace, professional judgement
Understanding Your Students:
Baseline & Predictive Data
Intake Profiles
Intake Profiles (Historical)
Full Alis 2009 Demo School (999)
IPR...
Banana, Brian
Studying :
Maths
Physics
Chemistry
Biology
?
Banana, Brian
Prediction Reports
Probability of achieving
each grade
Expected Grade
Which predicted
grades are the most
appropriate for ths
student ?
Predictions
Based on GCSE
(7.0)
B
B
C
B
B
Predictions
Based on Test
(106)
What is this Student’s ability ?
What Grades should we expect her
to get ?
If she gets C’s instead of B’s, is
this a problem ?
C
B
D
B
C
Why is the pedicted grade not always equal
to the highest bar ?
Predicted (‘expected’) grade
Most likely grade
Prediction Reports
Subject
Report
A2 vs AS predictions and
the impact of the A* Grade
2009 Regression Equations
70
140
AS Physics
120
A2 Physics
50
100
40
80
30
60
20
40
10
20
0
A2 UCAS Points
0
4
4.5
5
5.5
6
6.5
7
7.5
8
Average GCSE Score
2010 Regression Equations
70
140
AS Physics
60
120
A2 Physics
50
100
40
80
30
60
20
40
10
20
0
0
4
4.5
5
5.5
6
6.5
Average GCSE Score
7
7.5
8
A2 UCAS Points
AS UCAS Points
AS UCAS Points
60
2009 Regression Equations
70
140
AS Psychology
120
A2 Psychology
50
100
40
80
30
60
20
40
10
20
0
A2 UCAS Points
0
4
4.5
5
5.5
6
6.5
7
7.5
8
Average GCSE Score
2010 Regression Equations
70
140
AS Psychology
60
120
A2 Psychology
50
100
40
80
30
60
20
40
10
20
0
0
4
4.5
5
5.5
6
6.5
Average GCSE Score
7
7.5
8
A2 UCAS Points
AS UCAS Points
AS UCAS Points
60
Worked Examples:
Baseline Data & Predictions
Refer to the Intake Data on the next 2 slides
• For each school what deductions might you make ?
• What implications are there (if any) for teaching &
learning ?
School A
School B
Refer to the Y12 data on the next 2 slides.
• What impact might there be on the pupil’s learning ?
• What subjects would you be worried about them
studying ?
Note : Non Verbal section includes Perceptual Speed and Accuracy, Pattern Matching, logical
reasoning and dice folding
Y12 - Pupil D
Y12 – Pupil E
Refer to the data on the next 3 slides.
• Does the data show any ‘warnings’ about future
potential achievement?
• Based only on the information provided, what
would be realistic subject targets for the students,
and why?
Student 1
Student 2
Student 3
Worked Examples:
Target Setting
Basing Targets on Prior VA – One Methodology from an
Alis School
•
Discuss previous value added data with each HoD
•
Start with an agreed REALISTIC representative figure based, if available
on previous (3 years ideally) of value added data
•
add to each pupil prediction, and convert to grade (i.e. in-built value
added)
•
Discuss with students, using professional judgement and the chances
graphs, adjust target grade
•
calculate the department’s target grades from the addition of individual
pupil’s targets
DEPARTMENT: A
Target Setting
year
2002
2003
2004
2005
no. of
pupils
av. GCSE
av. TDA
2
7
6
6.8
7.1
6.6
49.0
49
51
12
raw resid.
Std.
Resid
3yr. Av.
Std resid
24.5
13.3
18.2
1.2
0.6
0.7
0.7
0.8
0.8
6.17±0.22 44.50±3.84 12.82±4.05 0.60±0.29
0.65
From and including 2002, a raw residual of 20.0 is equivalent to one grade
SUGGESTED TARGETS FOR 2007, based on ALIS pred and dept's value added history
The target grade has an in-built value added of 15 points (one grade is 20 points)
target
dept adj
grade
target
the total target grades are as follows:
A
1
0
B
2
3
C
6
5
D
1
1
E
0
0
Surname
Forename
AveGCSE
4.7
5.8
6.9
6.2
5.1
5.5
5.4
5.2
6.1
AVERAGE
5.7
TDA
28
30
48
61
39
30
54
33
53
41.8
Prediction TARGET
49.3
64.3
73.2
88.2
96.4
111.4
80.8
95.8
57.8
72.8
66.3
81.3
63.4
78.4
59.9
74.9
79.1
94.1
69.6
84.6
target
grade
D
C
A
B
C
C
C
C
B
C
Teacher
adj target
D
C
B
C
B
C
C
C
B
RESULT
D
C
B
C
B
D
B
C
B
DEPARTMENT:
B
year
no. of
pupils
2005
6
av. GCSE av. TDA
raw resid.
av. Std. 3yr. Av.
Resid Std resid
5.41±0.20 45.33±3.34 -15.42±14.15 -0.60±0.41
SUGGESTED TARGETS FOR 2007, based on ALIS prediction
The target grade has an in-built value added of 0 points (one grade is 20 points)
target
grade
the total target grades are as follows:
A
0
B
1
C
6
D
1
E
0
Surname
Forename
AveGCSE
4.9
6.3
6.5
5.8
7.4
6.3
6.1
6.2
AVERAGE
6.2
TDA
50
38
53
34
53
42
46
59
46.9
Prediction
50.7
83.4
88.2
71.7
108.4
82.7
78.7
81.1
80.6
TARGET
50.7
83.4
88.2
71.7
108.4
82.7
78.7
81.1
80.6
target
grade
D
C
C
C
B
C
C
C
dept adj
target
2
1
4
1
0
dept adj
grade
D
C
B
C
A
A
C
C
RESULT
D
C
A
B
A
A
B
D
Discussion
•
Assess the merits and concerns you may have with this valueadded model of setting targets
Dr Robert Clark
Alis Project Manager
robert.clark@cem.dur.ac.uk
0191 33 44 193