Peter Finlayson
Quality improvement Officer
February 2013
Why Highland are using CEM data
1.
2.
3.
4.
5.
6.
How CEM data might help inform the
Transition process
Some knowledge of pupil feedback
information from SOSCA
Pupil Tracking-how SOSCA helps
Where CEM Data might fit in a school’s self
evaluation
Where added value can be identified
Self Evaluation for the Authority






29 Secondary schools
183 Primary Schools
All Primary schools use InCAS in P3,P5 and P7
All secondary schools use SOSCA in S2
A few use MidYIS in S1
Financial Commitment!!
•
•
•
•
•
Improve outcomes for Young People
Teachers still developing Assessment
Standards
Qualifications will still be a major part of a
pupil’s portfolio of achievement
Qualifications will be norm referenced
CEM Background
Key Questions


How do we know how well pupils are
progressing
Are they achieving their potential?
About Assessment for Excellence (AfE)
· AfE is a suite of assessments that provides information about pupils'
attainment and attitudes, complementing teachers' assessments of pupils'
progress through CfE.
· CEM's AfE tests and feedback software enable a personalised approach to
assessment in a format that pupils can relate to.
· AfE can ease assessment processes and aims to provide a stable and
long-term approach to school evaluation.
· CEM assesses more than one million pupils every year and has a wealth
of experience in providing accurate, reliable information. This information
can help inform teaching and management decisions and can really make a
difference to the outcomes of the child.
· CEM's AfE produces data which is nationally representative and have
year-on-year comparisons so teachers can compare students' performance
against established norms.
Key Principles
"It comes down to the quality of what happens in the classroom, putting
children at the centre. If we know a child can attain better, how can we
help? We need to become interventionists.“
Head of Education for Fife, Craig Munro
"The big question for many people is, 'How do we translate this data into
improving learning?' We (CEM) offer research-based advice on how to help
children succeed in the classroom and we give detailed feedback which the
teacher can use alongside their detailed professional knowledge of each
pupil.“
Director of Research and Development for CEM, Christine Merrell
"We don't want assessments that don't support learning. We want to
improvements in learning, teaching and schools' performance.“
promote
Ken Greer, Executive Director of Education for Fife Council
The information provided through CEM assessments can help teachers
monitor pupils' progress, set targets, facilitate school improvement and
inform the teaching and learning process.
Transition





All primary schools in Highland use InCAS
All Primary schools use the Highland SPP
Approach
Professional Judgement backed up by InCAS
data
5-14 on its own was flawed!!
All Secondary schools receive the AfE data
from Primary as Standardised Scores through
Phoenix e1
Moving Forward with Assessment
The Personal Progress Scale
1 – is consistently achieving his/her potential
2 – is consistently achieving success but has the potential to achieve more
3 – is only sometimes achieving potential
4 – is yet to achieve potential
Moving Forward with Assessment
The Performance Scale
A
B
C
D
- is consistently performing beyond the expected standard for his/her stage
- is consistently performing at the expected standard for his/her stage
- is sometimes performing at the expected standard for his/her stage
- is not yet performing at the expected standard for his/her stage
SOSCA
Feedback
• Assessed by a computer adaptive test
• School and students’ test performances are compared
to a nationally representative sample







Assessment Feedback:
Pupil scores and records, and school Band Profiles
‘Predictions’ Feedback:
To S4 qualifications (currently Standard Grade and will for
National 4 and 5) based on overall curriculum score
Value-Added Feedback:
For Standard Grade and will for National 4 and 5 from
curriculum baseline (SOSCA)
for curriculum assessment (SOSCA) from P7 InCAS or S1
/S2 MidYIS baselines
InCAS and SOSCA Data is
used to inform teacher
judgements around the
SPP

Chances Graphs
Individual Chances Graph for student 3- Standard Grade English
Overall AfE Score 111, Band A
55
49
50
45
49% chance of a
grade 2 – the
most likely single
grade. 51%
chance of a
different grade
40
Percent
35
28
30
25
17
20
15
10
5
5
0
0
0
7
6
5
0
4
Grade/Level
Chances Graphs based on Pupil’s AfE Test Score
3
2
1
‘Predictions’…...are based on
Average performance by similar
students in past examinations
So the term ‘prediction’ should not be used. Alternatives
could include:
‘expected grade’: from the table of data
‘most likely grade’: from the chances graphs
Reliability of ‘predictions’?
• School factors inc. catchment area
• How seriously the pupil took the assessment.
Quotes from Highland Schools
“We have used them for target setting in S3.”
“We do 'working at grades' twice a year and this year we used SOSCA as a guide to what
they should be working at.”
“I have used them to compare results and predictions in the department.”
“All HOD's have been trained on the use of SOSCA.”
“All departments are aware of the information and should know what the
information means (!) More time was spent with Guidance, LS, English, Maths
and Science going over the data. Guidance staff have all printed the IPRs,
they have been placed in the pupil files and refer to them regularly. ”
“Learning Support and English make regular use of the information. ”





Important role for the Key Adult?
Tracking of attainment-one aspect
Individual Pupil record from SOSCA?
Course Choice at end of S2?
Course Choice at end of S3?




STACS –comparator schools
SOSCA Tables
Band Profile Graphs
Value Added Data
AfE S2 Curriculum based Standardised Feedback 2012
Scores Standardised On AfE Curriculum based Sample 2012
Standardised Scores
Sort Data By...
Mean Score: 100
Standard Deviation: 15
School A
Average
St. error
St. error
97.0
98.7
97.3
91.3
93.4
94.9
92.2
94.0
98.0
94.0
94.9
1.3
1.3
1.3
1.3
1.3
1.4
1.3
1.4
1.3
1.3
1.3
1.3
103.7
101.1
100.7
102.2
103.1
106.2
101.1
103.7
101.5
105.0
103.2
103.8
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
Band
St. Score
Overall Science
Energy & Forces
Band
St. Score
Earth & Space
Band
St. Score
Band
Living Things &
the Processes of
Life
St. Score
Band
St. Score
Band
Overall
Mathematics
Science
Shape, Position &
Movement
St. Score
Band
Information
Handling
St. Score
Band
Number, Money &
Measurement
St. Score
Overall Reading
Band
St. Score
Band
Passage
Comprehension
St. Score
Band
St. Score
Mathematics
97.7
School B
Average
Band
St. Score
Speed Reading
Text
Comprehension
Reading
School A
Overall Mathematics 2012: Percentage of Pupils in each Band
100
90
90
80
80
70
70
60
60
Percent
100
50
40
34
24
30
44
50
40
30
23
19
20
23
18
20
10
15
10
0
0
D
C
B
Band
A
D
C
Overall Science 2012: Percentage of Pupils in each Band
100
90
80
70
60
Percent
Percent
Overall Reading 2012: Percentage of Pupils in each Band
50
40
35
30
29
22
15
20
10
0
D
C
Band
B
A
Band
B
A
School B
Overall Mathematics 2012: Percentage of Pupils in each Band
100
90
90
80
80
70
70
60
60
Percent
100
50
40
29
30
20
50
38
40
30
25
30
16
10
23
22
17
20
10
0
0
D
C
Band
B
A
D
C
Overall Science 2012: Percentage of Pupils in each Band
100
90
80
70
60
Percent
Percent
Overall Reading 2012: Percentage of Pupils in each Band
50
37
40
30
20
17
24
22
10
0
D
C
Band
B
A
Band
B
A
Average Standardised Residuals
MidYIS Y8/S1 to SOSCA Value-Added 2012
Overall Mathematics
Standardised for Nationally Representative Sample
0.5
0.4
0.3
0.28
0.1
0
-0.1
-0.2
-0.3
-0.4
-0.5
5142830
Average Standardised Residual
0.2
Average Standardised Residuals
MidYIS Y8/S1 to SOSCA Value-Added 2012
Overall Reading
Standardised for Nationally Representative Sample
0.5
0.4
0.3
0.17
0.1
0
-0.1
-0.2
-0.3
-0.4
-0.5
5142830
Average Standardised Residual
0.2
Value Added Data
Added Data
Authority View




School A and School B
Inspection reports
Attainment Data
SIMD Data
School A 2008
By the end of S2, pupils were making very good progress and
attaining well across the Curriculum..
Most pupils were attaining appropriate levels in mathematics,
reading and writing.
By the end of S4 , the proportion of pupils attaining five or more
Awards at SCQF levels 4 and 5 was well above national averages,
and in line with schools with similar characteristics.
School B 2009
Across the curriculum, young people achieve well in S1/S2
The majority achieve appropriate national standards by the
end of S2
In recent years, the success of young people in S4, in national
examinations, has generally been in line with the national average.
It is stronger than in schools which serve young people
with similar needs and backgrounds
Centre name
School B
School A
% pupils % pupils
from 15% from 20%
most
most
No. pupils from 15% No. pupils from
deprived deprived
No. pupils in known most deprived
20% most deprived
datazones datazones
datazones
datazones
datazones
427
35
81
692
0
0
8.2%
0.0%
19.0%
0.0%
SOSCA Information
0.5
1.5
AfE S2 Curriculum based Baseline 2011/2012
Overall AfE Scores Standardised for AfE S2 Sample
Selected Schools
2.5
3.5
4.5
170
160
150
140
130
120
110
103.1
100
90
80
70
School A
School B
60
50
40
30
Cohort Mean
Outliers
Consortium Mean
Percentages and NCDs based on S4 Roll for passes achieved by end of S4
All Candidates
School A
School B
5+ Level 4
5+ Level 5
%
NCD
%
NCD
2012
92
2
55
2
2011
95
1
55
2010
91
2
2009
93
2008
90
5+ Level 4
5+ Level 5
%
NCD
%
NCD
2012
82
5
30
7
2
2011
72
8
22
9
55
2
2010
85
4
29
7
1
54
1
2009
84
4
32
6
2
48
2
2008
85
3
31
6