Introduction to CEM and the
Baseline Tests
Robert Clark
Neil Defty
Nicola Forster
Who / What is CEM ?
Centre for Evaluation & Monitoring
•
•
•
•
•
•
•
•
•
Not for Profit Organisation
Part of Durham University
Close links to School of Education
Established 1985
Nursery / Reception  Post-16
Monitoring Systems, Research and Evaluation Projects
UK & International
Over 1/3 Secondary & Post-16 Establishments
Informed by Evidence from Research
The Systems
The Systems
A / AS / Btec / IB etc
Year 13
Year 12
Alis
GCSE
Computer Adaptive Baseline Test
GCSE
KS3
Year 11
Yellis
Year 10
Computer Adaptive Baseline Test
INSIGHT
Year 9
Year 8
Year 7
Combines curriculum tests
with developed ability
MidYIS
Computer Adaptive Baseline Test
Typical Timeline
Typical Timeline
Early Testing &
Predictions
START:
Measure Baseline
Autumn Term
Feedback
Reports
A ugust
END:
September
Collect and Send Results
Value-Added
Feedback
Testing is available from June at end
of previous academic year
e.g. Y6 / Y9 / Y11
The baseline score is measured as
early as possible and the information
sent to CEM
The institution then receives feedback
from CEM: baseline test results and
predictions
The public examination results are
sent to CEM when received by the
institution
The value added data is available
from September via the Institution’s
website at CEM
Baseline Assessments of
Developed Ability
Baseline Assessment
•
•
•
•
Year 7 (8)
Year 9
Year 10 (11)
Year 12 (13+)
Computer Adaptive Baseline Test
Vocab
Maths
Non Verbal
What is an Adaptive test ?
•
All questions allocated to groups of different difficulty
•
All students in a year group get common difficulty starting question
•
If answer correct, next question harder
•
If answer wrong, next question easier
•
Test looks at all answers in a section and homes in on questions of a
suitable difficulty for student
Try it yourself at www.intuproject.org/demos
X

 






Standardisation
•
Test scores are standardised; Mean = 100, SD = 15
Standardised
Score
National
Percentage
Comment
>130
Top 2.5%
Traditional classification of ‘mentally gifted’
>120
Top 10%
>100
Top 50%
<80
Bottom 10%
<70
Bottom 2.5%
Potential special educational needs ??
The Computer Adaptive Baseline Test:
• Adaptive components common to MidYIS, Yellis, and Alis
are:
–
–
Vocabulary
Maths
• Non- adaptive sections:
–
–
Non-verbal (MidYIS, Yellis, Alis)
Skills (MidYIS only)
• Assessments are ‘curriculum free’: i.e. students are not
taught prior to the test
• Measure of acquired ability
• There is no preparation for the test
Difficult vocabulary
Difficult maths
Non-Verbal: Cross-sections
Non-Verbal: Blocks
Non-Verbal:
Pictures Pictures
Skills: Proof Reading
Skills: Perceptual Speed and Accuracy
INSIGHT
Curriculum-based assessments covering:
Maths
- Number & Algebra
- Handling Data
- Space, Shape & Measurement
Science - Biology, Chemistry, Physics,
- Attitudes to Science
Reading - Speed Reading
- Text Comprehension
- Passage Comprehension
Additionally for INSIGHT:
Developed Ability
•
Vocabulary
•
Non verbal, Skills
•
Attitudinal measures
INSIGHT - Maths
INSIGHT - Maths
INSIGHT - Science
Plus Attitude to Science: Perception of Science, Relevance to Self,
Environmental Actions, Enjoyment of Science
INSIGHT – Reading
(Text Comprehension)
INSIGHT – Reading
(Passage Comprehension)
INSIGHT – Reading
(Speed Reading)
INSIGHT – Developed Ability
Measuring Ability
Key Stage Data
or Baseline Test ?
Key Stage Baselines – KS2; GCSE
Strengths
Weaknesses
Related to curriculum
Dependent on teaching effectiveness (prior ValueAdded) as well as student ability
Automatically available
Open to manipulation
In Depth
Not all students have KS data
Linked to student learning
experience
Reliability of KS measure
Consistency of KS score (i.e. GCSE & Btec First etc)
Independent Baseline Test – CEM Adaptive Test
The Effect of Prior Value Added
Beyond Expectation
In line with Expectation
Below Expectation
+ve Value-Added
0 Value-Added
-ve Value-Added
Average GCSE = 6
Average GCSE = 6
Average GCSE = 6
Do these 3 students all have the same ability ?
Baseline Feedback
Feedback from Baseline Test
• Intake Profiles
• Individual Pupil Records (IPRs)
• Predictions & Chances Graphs
Individual Pupil
Record Sheets
(IPRs)
Look for sections that
are inconsistent
Available in all
systems pre & post 16
Predictions
.
How CEM ‘Predictions’ are made…
Subject
Subject X
X
Outcome
140
A* / A120
100
C 80
60
40
20
0
04
25
46
67
8
Baseline
Regression Line (…Trend Line, Line of Best Fit)
Outcome = gradient x baseline + intercept
Correlation Coefficient (~ 0.7)
Some Subjects are More Equal than Others….
140
A*
A
Grade
120
B
100
>1 grade
Photography
Sociology
English Lit
Psychology
Maths
Physics
Latin
80
C
60
D
E
40
5
C
6
B
A7
Average GCSE
A*8
Some Subjects are More Equal than Others….
GCSE (MidYIS or Yellis)
A*
A
GCSE Grades
B
1 grade
C
D
E
F
Test Score
Art & Design
Biology
Chemistry
Economics
English
French
Geography
German
History
Ict
Mathematics
Media Studies
Music
Physical Education
Physics
Religious Studies
Science (Double)
Spanish
Predictions – MidYIS example
Similar spreadsheets available from Yellis & INSIGHT
Individual Chances Graph for Student no.5 - GCSE English
MidYIS Score 82 MidYIS Band D
40
33
35
Prediction/expected
grade: 3.8 grade D
Percent
30
23
25
23
Most likely grade
20
15
10
10
5
2
5
3
1
0
A
A*
0
U
G
F
E
D
Grade
C
B
Predictions - Alis example
Why KS Baselines do not
always tell the whole story
Student 1
Student 2
Student 2 - IPR
Student 3
Student 3 - IPR
Vocab Item Responses
Correct
30
Response Time (s)
25
20
15
10
5
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
Question
Wrong
Thank You
Robert Clark – robert.clark@cem.dur.ac.uk
Neil Defty – neil.defty@cem.dur.ac.uk
Nicola Forster – nicola.forster@cem.dur.ac.uk