Introduction to CEM and
Computer Adaptive
Assessments
Belfast, March 6th 2013
Neil Defty
Business & Development Manager
CEM
Neil@cem.dur.ac.uk
Who / What is CEM ?
Centre for Evaluation & Monitoring
•
•
•
•
•
•
•
•
•
Not for Profit Organisation
Part of Durham University
Close links to School of Education
Established 1983
Nursery / Reception  Post-16
Monitoring Systems, Research and Evaluation Projects
UK & International
Over 1/3 of UK Secondary & Post-16 establishments
Informed by Evidence from Research
CEM Provides
• Information Systems from Nursery to Post-16
 Student Guidance & Chances
 Performance Monitoring
• Evaluation & Research Projects
• Entrance Testing
• Support – Phone, email, documentation
• Training
• INSET
• Roadshows
• Conferences (including exhibitions)
The Systems
The Systems
A / AS / Btec / IB etc
Year 13
Year 12
Alis
GCSE
Computer Adaptive Baseline Test
GCSE
KS3
Year 11
Yellis
Year 10
Computer Adaptive Baseline Test
INSIGHT
Year 9
Year 8
Year 7
Combines curriculum tests
with developed ability
MidYIS
Computer Adaptive Baseline Test
Typical Timeline
Typical Timeline
Early Testing &
Predictions
START:
Measure Baseline
Autumn Term
Feedback
Reports
A ugust
END:
September
Collect and Send Results
Value-Added
Feedback
Testing is available from June at end
of previous academic year
e.g. Y6 / Y9 / Y11
The baseline score is measured as
early as possible and the information
sent to CEM
The institution then receives feedback
from CEM: baseline test results and
predictions
The public examination results are
sent to CEM when received by the
institution
The value added data is available
from September via the Institution’s
secure website at CEM
Basic Principles:
•
•
•
•
Measure Student’s ability


Intake Profiles
IPR - Strengths and Weaknesses
Project future performance from national trends

Predictions
Record Final Grade
Compare actual performance to Projected
Performance

Value-Added Reports
Baseline Assessment and the
Computer Adaptive Baseline Test
(CABT)
Baseline Assessment
•
•
•
•
Year 7 (8)
Year 9
Year 10 (11)
Year 12 (13+)
Computer Adaptive Baseline Test
Vocabulary
Maths
Non Verbal
What is an Adaptive test ?
•
All questions allocated to groups of different difficulty
•
All students in a year group get common difficulty starting question
•
If answer correct, next question harder
•
If answer wrong, next question easier
•
Test looks at all answers in a section and homes in on questions of a
suitable difficulty for student
Try it yourself at www.intuproject.org/demos
Adaptive approach
Low
Average
High
X

 






Standardisation
•
Test scores are standardised; Mean = 100, SD = 15
Standardised
Score
National
Percentage
Comment
>130
Top 2.5%
Traditional classification of ‘mentally gifted’
>120
Top 10%
>100
Top 50%
<80
Bottom 10%
<70
Bottom 2.5%
Potential special educational needs?
The Computer Adaptive Baseline Test:
• Adaptive components common to MidYIS, Yellis, and Alis
are:
–
–
Vocabulary
Maths
• Non- adaptive sections:
–
–
Non-verbal (MidYIS, Yellis, Alis)
Skills (MidYIS only)
• Assessments are ‘curriculum free’: i.e. students are not
taught prior to the test
• Measure of acquired ability
• There is no preparation for the test
Difficult vocabulary
Difficult maths
x/4 + 2
Non-Verbal: Cross-sections
Non-Verbal: Blocks
3
Non-Verbal:
Pictures Pictures
Skills: Proof Reading
Skills: Perceptual Speed and Accuracy
Alis - Patterns
Alis – Logical Reasoning
INSIGHT
Curriculum-based assessments covering:
Maths
- Number & Algebra
- Handling Data
- Space, Shape & Measurement
Science - Biology, Chemistry, Physics,
- Attitudes to Science
Reading - Speed Reading
- Text Comprehension
- Passage Comprehension
Additionally for full INSIGHT:
Developed Ability
•
Vocabulary
•
Non verbal, Skills
•
Attitudinal measures
INSIGHT - Maths
7
INSIGHT - Maths
INSIGHT - Science
Plus Attitude to Science: Perception of Science, Relevance to Self,
Environmental Actions, Enjoyment of Science
INSIGHT – Reading
(Text Comprehension)
INSIGHT – Reading
(Passage Comprehension)
INSIGHT – Reading
(Speed Reading)
INSIGHT – Developed Ability