eaa
The e-Assessment Association
What can e-assessment
do for
Teaching & Learning?
Cliff Beevers and Bill Foster
e-Assess Association
On behalf of eAA expert panel
www.e-assessmentassociation.com
eaa
The e-Assessment Association
Introduction
• caa.ecs.soton.ac.uk/Papers/
• Technology to serve Education
– Build on research
• More complex question & test structure
– Fast/accurate feedback
• Efficient use of data
www.e-assessmentassociation.com
eaa
The e-Assessment Association
Definition
E-assessment occurs when there is an
automated marking/response, to
student input on-screen in a test,
informing on the process of answering
a question and providing feedback to
learners and their teachers through
well-crafted advice and reports.
www.e-assessmentassociation.com
eaa
The e-Assessment Association
Outline
• ARG principles
– Successful e-assessment
• Other research
– Greenwood et al (1998)
Int Conf Mathematics / Science Educ and Tech, Vol. 2000, Issue. 1, 2000, 179
– Sandine et al (2005)
“The National Assessment of Educational Progress in Online Assessment in
Mathematics and Writing”
– Ashton et al (2005)
BJET, 36(5), 771-787
www.e-assessmentassociation.com
eaa
The e-Assessment Association
ARG’s formative principles
• Effective planning of T&L
• Focus on how students learn
• Centre of classroom practice
• Promote professional skills
• Sensitive/constructive
www.e-assessmentassociation.com
eaa
The e-Assessment Association
ARG’s formative principles
• Foster motivation
• Promote learning goals
• Help improve learning
• Develop self-assessment
• Recognise range of achievement
www.e-assessmentassociation.com
eaa
The e-Assessment Association
ARG’s summative principles
• Valid: measure appropriate achievements
• Reliable: accurate and consistent
• Impact: desirable for T&L
• Practical: resources provide value
www.e-assessmentassociation.com
eaa
The e-Assessment Association
Examples of formative success
• ALTA at Primary/Secondary
• SCHOLAR at Secondary/Tertiary
• Medicine at Dundee
• Maths at Newcastle/Birmingham
• Physics at Surrey
• Open University courses
• Professional eg Accountancy
www.e-assessmentassociation.com
eaa
The e-Assessment Association
Newcastle Experience:
Formative Headlines
•Fully integrated e-learning and formative
assessment system in the School of Maths &
Stats. since 2006.
•Used for all M & S modules in first two years
as well as some large service courses.
•Easy to use, good feedback from students.
• Challenging questions at the right level
(symbolic input - little or no MCQ).
•More opportunities for practising skills
•Use: ~4000 students from all three
faculties. Around 180,000 sessions so far.
www.e-assessmentassociation.com
eaa
The e-Assessment Association
Newcastle: Why?
•More formative assignments needed to
monitor progress and with timely feedback.
•More opportunities for student practice in
basic skills needed.
•Diversify assignment methods.
•Marking issues of standard paper based
assignments for first year courses: time,
money and uniform standards in marking.
•Decided on CBA – biased towards formative
assessment based on experience at Brunel,
Heriot-Watt and Southampton Universities.
www.e-assessmentassociation.com
eaa
The e-Assessment Association
Newcastle:What Happened?
•
•
•
•
•
•
Decided on i-assess, derived from CALM.
Developed initially at Heriot-Watt.
School policy developed** Report: Using computer
based assessment in first year mathematics and statistics degree
courses at Newcastle University:
http://mathstore.ac.uk/headocs/Foster_B.pdf
Student engagement high (90%
completion of in-course assignments).
Standards have been maintained.
Evidence that 1st year retention improved.
Formative e-Assessment Award Winner
(eAssessment Scotland 2010).
www.e-assessmentassociation.com
eaa
The e-Assessment Association
Newcastle: Summative
Assessments
•High volume, high stakes exams using
LMSs (Moodle, Blackboard etc). Secure
environments with secure databases.
•Created SCORM objects from i-assess
assessments.
•Multi-use across many platforms.
www.e-assessmentassociation.com
eaa
The e-Assessment Association
Newcastle: How?
SCORM objects
On DVDs
Distance learning
External World
via RAS
Clusters
www.e-assessmentassociation.com
i-assess on
separate
server
i-assess
on Univ
Server
University
via LAN
External World
via Web
SCORM
Moodle
Blackboard
eaa
The e-Assessment Association
Further Example of summative
success
•CALM/Mathwise formative from 1985 at HW
•1995 research to check on Reliability & Validity
• Impact: encouraged use of formative e-resources
• Practical: 500 undergraduates tested over week
– randomisation
– mathematical expressions
– steps
– provided immediate re-test.
www.e-assessmentassociation.com
eaa
The e-Assessment Association
QTI
• Benefits:
– no lock in to supplier
– question sharing potential
• Barriers:
–
–
–
–
–
reduced range & quality of question types
MCQ + variants too restrictive
lessons not passed on
not appropriate in STEM subjects
lack of adaptive modes
www.e-assessmentassociation.com
eaa
The e-Assessment Association
Barriers
•
•
•
•
•
Time to create good questions
High initial costs
Poor collaboration history
NIH
Digital divide
www.e-assessmentassociation.com
eaa
The e-Assessment Association
PASS-IT in Scotland
Collaborative project:
HW/LTS/BBC/SFEU/SQA
•Explored formative/summative e-testing
in post-16 education
• Pilots delivered: 6 levels in 1 subject
– Variety of levels in 5 other subjects
• Support candidates with additional needs
•Summary at www.passit.org.uk/project_materials.asp
www.e-assessmentassociation.com
eaa
The e-Assessment Association
Conclusions
• What works in Education? Make that the
standard.
• E-assessment can:
Enhance teaching, improve learning and measure
part of it.
• Formative: SCHOLAR (Scotland) from 1999
• Research: www.pass-it.org.uk, 2003-5
• Summative: e-NABs from 2006.
www.e-assessmentassociation.com