Modeling Units of Assessment for Sharing
Assessment Process Information:
towards an Assessment Process Specification
Yongwu Miao, Peter Sloep, and Rob Koper
Educational Technology Expertise Centre
Open University of The Netherlands
ICWL 2008,
20-22 August 2008, Jinhua, Zhejiang, China
Overview
•
•
•
•
•
•
•
Where are we from: OUNL
What we have done: EML => IMS LD
What is missing in IMS LD: Assessment
What is our objective: A New Specification
What is the current status: Conceptual Model
How can it be used: Olympic Games
Summary
Where are we from: OUNL
Where are we from: OUNL
The Open University of the Netherlands (OUNL)
– Started in 1984; national institute;
– Open distance education provider
• 7 faculties, 24000 students
• 24 study centres in Netherlands and Belgium
• education delivered using a variety of
technologies (print, cd-rom, internet, face to face
contact sessions, practical rooms, etc.)
– R&D: Educational Technology Expertise Center
What we have done: EML => IMS LD
• Educational Modelling Language (EML)
• IMS Learning Design (LD): An e-learning technical specification
IMS LD is derived from EML
IMS Global Learning Consortium, Inc.
OUNL is contributing member (voting rights) of IMS
– Also Apple, Blackboard, Microsoft, Cisco, Sun, Texas Instruments,
…
1998
2003
* Information model
* Best practice guide
* XML binding
What we have done: EML => IMS LD
Basic idea:
Develop a ‘language’ to describe the learning/teaching
processes with associated content in a formal, semantic
way, so that it can be interpreted by computers.
Compare: music notation
This description should be independent of any delivery
platform => can be deployed in any platform that provides
the adequate functionality
What we have done: EML => IMS LD
Which
notations
are
needed
to
describe
a
lesson
plan?
Learning object:
- Book,
- Audio & video
What we have done: EML => IMS LD
IMS LD, an education-specific process modelling language,
can be used to describe a wide variety of pedagogical
strategies or approaches in terms of a play including acts
and role-parts (based on metaphor of theatrical play)
•People act in different roles
•working towards certain objectives
•by performing learning and/or support activities
•within an environment, consisting of learning objects and
services used in the performance of the activities.
Moving focus from what to learn to how to learn:
Lesson plan or learn-flow
What we have done: EML => IMS LD
method
play
Act 1
Act 2
Act 3
Act 4
Role-part 1
Role-part 2
Role-part 3
Role-part …
Role
components
Activity
Environment
ActivityDescription
Learning objects
Learning services
Act 5
What we have done: EML => IMS LD
Design-time
the events that can be
observed or expected
actual or imaged pedagogical processes
LD authoring tool
author
represented in LD
(in the form of XML)
LD-compatible
run-time environment
a formal pedagogical model
learner,
teacher,
and other stakeholder
Run-time
multiple executions of the same model
carried out by different groups
What is missing in IMS LD: Assessment
LD can not explicitly support various types of assessment.
Assessment components within the Educational Modeling
Language were excluded when LD was adopted by IMS,
because of the existence of IMS Question and Test
Interoperability (IMS QTI).
QTI is also an open e-learning technical standard, which
describes a data model for the representation of assessment
item/test and their results. It defines a set of interaction types
which can be used to specify basic question types (e.g.,
multiple-choice, fill_in_the_blank, and slider) and complicated
question types through combination.
What is our objective: A New Specification
Develop a high-level assessment process modeling
language for facilitating both experts and practitioners to
share assessment process information by extending QTI
and LD with an additional layer.
1.
facilitating representation, understanding, communication,
comparison, and reuse of a variety of assessment practices at
design-time;
2.
guiding and scaffolding learner, tutors, and other stakeholders
to perform a variety of activities at run-time by providing
guidance and awareness information, by configuring a
workspace for carrying out prescriptive tasks, by controlling
and changing the sequence of activities based on the execution
state and circumstantial information, and by orchestrating the
efforts made by different roles/participants.
What is the current status:
Conceptual Model
Developed a conceptual model, the core of the
specification, which can be represented as:
• Semantic aggregation model
• Conceptual structure model
• Process structure model
The first validation study has been conducted through
investigating whether the conceptual model of specification
meets the requirements of completeness, flexibility,
adaptability, and compatibility.
What is the current status
Semantic aggregation model:
What is the current status
Conceptual structure model:
What is the current status
Process structure model
What is the current status
Restrictions between stages and activities:
How can it be used: in Olympic Games
It is not necessary to develop a software
applications for supporting each assessment
How it can be used: in Olympic Games
…
Athlete
perform1
Rater 1
assessmt.
form
rate 1.1
rubric
values 1.1
Rater 2
Rater n
assessmt.
form
rate 2.1
system
assessmt.
form
…
rubric
values 2.1
rate n.1
rubric
values n.1
decision
rules
calculate1
publish and view result1
score1
…
perform5
assessmt.
form
rate 1.5
rubric
values 1.5
assessmt.
form
rate 2.5
rubric
values 2.5
assessmt.
form
…
rate n.5
rubric
values n.5
collect and calculate5
publish and view result5
score5 and total
How it can be used: in Olympic Games
Player.1
Player.2
…
Player.n
internet
Dispatcher/Service broker
A
A
IMS LD engine
IMS QTI engine
an assessment model
multiple instances
UoL management Assessment
and enactment service
service
server
Use existing Run-time Environment
How can it be used: in Olympic Games
An assessment model can be reused and
modified to support other assessment
Summary
IMS LD can support interoperability and reusability of a
variety of pedagogical models and systems.
IMS LD provides insufficient expressiveness to model
integrated learning, teaching and assessment processes.
A high-level assessment process modeling language can
be developed by extending IMS QTI and IMS LD with an
additional layer.
The conceptual model of the specification has been
developed and validated.
Thanks for your attention
Questions?