Leonardo da Vinci
Programme
GUIDE 6
Procedures and instruments for on-line
assessment
Project N° I/01/B/F/PP-120550
This is a product of the SOLE Project partnership, developed by Scienter (Bologna,
Italy): Michela Moretti.
English version provided by Scienter and checked by Cork College of Commerce
(Ireland).
SOLE Partnership
Promoter
ITSOS “Marie Curie” – ITALIA – Cernusco sul Naviglio, ITALY
GREECE
Lambrakis Research Foundation - Athens
FRANCE
CAFOC - Rennes
ITALY
ADECCO - Milano
CEP - Torino
CGIL - Formazione e ricerca - Roma
CISL – Studi e ricerche - Roma
CONFAPI - Roma
IMQ - Milano
ITCS “Mario Pagano” - Napoli
ITIS “Vito Volterra” - Ancona
SCIENTER - Bologna
UIL - Servizio Politiche del lavoro e FP - Roma
Università Roma3, Facoltà di Scienze della Formazione- Roma
IRELAND
Chambre of Commerce - Cork
College of Commerce - Cork
DEIS – Cork Institute of Technology - Cork
ROMANIA
Camera de Comert si Industrie - Galati
Consiliul National Al Intreprinderilor Private - Galati
Universitatea “Dunarea da Jos” - Galati
© The content of the SOLE Project - “Leonardo da Vinci” Programme, Contract
I/01/B/F/PP-120550 - can be freely used by Schools, Universities and Training
Institution provided that the SOLE logo is maintained and the source fully cited.
No part can be published for commercial use without formal permission of the
Project promoter.
INDEX
EXECUTIVE SUMMARY ......................................................................................... 1
1.
INTRODUCTION ............................................................................................... 2
1.1
1.2
1.3
1.4
2.
GUIDE AIMS ...................................................................................................................... 2
TARGET GROUPS............................................................................................................ 2
STRUCTURE ...................................................................................................................... 3
HOW TO USE IT ................................................................................................................ 3
THE EVALUATION, CERTIFICATION AND RECOGNITION
PROCESSES........................................................................................................ 4
2.1 THE EVALUATION PROCESS ........................................................................................ 4
2.2 THE CERTIFICATION PROCESS .................................................................................... 6
2.2.1 The European computer’s driving licence (ECDL) ................................................. 7
2.2.2 ECTS European Credit Transfer System ................................................................. 8
2.2.3 Transparency of vocational qualifications .............................................................. 9
2.3 THE RECOGNITION PROCESS..................................................................................... 10
3.
THE ASSESSMENT PROCESS ..................................................................... 11
3.1 THE OBJECTS OF THE EVALUATION........................................................................ 11
3.1.1 Prior and experiential knowledge/learning ........................................................... 13
3.1.2 The online path ...................................................................................................... 15
3.1.3 The learning achievements .................................................................................... 16
3.1.4 The competencies developed through a learning path .......................................... 19
4.
THE TOOLS ...................................................................................................... 22
4.1. SOFTWARE AVAILABLE ON THE MARKET ............................................................ 23
4.2. OBJECTIVE TESTS: SOME USEFUL AND PRACTICAL INSTRUCTIONS ............. 25
4.2.1 Steps to design an objectives test ........................................................................... 26
5.
CONCLUSION .................................................................................................. 31
BIBLIOGRAPHY ..................................................................................................... 32
GLOSSARY ............................................................................................................... 33
Sole Project – GUIDE 6
1
EXECUTIVE SUMMARY
On-line assessment and certification are topics of increasing interest and concern
amongst many sectors particularly those of education and industry, who recognise
that the increasing availability of online learning supply necessitates suitable control
in terms of quality and the availability of objective and standardised measures to
ensure cross sectoral certification and recognition of the results achieved by the
learners through an online course. This guide provides a background on the relevant
issues, available tools & software and presents a number of examples of assessment
software in use, together with an experimental framework that includes practical
indications for its use.
The critical processes, of On-Line assessment are Evaluation, Certification, &
Recognition.
The evaluation phase includes student assessment as well as material and course
evaluation and is a crucial element of the whole strategy. Diagnostic evaluation is a
prerequisite of both the Certification and Recognition processes, permitting
customisation of the learning paths based on student knowledge and prior experience.
Evaluation per se is not limited to the early stage of the overall process and
Formative, Summative and Impact evaluation elements allow suitable feedback
mechanisms operate to allow improvement and feedback throughout the life cycle of
the process. Effective evaluation is the key to ensuring validity & reliability of both
procedures and tools. Certification is dependent on the existence of homogeneous
methodologies and shared criteria and the aim of such a process should be to provide
transparency and visibility of both formal & informal qualifications. This in turn
allows recognition of all competencies and skills. A number of existing European
initiatives are described within the guide. Application of evaluation is dependent on
accurate and detailed identification of the objects to be evaluated or measured
throughout the learning process. These include, Prior Knowledge & Experience,
(formal, non-formal & informal), On-Line learning Paths, Learning Achievements
(both formative & summative) and Competencies (interim and end-stage) already
possessed by the learner or developed as a result of the programme or course. The
suitability and effectiveness of the learning tools must be matched to the needs of
course and user. Suitable criteria for tool selection are discussed and a selection of
currently available computer assessment software is described within the guide. The
guide also includes guidelines and practical indications to aid readers design and
develop suitable Objective Tests, which may be used to support effective evaluation.
Sole Project – GUIDE 6
2
1. INTRODUCTION
If you are interested in the design and delivery of online training you might face
problems or needs such as:
The need to understand the “issues” and “open problems” relevant to the certification
of online training;
The need to provide your final users with effective assessment procedures;
The need to provide clear instruction and indication relevant to the tools being
applied for the assessment of online training and learning.
In this GUIDE “Procedures and instruments for on-line training assessment” you
can find some answers.
As the title makes clear, this is neither a handbook nor a treatise. The main goal of its
publication is to create a useful tool to help the reader understand the topics related to
the certification and evaluation of online training and in addition provide an initial
framework for the implementation.
1.1
GUIDE AIMS
The main goals of this guide are:

to briefly outline “state of the art” issues relevant to online certification;

to introduce the elements applied in the evaluation of online training;
to clarify the rationale used to define the evaluation elements and to offer some
suggestions for their use;
to provide some guidance on available tools currently used for the assessment of
online training.

1.2
TARGET GROUPS
The main targets of the guide are:

Coordinators and managers of online training centres in the educational sector;

Trainers and tutors at vocational training centres and companies.
Even though not included in this list, anyone interested in the subject and eager to
understand how the certification process of online training can be run, may find
stimulating suggestions and information within the guide.
Sole Project – GUIDE 6
1.3
3
STRUCTURE
The guide is organised into six main parts:
Executive summary
1. Introduction
This is the part that you are reading now.
2. The evaluation, certification and recognition processes
Provides the readers with a general background regarding “the state of the art” of
online assessment and certification and the open problems to be faced.
3. The Assessment Process
In this part the objects of certification and instruments for on-line training
certification are presented.
4. The Tools
This part presents a description of possible tools for use in online training
Certification. A section is devoted to providing instruction for the design of an
objective test.
5. Conclusion
Bibliography
Presents the main reference books and web sites.
Glossary
Presents an explanation of the main terms used in this guide.
1.4
HOW TO USE IT
This guide can be approached at different levels. The different sections do not need to
be read in sequence; nevertheless a sequential approach allows the reader with a full
comprehension of the contents presented in the Guide.
Sole Project – GUIDE 6
4
2. THE EVALUATION, CERTIFICATION
AND RECOGNITION PROCESSES
The concept of online learning refers to a specific format of learning in which
students, teachers, tutors and technical staff, mostly “meet”, “learn/teach” and/or
“communicate” through technological links within a virtual environment.
In some courses, the certification represents the conclusion of the process
(articulated in analysis/measurement and assessment) since it translates into
formal titles the results of the evaluation.
Formative
Assessment and recognition of
previous knowledge and
competencies and
prerequisites
Certification
Definition of the
individual learning path
Assessment
Analysis of the
result
(measurement)
Delivery of the
related modules
and online
learning activities
evaluation (e.g.
at the end of
each module/
activitiy)
Summative
evaluation (Final
test/ exam)
()
Not all online courses (as well as not all classroom-based courses) once completed,
provide the possibility to obtain a formal certification in terms of an official
recognition of the learning/competence achieved. However, it must be said that the
majority of ODL and online courses supply the learner with an attendance certificate.
2.1
THE EVALUATION PROCESS
The use of the term “evaluation” within the body of this text collectively describes all
evaluation phases, including course/programme evaluation and assessment of student
competency or output.
The phase of evaluation or assessment within an ODL and/or online course is a key
element of the overall process and is a crucial aspect of the didactic strategy. It is
probably more important than in a traditional classroom based course, since both
training and learning as well as assessment are taking place within an environment
where exists both a space and time separation. In addition it must to be mentioned
that assessment is a fundamental aspect in order to sustain motivation during the
learning process. Within any training course and in particular within an ODL and/or
online courses the role of evaluation & assessment has different functions:
Sole Project – GUIDE 6
5
1. Diagnostic Evaluation as an initial assessment aims to:
 identify the characteristics of the learner (e.g. interest/motivation/
expectations, prevalent learning styles, and point of view on the course, etc):
 measure the possession of the necessary prerequisites for course attendance;
 identify and recognise previous knowledge or competencies already possessed
by the learner that facilitate the individualisation of the learning path within
an ODL course or allow the learner to apply directly for a final exam or
specific exam module.
2. Formative Evaluation (in-itinere) aims to:

Allow the learner to self-check his/her progress within the learning path,
and reflect on his/her strategy of learning;

Allow the tutor and the co-ordinator (in the case where the self- assessment
is accessible to them) to provide pedagogical and content support to the
learner.
3. Summative Evaluation:
This refers to an interim evaluation placed at the end of modules or in a specific
part of the course with the aim to:

allow the learner to control the achievement of the learning goals;

allow the tutor and the co-ordinator to collect data on the training actions
and to introduce corrective actions and improvements.
The summative evaluation when located at the end of the course facilitates:

Assessment of the different results (output and outcome) achieved by the
learner and collection of the data and the information necessary for the
final certification.

The tutor and the co-ordinator to collect data on the whole experience.
4. Impact Evaluation facilitates:

Verification of course impact in terms of the usefulness, applicability,
solidity, etc of the results achieved by the learner at a later date, post course
completion.
In this respect it has become clear that all the activities related to evaluation have to
be established carefully in order to be part of a “coherent” and “effective” evaluation
model. The word “coherent” used in the previous phrase underlines the fact that the
set of evaluation activities must take into consideration the objectives and the overall
didactic strategy adopted by the course, as well as, the psychological, organisational
and cultural characteristics of the learners. By “teaching strategy” is meant the
organisational model employed in order, to facilitate the users achieve the set
objectives, to stimulate interest and facilitate learning. In other words, if, for example,
the course is designed following a “blended model” (such as classroom sessions,
videoconference, online and practical activities) the tools for the evaluation have to
Sole Project – GUIDE 6
6
be logically selected and delivered at the right time, taking into consideration the
specificity of each single part of the course and the aims of same.
The word “effective” stresses the fact that a correct evaluation should assure:

the validity of the procedures and tools used - whether they are appropriate for
the required purpose;

the reliability of the procedures and tools - whether they will produce consistent
results, which give a true reflection of the knowledge and skills of the
candidates.
2.2
THE CERTIFICATION PROCESS
The issue of the certification of competencies or title opens the door to both
political and institutional issues, relating to the requirement that the certification
acquired is recognised in different sectors such as: educational, vocational or labour
market. To date some progress has been made in the different countries on the issue
of cross-sectoral certification.
The problem concerning assessment mostly arises from a lack of shared criteria that
could be used to establish homogeneous methodologies and tools. As in many cases
there does not exist a recognised reference framework that can guarantee learners the
recognition of their acquired skills and knowledge.
Through which criteria is assessment achieved? The perspective of certification of the
assessed results supposes an explicit and transparent recognition of the criteria
through which a certain judgement is given. This poses the problem of defining
criteria and standards of reference that could then be applied and used to transform
the data of an assessment in to a formal result, recognisable in different domains and
varied contexts. This is due to the fact that certification is an inherent property of the
public domain, requiring complete transparency, free from implicit, subjective, and
arbitrary influences.
The authority enabled to award a recognisable certification is usually a commission
or body (formed by experts and representatives from institutional bodies -e.g.
Ministry of Education -, Universities or other recognised institutions), which plays a
role external to the training process.
Where such certification is foreseen, mostly coming from super partes institutions
and bodies, its use is strictly enforced by the authority of the Certification body with
reference to a specific sector.
In cases where certification is delivered by the intervention of public institutions or
universities, the transferability of certification will reflect the model of the
institutional system of credit recognition and passage from one domain to another.
Sole Project – GUIDE 6
7
In many cases On Line Learning is considered, at juridical level, as a modality to
deliver training that follows the same rules of the traditional training certification. In
other words where a certification procedure is foreseen (formal recognition is given)
at the end of face to face courses or On Line Learning courses, the final exams in
either case must be taken “at the presence”.
Just to make an example, in the Emilia-Romagna Region (Italy), within the
Vocational training sector, On Line Learning is completely integrated as a “normal
modality” within the regional training framework. Although the Open and Distance
Learning courses are recognised as well as the traditional classroom based courses,
the Region doesn’t admit the final assessment and the related certification at distance:
the student has to go to the training centre to take the exam.
In the UK they have adopted a model of the “Assessment Centre/Awarding bodies”,
where the learners have to come to an external centre for assessment.
With the support of the Community Education, Training and Youth programmes, a
number of assessment and recognition instruments have been developed.
It is also worth mentioning and briefly outlining the following initiatives: the
European Computer Driver’s Licence (ECDL), the CEDEFOP forum for the
transparency of qualifications and the ECTS (European Credit Transfer Scheme).
2.2.1 The European computer’s driving licence (ECDL)
The ECDL is an internationally recognised qualification that enables people to
certify their competence in essential computer skills and knowledge. It is designed for
those who want to gain a basic qualification in computing and develop their IT skills,
and doesn’t require a prior knowledge of IT or computer skills. The ECDL
certification is managed and promoted by the ECDL Foundation. The ECDL/ICDL is
a standard definition and measure of computer skills, aimed at providing a balance
between employer and employee needs and subsequent benefits to both. To achieve
an ECDL or ICDL, the candidate must successfully pass one theoretical and six
practical tests. These tests can be attempted in any order and at any time and must be
successfully completed within a maximum duration of three years. Also for the
ECDL the exams have to be taken in one of the accredited Test Centres, though the
test is still in an online format.
Each successfully completed test is endorsed on an ECDL or ICDL Skills Card. Only
an accredited Test Centre can carry out testing and issue Skills Cards.
When tests for all 7 modules are successfully completed, the Test Centre endorses the
Skills Card, which is then sent to the National ECDL or ICDL Operator and a
personalised Certificate (Licence) is issued.
Sole Project – GUIDE 6
8
2.2.2 ECTS European Credit Transfer System
“Establishment of a system of credits - such as in the ECTS system - as a proper
means of promoting the most widespread student mobility. Credits could also be
acquired in non-higher education contexts, including lifelong learning, provided they
are recognised by receiving Universities concerned.”1
To help students make the most from their study abroad, the European Commission
has developed a European Credit Transfer System, which provides a way of
measuring and comparing learning achievements, and transferring them from one
institution to another. ECTS helps higher education institutions to enhance their cooperation with other institutions by improving access to information on foreign
curricula and providing common procedures for academic recognition. The ECTS
was initially established under the Erasmus programme (1988-1995). It has been
tested over a period of 6 years in a pilot scheme involving 145 higher education
institutions (“the so-called "inner circle" institutions”) in all EU Member States and
EEA countries, operating in five subject areas (Business Administration, Chemistry,
History, Mechanical Engineering and Medicine). The European Commission
developed the ECTS to provide a way of measuring and comparing learning
achievements and transferring them from one institution to another. This System
finds a collocation in the framework of promotion of study abroad of the EC,
guaranteeing the recognition of knowledge and skills acquired by studies abroad,
enabling access to regular courses for full participation in the academic life of the
host institution. This characteristic of ECTS distinguishes it from many other student
mobility programmes. ECTS enables further studies abroad. A student may prefer not
to go back to the home institution after the study period abroad, but rather to stay at
the host institution -- possibly to gain a degree -- or to move to a third institution. The
institutions themselves decide whether or not this is acceptable and what conditions
the student must fulfil in order to get a diploma or transfer registration.
ECTS credits are a value allocated to course units to describe the student workload
required to complete them. They reflect the quantity of work each course requires in
relation to the total quantity of work required to complete a full year of academic
study at the institution, that is, lectures, practical work, seminars, private work-- in
the laboratory, library or at home -- and examinations or other assessment activities.
In ECTS, 60 credits represent one year of study (in terms of workload); normally 30
credits are given for six months (a semester) and 20 credits for a term (a trimester).
ECTS credits are also allocated to practical placements and to thesis preparation
when these activities form part of the regular programme of study at both the home
and host institutions. ECTS credits are allocated to courses and are awarded to
1
THE EUROPEAN HIGHER EDUCATION AREA, Joint declaration of the European Ministers of Education
Convened in Bologna on the 19th of June 1999
http://www.unige.ch/cre/activities/Bologna%20Forum/Bologne1999/bologna%20declaration.htm
Sole Project – GUIDE 6
9
students who successfully complete those courses by passing the examinations or
other assessments.
2.2.3 Transparency of vocational qualifications
Transparency of vocational qualifications refers to the degree of visibility of
vocational qualifications acquired by individuals and its goal is to:
- support professionals of vocational education and training to improve the visibility
of national qualification systems;
- improve co-operation between Member States on issues such as exchange of
experience or comparability of curricula;
- help holders of Vocational Training Certificates explain their competencies to
potential employers;
- support the recognition of non-formally gained competencies (on-the-job, in daily
life, etc.);
- support sectoral or geographical mobility of workers;
- increase the visibility of new certificates issued by national training systems.
A European Forum on Transparency of Vocational Qualifications was established in
1998 as a joint venture of the European Commission and CEDEFOP (Promoting
mobility of individuals by making qualifications visible and by making learning
visible). Participation in the actions proposed are voluntary for each Member State.
The tasks of the Forum are:
- to stimulate dialogue on existing initiatives, instruments and institutions;
- to encourage the implementation of existing political solutions;
- to consider new initiatives.
Achievements of the forum:
- The “certificate supplement” gives a description of competencies acquired by
certificate holders;
- The “European curriculum vitae” gives a comprehensive standardised overview
of education as well as work experience;
- The “national reference points” are the national contact points giving access to
information on qualification systems. The national reference points are networked;
- The “recognition of non-formal learning” is another attempt to enhance the
visibility of skills and competencies on the labour market.
More information can be found at http://www.cedefop.eu.int/transparency/default.asp
In this framework, we can quote as an example the EuroCert project. This is a
Leonardo Project in collaboration with the above mentioned Forum to develop a
model for a certificate supplement.
EuroCert is an innovative solution to trans-national mobility and will enable qualified
people to find a job or an education/training placement in another member state.
Sole Project – GUIDE 6
10
EuroCert is a certificate supplement - i.e. a single page document to be issued on a
voluntary basis, by an awarding body, in addition to the certificate of the awarding
body and is accessible through the website. The multiplier project EuroCert promoting mobility through transparency of certificates (1998-2001) aims to transfer
and disseminate the certificate supplement and to promote its general use in the EU,
within a five-year period (http://www.eurocert.ie/ENG/transparency.htm).
2.3
THE RECOGNITION PROCESS
Certification provides visibility and recognition of achievements (in terms of
knowledge or competencies) acquired in training or in working situations in a larger
context. The certification can be seen as a kind of “label” of the achievements.
As it concerns ODL or online courses it is of fundamental importance to foresee
recognition of the credits at the end of every module, (single unit of competence). In
fact this modularization is one of the main aspects of ODL that permits the learner to
personalise his/her learning path. This aspect has to be considered in certification: the
learner should be interested not only in the whole training path, but also in deepening
their knowledge in a selected number of the proposed subjects. The certification of
competencies and skills acquired with every module will also be a further motivation
for the learner.
Sole Project – GUIDE 6
11
3. THE ASSESSMENT PROCESS
3.1
THE OBJECTS OF THE EVALUATION
When we speak about the “evaluation” the first question to answer is: what is the
object of the evaluation?
It is possible to assess:
 The prior and experiential knowledge
 The online training/learning path followed
 The learning achievements
 The competencies developed through training actions
Each object has a specific function and needs coherent methodology and tools to be
used.
When
Why
The prior and
At the beginning
For differentiating
experiential
of a training action the learning path
learning/knowledge
The online
training/learning
path followed
The intermediate
learning
achievements
At the end of the
course
During each
module
For allowing the
learner to get a
final certification
(in case s/he
already has all the
competencies or
knowledge
required)
For gaining an
attendance
certification
For self-evaluation
Who
By means of
(examples)
Coordinator,
tutor/guidance
counsellor


Initial test,
Previous titles or
certificates,
 Portfolio of
evidence
 Observation in the
workplace
As above
Tutor or
coordinator
Registration of the
selected modules
Learners


Objectives test,
Resolution of
cases study
Games
Simulations
As above
The learning
achievements
At the end of each
module
For accessing the
following module
Tutor/teachers



The learning
achievements
At the end of the
learning
experience/course
At the end of the
course
For gaining
certification
External
Commission
Official exams (tests,
interviews, portfolio)
For gaining
certification
External
Commission
Official exams (in some
cases also a portfolio of
evidence and project
The competencies
developed through
a training actions
Sole Project – GUIDE 6
12
work)
Sole Project – GUIDE 6
13
3.1.1 Prior and experiential knowledge/learning
ODL and online courses mostly foresee an initial test which is aimed at verifying that
the user possesses the necessary prerequisites for assessing the course and the results
from the test can be used to personalise the learning path. The individualisation of the
training path is achieved in an easier way if helped by a evaluation ex-ante, aiming at
establishing the competencies, interests and training needs of the learner.
The provision of an entrance text can be foreseen both in the face-to-face context and
at a distance, once the presence of the examinee can be certified.
The results of the process must allow for the establishment of:
 which didactic modules do not need to be part of a given learner’s learning path
(because the learner already has wide and deep competencies)
 which didactic modules need only be used in those parts of study which require an
in-depth knowledge of the subject (because the learner already has the
prerequisites and basic knowledge foreseen by the module)
 which didactic modules must constitute a fundamental part in the whole training
path (though the learner does not have any specific competence, s/he has the
useful prerequisites to deal efficiently with the module)
 which didactic modules must also include one or more didactic preparatory units
(because the learner lacks the fundamental prerequisites to be able to deal
efficiently with the module)
The initial assessment also has the goal of verifying whether the learner already
possesses the knowledge and the competencies necessary to pass the final exam.
Individuals can gain entry to the different levels of the programme through APFL
(Accreditation of Prior Formal Learning), APEL (Accreditation of Prior Experiential
Learning) or a combination of both with exact placement arrived at through
consultation with a nominated academic adviser. (for more details, please refer to
Guide 7 “Accreditation of prior learning”).
3.1.1.1
Prior Learning Assessment
The definition of formal, non-formal and informal learning from the Memorandum on
Lifelong Learning (European Commission, page 8) specifies:
“Formal learning takes place in education and training institutions, leading to
recognised diplomas and qualifications.
Non-formal learning takes place alongside the mainstream systems of education and
training and does not typically lead to formalised certificates. Non formal learning
may be provided in the workplace and through the activities of civil society
organisations and groups (such as in youth organisations, trades unions and political
Sole Project – GUIDE 6
14
parties). It can also be provided through organisations or services that have been set
up to complement formal systems (such as arts, music and sport classes or private
tutoring to prepare for examinations).
Informal learning is a natural accompaniment to everyday life. Unlike formal and
non formal learning, non formal learning is not necessarily intentional learning, and
so may well not be recognised even by individuals themselves as contributing to their
knowledge and skills.”
“Assessment of Prior Learning (APL) is a process that involves the identification,
documentation, assessment, and recognition of learning acquired through formal and
informal learning.2”
This may include work and life experience, training, independent study, volunteering,
travel, hobbies, and family experiences. The recognition of prior learning can be used
toward the requirements of
an academic
or training program,
occupational/professional certification or for employment/labour market entry
purposes.
Assessment of Prior Learning is based on the following assumptions:
- Significant learning occurs outside of the classroom.
- Adults acquire important knowledge, skills, and abilities through many means of
formal and informal study.
- This learning can and should be evaluated for credit by educational institutions
and in the workplace for hiring and promotion.
- Evaluation policies and procedures should be documented, clearly articulated, and
available to the public.
- Practices that force adults to repeat learning, which they have already mastered,
are inefficient, costly, and unnecessary.
3.1.1.2
Accreditation of prior (experiential) learning of individuals
The term ‘Accreditation of Prior Learning’ (APL) is used in connection with learning
that has been certified. The term ‘Accreditation of Prior Experiential Learning’
(APEL) is used in connection with informal uncertified learning through experience,
whether gained through work, voluntary activity or at home.
AP(E)L may be used by individuals as:
- a means of gaining entry to award programmes part way through the programme,
thus shortening the time required to complete a programme of study;
- as evidence to gain credits in individual modules.
2
This definition is taken from Canadian Association of Prior Learning Assessment: http://www.capla.ca
Sole Project – GUIDE 6
15
For this specific part please refer to the Guide 7.
3.1.2 The online path
In this case the object of the certification or better the object of the recognition is the
path followed: no judgement is given with regard to the actual learning results.
The result will be an attendance certificate, which specifies the modules taken by the
learners.
The assessment can be based on:
 The modules and selected path chosen
In this case the certification is based on the fact that the learner has chosen a
specific course or within a course some learning modules. The certification states
learning path followed or the number of modules selected, not the achievements in
terms of learning. It can be compared to the “certification of attendance” in faceto-face courses.
 Self-certification
The learner self-certifies his/her learning path. This is particularly relevant for
ODL courses where part of the learning is based on self-learning using off-line
resources. In this case in order to take into consideration the time spent by the
learners on offline resources (videotape, multimedia, paper based materials) the
system requires the learner to self-certify the time spent.
 The data collected from the tracking of the system
The system is able to track and record some data related to online activities, e.g.
the amount of time the learner has been connected to a specific web site, how
many e-mail messages have been sent, etc..
These data can be useful for the certification of the learner’s course attendance
More and more in an online environment the importance of participation in the
virtual community of learners (forum, chat, videoconference) is seen as a key
element in the evaluation process.
 Tutors and the Teachers
The tutors and the teachers and any other person who is in charge of supporting
the learners during the learning path will be important sources of information with
regard to the online path taken by an individual student.
Sole Project – GUIDE 6
16
3.1.3 The learning achievements
As mentioned before, the assessment of the learning achievement should follow the
life cycle of the training action.
This means both “during” and “at the end” of the learning path.
If the assessment is carried out during the learning path, it can have a dual formative
and summative purpose:
- formative (by allowing the learner to reflect on his/her progress) and/or
- summative (by determining whether the user has acquired the necessary
knowledge or skills for accessing another module in which the prerequisites are
the objectives of the previous one).
As previously stressed, the formative assessments are particularly useful in order to
sustain a coherent didactic path; this aspect is far more relevant if the training module
is developed within ODL or an online modality.
The assessment of the learning achievements can be based on:
 On line tests
The term ‘computer delivered assessment’ covers any system in which tests,
exercises or other forms of assessment are presented to the user by computer. In
some cases, the software marks the user’s responses, records the score and
provides feedback on screen or on paper. In other cases, the correction of the test
and the feedback is given directly by the tutor or the teacher who is in charge of
monitoring and supporting the learning process.
New communication technologies, such as the Internet, are providing new ways of
delivering assessment material. Computer-delivered tests can be taken on-line or
can be downloaded for use off-line.
Computer delivered assessment can assess knowledge related to almost any
subject. This is usually done through objective questions. Many computerdelivered tests use multiple-choice questions, in which each question is followed
by several possible answers (often four or five) and the candidate has to choose
which one is correct. Multiple-choice questions can test a wide range of
knowledge, but for some applications there are advantages in using one of the
other forms of objective questioning, for example, “multiple-response” (in which
more than one answer is correct), “matching”, ‘hot spot’ (in which the candidate
must click on the appropriate point in a diagram) or short answer (in which the
candidate types in the correct answer). The questions may include diagrams,
sound or video clips.
The short essay may be considered as a semi-structured assessment tool. In fact it
is a precise question that doesn’t envisage a pre defined answer, but allows the
Sole Project – GUIDE 6
17
examinee relative freedom to give his/her own answer, albeit within a precise
scheme or predefined format. In this way, the short essay allows verification of
comprehension and knowledge skills as well as those of analysis and evaluation.
In fact the question may inherently incorporate problems that require further
analysis, or considerations requiring further judgement in terms of coherence,
pertinence, etc.
The text named “short essay” consists of a question (that can be eventually
articulated in more specific questions) that requires a precise and essential answer
from the learner.
In fact, precise criteria of length and time are given to the examinee (for example:
no more than 12 lines in no more than 10 minutes); the answer could be
schematic, or developed through essential points. For training assessment texts,
the short essay can easily be delivered at distance as it does not involve any
particular technical problems or a high degree of technological complexity. The
learner must be informed about the modalities of answer and the limits of space
and time. The question must be formulated in a simple, clear and unambiguous
language. After writing the answer, the user must have an automatic procedure
through which they can send the formatted response to the teacher.
Testing of knowledge is not limited to simple recall of fact. Questions can test
understanding and can be based on scenarios, thus testing the learners ability to
use knowledge in realistic situations.
A range of skills relating to understanding of language (the user’s own language
or a foreign language) and mathematical skills can be tested. This is usually done
by the use of objective questions.
Computer delivered assessment is not usually a suitable means of assessing
manual skills, but there are exceptions. Computer delivered assessment can test a
range of skills in computer use. The simplest software packages assess the
candidate’s speed and accuracy in typing. More sophisticated software packages
test the accuracy of typing from more complicated copy and the ability to use the
features of application software.
Computer programmes have also been used to test reaction times. Linking the
computer to a hardware kit allows further manual skills to be assessed.
Other computer delivered assessment uses simulations, for example in faultfinding situations or in the control of industrial processes. In simulations, the
computer programme reacts to the responses given by the candidate. For example,
to test fault finding skills, the candidate chooses which of several investigations to
perform first, the computer responds by reporting what the result of that
investigation are and offering a choice of subsequent actions to which the
candidate again responds with his or her choice, until the fault is traced. The
programme records the sequence of actions taken.
Computer delivered assessment may be used for testing aptitude and intellectual
skills, for example as ‘intelligence tests’, and also for the assessment of
personality. Tests of this type are often computerised versions of well established
Sole Project – GUIDE 6
18
paper and pencil tests, developed by Chartered Psychologists and validated by
large-scale field testing.
Some software uses adaptive testing techniques to vary the selection of questions
according to the user’s response. For example, in a test of intellectual skills
intended for users with a wide range of intellectual ability, the user’s score on the
first group of questions would determine the level of difficulty of the subsequent
questions presented. This ensures that the test is applied at the correct level of
difficulty for the user.
 Resolution of structured case studies
The case study is a predefined example, in which subject content can be linked or
matched, within the context of the learner or to a specific situation related to a
specific course topic. The learner is requested to comment on the case or to
answer the questions which are related to the case study solution.
 Games and simulation
Games may be classified and sub-classified according to many different criteria.
Business games simulate business process and business situations. Business
games can be classified by pedagogical purpose, number of decisions, types of
decisions, number of companies involved in the game, number of products, the
amount of computerization, amount of interactivity, time frame for decision and
more. Games and simulation can be valuable tools for practicing and checking
different skills.
 Participation to synchronous activities
In the case where the course envisages synchronous activities such as videoconferencing or chatting, the certification can also be based on the assessment of
the learner’s participation in the synchronous events. In this case a specific grid
for analysing the learner’s performance/action during the event should be used, in
order to guarantee a comparable assessment between learners.
 Participation to asynchronous activities
In a case where the course has foreseen the need for collaborative work in which a
group of learners have to reach some joint objectives, the certification can be
based on the contribution of the learner to the achievement of the final results.
Unfortunately, even though the share and the construction of knowledge is
considered to represent one of the higher levels of the collaborative learning
approach, data related to the learner’s role in the virtual class still receives little
recognition in the evaluation process. In other words the final result of a test is
recognised as being more important in the evaluation process than the process
(and the contribution of each learner), which has lead to that result.
Sole Project – GUIDE 6
19
3.1.4 The competencies developed through a learning path
Competence can be defined as “the capacity of using and combining knowledge and
skills in order to master a working situation and obtain the achieved results”.
A competence is not only formed by knowledge and skills but also by attitudes and
behaviours (adaptability, reactivity, group work capacity, learn to learn,
communication) which allow the person to apply knowledge within a working
situation.
Competences are “context based” (a person which is competent in one specific
situation will not necessarily be competent in another context) and are constantly
evolving.
Competence is always expressed through observable behaviour in a
workplace/context and is closely linked to a particular activity or specific context (in
other words, specific professional situations).
Nevertheless, many definitions of “competence” have recently been put forward by
different authors and researchers. Almost all the definitions agree on the fact that
competence has two levels:
- the first one concerns the individual : what the individual has learned, is able to
do, the level of his/her motivation and is independent of a specific context.
-
the second includes consideration of the firm/enterprise or, rather, the context in
which the individual works or acts.
Therefore it can be possible to distinguish between:
- “Context based competence” in other words competence observable within a
working context identified through the analysis of working processes;
-
“Competence as the result of a learning process” which is independent from any
context.
Context based competence
Competence as result of a
learning process
If a competence is formed by knowledge, skills, attitude that are the results of a
learning process, these three main aspects of each competence can be expressed in
terms of learning objectives to be achieved, which can be articulated in:
Sole Project – GUIDE 6
-
20
Cognitive objectives (the comprehension and organisation of the main concepts
which are involved in the different situations in which the competence may take
place );
- Operational objectives (the ability/skills used in order to face different situations);
-
Attitudinal objectives (such as beliefs, attitudes, motivation, feelings which give
meanings to the different situations).
The assessment of the competencies developed through a learning path can be based
on:
 Portfolios of Evidence
The PORTFOLIO can be defined as the container/file/folder which contains any
element/evidence which demonstrates the level reached by the trainee in
possession of a certain skill/ability.
The portfolio can also assume value as a motivational force and aid building of
self-esteem because it contains tangible proof of the learner’s skills and abilities to
correctly and effectively perform certain activities.
Some examples of evidence that could be included in the portfolio are:

References from employers or managers (current or previous)

Curriculum vitae

Certificates from training courses attended in the past

Personal development plan

Examples taken from work activities: final or intermediate reports,
letters/communications, various documents (transparencies, presentations,
work sheets)

Reports from project work

Videotapes

Analysis and discussion of case studies developed at a training center

Documentation of activities performed outside the workplace
 Case studies
The case study is a predefined example, in which subject content is directly
related and relevant to the context in which the learner finds themselves. The
learner is requested to comment on or to answer questions that are related to the
case studies.
 Project work
The project work should include relevant professional situations and provide
experience that incorporates difficulties, changes in responsibilities and
Sole Project – GUIDE 6
21
challenges. In the lifelong learning initiative, project work has become an essential
part of the learning action. The project work itself is the object of evaluation (in
particular the way in which the learner has conducted the analysis and diagnosis of
situation, behaviors adopted, and results obtained).
 Observation
Observation is based on evidence of proof, i.e. on the verification of the real
behaviour of learner. In this case the observer could have at his/her disposal an
observation grid.
If the ethnographical approach is adopted, the observer will survey the learner
performance over extended periods of time and only at a secondary stage will the
observer attempt its systemization into conceptual categories.
Sole Project – GUIDE 6
22
4. THE TOOLS
As mentioned before, the term ‘computer delivered assessment’ covers any system in
which tests, exercises, or other forms of assessment are presented to the user by a
computer.
Software for computer delivered assessment may be purchased ‘off the shelf’ or may
be developed to your own specification. In addition ‘question setting shell systems’
allow you to write your own test questions on any topic; the software providing a
mechanism for storing and retrieving the questions, delivering them on computer and
reporting results.
It is important to underline that before choosing any assessment software, the objects
of the evaluation (what has to be assessed) have to be defined correctly and precisely.
Only after this clarification, it is possible to make a choice of the tool which better
will fit or which better will support the specific evaluation purposes.
One issue which should be brought to attention is related to the “per sè” quality of the
software (in other words who assesses the assessment software). In the majority of
cases it is the producer (e.g. background and knowledge in the field, statistics on the
usage, references and assessment done by experts) who guarantees the quality of the
products realised.
In addition, a clear specification of the objectives of assessment must precede the
delivery: the person must know that the results of the proof will meaningfully
contribute to the structure of his/her personalised learning path.
The modalities to fill in the text and the time available to answer to each part will
then be communicated to the examinee, with simple and clear language. The
examinee can choose, when to finish parts of the modules or suspend the text and
when to continue it in a following session.
Specific on-line Help must be foreseen to help the examinee to solve technical
problems. If the text is at a distance, direct access to the tutor should be possible, in
order to clarify technical and organisational matters. When the text is completed,
special modalities for the automated delivery of the text to the referent teacher must
be foreseen.
As mentioned before, some software packages are able to automatically mark the
user’s responses and give a score. Many learning theorists agree on the value of
mistakes in learning (or “learning by failure”) and on the importance of helping the
learners in reflecting on the process and on the reasons which have brought him/her
to answer wrongly. In this case the key role of the tutor and/or the teachers in
integrating the software performance with a more didactic and valuable feedback
becomes clear.
Sole Project – GUIDE 6
23
4.1. SOFTWARE AVAILABLE ON THE MARKET
There are many kinds of software relevant to computer assisted assessment available
on the market.
The offered software products include:
1.
Vendor Certification tests
These are tests of knowledge and/or skill related to a specific product. They are
usually managed by the producer of the product and intended to ensure that the
sales force and maintenance engineers have a sufficient level of knowledge of
the product to provide good service to the customers. Vendor Certification tests
usually relate to IT hardware or software. Examples are Novell Certified
Network Engineers and Microsoft Certified System Engineers.
2.
Other computer delivered tests or assessments of skills or knowledge
Software in this category contains assessment material which is specific to a
subject or occupation. The individual takes the test at the computer and usually
the results are marked and reported by the computer.
Many tests in this category consist of multiple-choice or other objective
questions, testing specific knowledge, for example in Engineering or Consumer
Law. Other software tests skills, for example in typing or keyboarding or in
using a spreadsheet. Some software tests knowledge or skill through simulation
(for example to test fault-finding skills); simulations may be designed to be
observed by an assessor.
3.
Computer delivered tests or assessments of aptitude or personality
Software in this category allows individuals to take aptitude or personality tests
at the computer, saving time in test marking and reporting. Frequently the tests
are computerised versions of well-established paper-based tests.
Tests/assessments of this type are frequently used in personnel selection and
career guidance.
Like the paper-based versions, many tests of this type are sold only to people
who have been trained to use the tests and interpret the results. Costs for
software of this type are often based on the number of tests taken.
4.
Computer delivered tests of learning style
This software assesses the preferred learning style of the individual and is used
in guidance related to careers or courses. A 'learning style' typically states
whether the individual prefers learning by doing or learning by reflecting on
things. There are a number of different systems for categorising learning styles;
two of the better known are those of Honey and Mumford and of Kolb.
Sole Project – GUIDE 6
24
5.
Question setting 'shell' software
These software products are intended for users designing their own test content.
They are designed to enable the user to store tests or banks of assessment
questions on computer, to select tests or compile tests from the question banks
and to deliver the tests either on computer (on-line or off-line) or in printed
form. Most support a range of question types, with computer marking of the
candidates’ responses. Some, but not all, include facilities for statistical item
(question) analysis and results reporting. Products in this category may be
intended for national awarding body and/or school, college or employer use.
6.
Statistical analysis
Many software products in this category are intended to analyse the marks
obtained on tests and the responses to individual questions (especially objective
questions) and to produce analyses by test and by question. They may be
combined in a single package with question setting shell systems.
Other software in this category is designed to process the results of tests of
personality or aptitude (taken either at the computer or with paper and pencil), to
undertake the complex calculations which may be associated with such tests and
to produce reports on the individual.
7.
Competence recording software
In the UK this category includes NVQ/SVQ tracking software. Software
products in the competence recording category are mainly intended for use at
local level, in assessment centres, schools, colleges, training providers and
companies. They record the results of assessments marked by a human assessor,
usually not at the computer and often in the workplace. The software is
intended to record each individual's competence (under different headings) and
his/her progress towards the desired competence profile or target qualification.
It assists the assessor, tutor or training manager to track the individual's progress
and to identify what still needs to be achieved. Design of competence recording
software depends upon its purpose. If it is intended to match the individual's
competence against the profile required for a job, then it will allow the user to
enter the required competence definitions and/or to select from a 'bank' of
competence definitions. If the purpose is to monitor progress towards a
qualification (eg a National Vocational Qualification) then it will be structured
in the same way as the qualification (with units, elements, performance criteria
and range for an NVQ) and may be supplied with the wording of the target
qualification(s).
Software in this category may also enable personnel or training staff to keep
records of training, training needs and the results of assessments undertaken at
the end of training It may form part of a larger personnel or HRD record
system.
Sole Project – GUIDE 6
25
8.
Accreditation of prior learning (APL) and portfolio-building
Software in this category is designed to enable the individual (usually with the
help of a tutor or adviser) to identify his or her existing knowledge, skills and
competence, as well as (in some software) his or her interests and aptitudes, and
to compile a portfolio of supporting evidence. The purpose may be to claim
credit towards a qualification (this is especially relevant to National Vocational
Qualifications and Scottish Vocational Qualifications in the UK), to support an
application for entry to a course of study or for the purposes of career planning
and guidance (this is particularly relevant to the bilan de compétence in France).
The software keeps a record of the individual's skills, knowledge, competence,
aptitudes, interests etc and (in the case of NVQs and SVQs) of the achievement
of the units and elements of the qualification. APL software may include
guidance on portfolio preparation and may be linked to software to provide
careers guidance.
9.
Awarding body administration
Software of this type is intended for awarding bodies and examining bodies
which are responsible for the award of qualifications and certificates, usually for
candidates outside their own organisation. The software provides facilities for
handling examination/assessment entries, processing results and printing
certificates.
10. Assessor and interviewer software
Some software packages in this category are intended to provide help or training
to assessors of National Vocational Qualifications in the United Kingdom. They
may be specific to one or a few NVQs. Other software in this category is
intended to guide interviewers to ask appropriate questions, sometimes building
on the results of a computer-delivered test.
11. Software to record the results of the assessment (including competence
assessment and appraisal) and to track the progress of students and trainees.
4.2. OBJECTIVE
TESTS:
SOME
PRACTICAL INSTRUCTIONS
USEFUL
AND
As far as the type of learning that it is possible to assess through a structured text is
concerned, it is useful to underline that this depends on how the text is built (what
kind of stimulation is given and what alternative answers are proposed). The simpler
example is that of “knowledge text” and, “comprehension”: in addition, there is the
possibility to also verify higher skills of application and analysis, by using questions
Sole Project – GUIDE 6
26
concerning problem solving or case analysis or texts with multiple pre-structured
answers.
In the table below the advantages of using objective tests are presented:
Phase and aspect to measure


Advantages
Stimulus:
 Representativeness
Is very high since it is possible in a fixed time
to submit a high number of questions
 Homogeneity
Is complete and the stimulus is the same for
all the learners

Can be very low if tests are designed by
following the rules
Ambiguity and embedded
solution clues
 Delivery of the stimulus
Many mistakes can be eliminated following
clear and predefined rules. As mentioned
before if the test is delivered at a distance
support has to be foreseen in order to
overcome possible misunderstanding (e.g. in
reading the instructions) and technical
problems.
 Recording
If the system works properly the test will be
recorded automatically.
 Reading of the result
It can be very precise.
4.2.1 Steps to design an objectives test
In order to correctly design an objective test it is necessary to clarify the answers to
the following questions:
1.
2.
3.
4.
5.
6.
7.
Who is the target group of the test?(Who has to be assessed)?
Why it has to be assessed? What is the aim of the test?
What has to be assessed and how? What are the assessment objects?
What is the relationship between the test and the other possible assessment
tools?
In what way will it be delivered? How long will the tests last?
How will the results be used?
Who will be interested in the results? Who will make use of the results?
Sole Project – GUIDE 6
4.2.1.1
27
Who is the target group of the test?(Who has to be assessed)?
The target groups of a test can be different, for example:
- Students, workers (considered as general categories);
- The students and the workers already attending specific training centres (their
characteristics are known and already defined);
- Candidates for work placement.
- Professionals in specific sectors seeking formal certification.
Being clear about the target group/s of the test (and their characteristics in the terms
of age, training, professional, cultural and organisational background) helps to
define which characteristics the test should have and which aspects have to be taken
into consideration by the designer.
The definition of the target group will provide information on the real availability of
the users and provide suggestions in selecting the best tools.
N.B.
Since the possible answers to the following questions will be different taking into consideration
the above-mentioned typologies of target group we have decided to consider our target users to be
students of an online course. This means that the suggestions provided in the following
paragraphs will refer to this specific typology...
4.2.1.2
Why it has to be assessed? What is the aim of the test?
The answers to these questions help to identify the main function/role of the
assessment.
Within a training course the test can be used as a means for an initial diagnosis, for
measuring the achievement of intermediate results, or for assessing achievement of
the final objectives.
In the case of initial assessment, the test should be able to define the possession of
prerequisites necessary to face a new training course and help users to define the
most suitable and motivating path. For this reason the initial tests are extremely
important.
The evaluation activities, which are foreseen during the course, are aimed at
measuring the achievement of the intermediate objectives following the sequence
anticipated within the course. This test typology is also essential at the end of
reinforcing activities. Therefore it is important to diversify the types of questions
(closed questions, planning a correct strategy starting from a set of sentences,
Sole Project – GUIDE 6
28
identification of mistakes) and if an efficient support system is available, it could be
useful to use open questions, in order to make comments on the answers given.
In particular in ODL courses the self-assessment test plays a fundamental role: it
allows the user to verify if the intermediate objectives are achieved. Since the
learners autonomously carry out these tests, the relevance of the answers given
becomes immediately clear and suitable feedback provided for the continuation of the
learning path.
For the final tests it is be sufficient to conduct less analytical measurement (since the
intermediate objectives should have been already achieved and checked) and in any
case the final objectives should be suitably relevant and numerous to cover the final
tests.
4.2.1.3
What has to be assessed and how? What are the assessment objects?
The objective tests will only measure certain particular abilities in a precise and
correct manner.
The most important aspect is to combine/associate in a coherent way the typologies
of tests with the objectives and abilities that have to be measured and assessed. It is
also important to define different levels of objective.
If there are objectives which can be considered more important, it appears evident
that the focus of the test should be on measuring them. The obvious procedure for
doing this is to attribute to them a larger number of questions.
The ensure that the knowledge and ability being measured by the test is
representative, the questions should be designed in the same way as it is envisaged
that the learners will gain and master them.
In other word, the test designer should be able to draw up a set of questions which are
a “representative sample” of the topics and problems which are the objectives of the
learning path.
The tests should be consistent with the contents and the objectives pursued, should be
clear, and should be able to support motivation and interest. Finally, they should be
introduced with suitable explanation of the acceptability criteria being adopted and by
some indication to facilitate the programming of the learning path.
A simple procedure is to design a table such as the following one: the table is defined
in order to plan the object of the measurement. In this case in the cross-areas we place
the stimulus (or each observation object) which corresponds to the learning
objectives.
Ability/skill levels
Content
Content
Content
Content
Content
Sole Project – GUIDE 6
4.2.1.4
29
What is the relationship between the test and the other possible
assessment tools?
In planning and defining the tests and tools it will be necessary to analyse and take
into consideration the specificity of each test typology. Often, in poor learning
materials self-evaluation tests are introduced as an appendix to the text and have a
very simple structure (yes-no, true-false), simply assessing the memorisation of
concepts or data.
Contrary to this approach, it is actually necessary to pay particular attention during
the design and elaboration of tests that they first of all, should match the objectives.
If a different typology of tools is foreseen (not only objective tests) it will be
necessary to plan in advance the relationship and the correct timing for delivery of
the full set of evaluation tools.
4.2.1.5
In which way will it be delivered? How long will the test last?
In administering tests, all the instruction and support should be given to the learner in
order to let him/her focus on the real object of the tests (measuring his/her learning)
and not waste time on other external issues and problems.
The time for completing a test is an element to take into consideration. This is
particularly true if for a particular test the available time is limited and predefined. If,
for example, the final exam for obtaining a formal certification is conducted within a
predetermined time frame, the intermediate tests and the rehearsals/trials of the final
tests should be carried out having the same time frame so as to help the learner to
practice and to make the best use of the time at their disposal.
The time for an objective test can be determined by external reason (as above) or by
the characteristics of the learners themselves (.e.g. resistance in carrying out this
task).
For self-assessment tests the time is never as important an issue, because the learner
does it when he/she feels ready, taking the time considered necessary. Nevertheless
an indication of the average amount of time necessary to conduct the test should be
given in order to put the learner in the position to plan the test session in a better way.
In defining the average time for conducting a test, the number of questions and the
test typology will make a difference. Some tests, such as true-false, can be completed
quite quickly, while short essays require more time to answer.
4.2.1.6
How the results will be used?
The result can be used and presented in different ways taking into consideration the
main function/aim of the test.
Sole Project – GUIDE 6
30
As an example, the results of the initial test are used, to check if the prerequisites are
posessed by the learner, for motivating him/her, for helping him/her to reflect on
his/her way to learn and in their understanding of those pre-concepts which can
prevent successful learning.
4.2.1.7
Who will be interested in the results? Who will make use of the
results?
Being clear about who will be interested in the results means, that on the one hand, to
clearly define the functions of the assessment and on the other foresee in advance
which mode will be chosen in order to present the results and what kind of feedback
needs to be given.
Sole Project – GUIDE 6
31
5. CONCLUSION
Matters related to procedures and instruments for online assessment both are very
complex and multifaceted.
Current trends show that the support of policy development, and effective
implementation of measurers, procedures and tools within the educational arena and
labour market requires the involvement of different actors at many levels.
This assures the availability of quality criteria and procedures for conducting valid
and reliable online assessment providing results which are readily recognisable
throughout the different sectors.
Many software for computer assisted assessment are available on the market (this is
one of the growing area of the eLearning market) nevertheless the main choices have
to be made at pedagogical and didactic levels in order to select the appropriate
methods and tools for the online assessment.
Sole Project – GUIDE 6
32
BIBLIOGRAPHY
G. Domenici “Manuale della valutazione scolastica”, 1993, Editori Laterza
M. Gattullo “Didattica e Docimologia - misurazione e valutazione nella
scuola”, 1985, VIII° ristampa, Armando Editore
Guildford Educational Services e Question Mark Computing “Better Testing a
computerised guide to writing good quality objectives tests”, 1994, Guildford
Educational Services Ltd
Vertecchi, “Valutazione formativa “, Loescher, Torino , 1976
Vertecchi “Manuale della valutazione. Analisi degli apprendimenti, Editori
Riuniti, Roma 1984
ML. Giovannini, “Misurazione e valutazione dell’apprendimento” in Quaderni
dei Nuova Formazione, Bologna, Cooperativa Nuova Formazione, 1984
M Gattullo e M.L. Giovannini “Misurare e valutare l’apprendimento nella
scuola media”, 1989, Edizioni Scolastiche, Bruno Mondatori
http://www.cedefop.gr/transparency/refpoint.asp
http://www.ecdl.com/
http://europa.eu.int/comm/education/socrates/ects.html
Sole Project – GUIDE 6
33
GLOSSARY
Accreditation of
Prior Experiential
Learning (APEL)
It is used in connection with informal uncertified learning
through experience, whether gained through work, voluntary
activity or at home.
Assessment
In the UK based context, the term refers to student results
assessment. In most American literature the terms
“Evaluation” and “Assessment” are used interchangeably.
Diagnostic
Assessment
Initial assessment of students knowledge, skills,
characteristics.
Evaluation
In the UK based context, the term refers to programme and
course evaluation. In most American literature the terms
“Evaluation” and “Assessment” are used interchangeably.
Formative
Assessment
Ongoing evaluation with the aim to improve learning
process and to help the learner to gain a sense of
achievement and to check the ongoing progress.
Impact Evaluation
Verification of course impact in terms of the usefulness,
applicability, solidity, etc of the results achieved by the
learner at a later date, post course completion.
Online learning
Specific format of learning in which students, teachers,
tutors and technical staff, mostly “meet”, “learn/teach”
and/or “communicate” through technological links within a
virtual environment.
Prior Learning
Assessment (PLA)
It is a process that involves the identification,
documentation, assessment, and recognition of learning
acquired through formal and informal learning.
Reliability of an
assessment
The procedures and tools are able to produce consistent
results.
Summative
Evaluation
At the end of a module or a path with the aim to check and
measure if (and to what extent) the learning objectives have
been reached.
Validity of an
The procedures and tools used in an assessment/evaluation
Sole Project – GUIDE 6
assessment
34
are appropriate for the required purpose (if they measures
what has to be measured).