Linking Learning
Outcomes,
Outcomes,
Teaching
g and
Learning
Activities and
Assessment
Dr Declan Kennedy,
Department of Education,
University College,
Cork.
1
In this talk:
Bologna Process.
European Credit Transfer and Accumulation
System (ECTS),
Framework of Qualifications.
Modularisation
Bologna Process and Learning Outcomes.
Linking Learning Outcomes,
Outcomes Teaching and
Learning Activities and Assessment
2
The Background – Bologna
Bologna Agreement signed in Bologna, Italy in 1999 by 29
countries A total of 45 countries have now signed up to this
countries.
agreement.
The overall aim of the Bologna Agreement is to improve the
efficiency and effectiveness of higher education in Europe in terms
off academic
d i standards
t d d off d
degrees and
d quality
lit assurance standards.
t d d
One of the main features of this process is the need to improve the
traditional ways of describing qualifications and qualification
structures.
B l
Bologna,
It
Italy
l (1999)
3
What countries have signed the Bologna Agreement?
European Union - all 27 countries
Austria
Belgium
Bulgaria
Cyprus
Czech Republic
Denmark
Estonia
Finland
France
Germany
Greece
Hungary
Ireland
Italy
Latvia
Lithuania
Luxembourg
Malta
Netherlands
Poland
Portugal
Romania
Slovakia
Slovenia
Spain
Sweden
United Kingdom
Non-European Union
Albania
Andorra
Armenia
Azerbaijan
Bosnia and Herzegovina
Croatia
Georgia
Holy See
Iceland
Montenegro
Moldova
Norway
Macedonia
Russia
Serbia
Switzerland
Turkey
Ukraine
4
What is the Bologna Process all about?
Setting up of European Higher Education Area (EHEA) to
ensure the increased international competitiveness of the
European
p
system
y
of higher
g
education.
The Bologna Process is not based on a European Union
initiative. The agreement is between both EU and nonnon-EU
countries.
countries
Setting up of system to make it easier to understand the
description of qualifications and qualification structures.
Every student graduating will receive a Diploma Supplement
describing the qualification that the student has received.
The purpose of the Diploma Supplement is to improve
transparency and facilitate recognition. A standard format will
be used to help compare qualifications and make them easier
to understand. The Diploma Supplement will also describe
the content of the qualification and the structure of the higher
education system in which it was issued.
5
Bologna Process:
As a step towards achieving
greater clarity in the description
of qualifications, by 2010 all
modules and programmes in third
level institutions throughout the
European Union must be written
in terms of learning outcomes.
“Learning outcomes represent
one of the essential building
blocks for transparency within
higher education systems and
qualifications”
- Bologna Working Group, p.18
(December 2004)
Staff training in UCC – lunchtime
session and setting up of
“Postgraduate Certificate /
Diploma in Teaching and
Learning at Higher Education”.
European Credit Transfer and
Accumulation System (ECTS)
The European Credit Transfer System (ECTS) was initially
set up
p in 1989 as a p
pilot scheme within the framework of the
Erasmus programme.
Its aim at that time was to facilitate the recognition of study
periods undertaken abroad by mobile students through the
t
transfer
f off credits.
dit
A credits system is a systematic way of describing an
educational programme by allocating a certain value (credits)
t each
to
h module
d l off th
the programme tto d
describe
ib th
the student
t d t
workload required to complete the module.
Bologna Process has developed the ECTS system from
simply being a system for recognising study at foreign
institutions into a Credit Transfer and Accumulation System.
This takes ALL learning into account – not just study in other
countries
countries.
Hence, ECTS now stands for “European Credit Transfer and
Accumulation System”.
7
National Framework of
Q
Qualifications
f
in Ireland
Available at : http://www.nqai.ie/en/Publications/File,1268,en.pdf
8
Verification of National Framework of Ireland with EHEA Framework
Verification of Compatibility of Irish
National Framework of Qualifications
with the Framework for Qualifications
of the European Higher Education
Area
Summary of
Final Report – November 2006
http://www.nqai.ie/en/International/VerificationofCompatibilityofIrishNationalFrameworkofQualifications/File,1797,en.doc
9
Modularisation
A module is a self-contained fraction of a student’s
workload for the year and carries a unique
examination/assessment mark
mark.
The size of a module is indicated by its credit weighting.
Under ECTS system,
y
, each year
y
of degree
g
p
programme
g
=
60 credits.
Modules are allocated 5, 10, 15 or 20 credits depending
on the fraction of the programme workload covered in
the module.
Each module is given a unique code, e.g. ED2013
ED2013
Education
Year 2 Number assigned to this module
10
Advantages of modularisation
Gi
t clarity
l it off structure
t t
dh
l tto establish
t bli h clear
l
Gives
greater
and
helps
relationship between credits and student workload in ECTS
system.
Reflects
the various
elements
R fl t more accurately
t l th
i
l
t off students’
t d t ’
workload.
Facilitates work abroad, work placement, off-campus study as
modules for degree examinations.
examinations
Gives greater clarity and consistency in assessment.
Provides flexibility in the design of degree programmes by
i
incorporating
ti modules
d l ffrom diff
differentt areas.
Facilitates credit accumulation, i.e. increases number of
pathways to final degree award. Hence, encourages greater
di ersit of st
diversity
students,
dents e
e.g.
g mat
mature
re and part time st
students.
dents
Allows third level institutions to participate in schemes like
SOCRATES so that students obtain ECTS credits towards
their degree.
degree
Facilitates greater ease of student transfer between
institutions offering ECTS-based programmes.
Modules Marks
Modules,
Marks, Exams in UCC
Module
Student
Marks
Workload
5 ccredits*
ed ts 125
5 – 150
50 hours
ou s 100
00
Exam
Paper
1.5
5 hours
ou s
10 credits 250 – 300 hours 200
3 hours
15 credits 375 – 450 hours 300
3 hours
20 credits 500 – 600 hours 400
2 x 3 hours
Note: Total per year = 60 credits = 1200 marks
12
In University College Cork, a
5
5-credit
dit module
d l normally
ll
consists of 24 hours of
lectures plus associated
tutorials/essays /
readings/practical/coursework
OR
The equivalent in student
workload such as literature
projects, field courses, or
indeed set reading assessed
b written
by
itt examination,
i ti
work
k
for problem sets, studying of
legal material and cases
outside of lecture hours, etc.
13
How do I link Learning
Outcomes to Teaching,
and Learning Activities
and Assessment?
14
Assessment of Learning Outcomes
Having designed modules and
programmes in terms of learning
outcomes, we must now find out if our
students have achieved these intended
learning outcomes.
How will I know if my students have
achieved the desired learning
outcomes? How will I measure the
extent
t t to
t which
hi h they
th have
h
achieved
hi
d
these learning outcomes?
Therefore, we must consider how to
match the method of assessment to the
different kinds of learning outcomes e
e.g.
g
a Learning Outcome such as
“Demonstrate good presentation skills”
could be assessed by the requirement
that each student makes a presentation
p
to their peers.
When writing learning outcomes the verb
is often a good clue to the assessment
technique.
How can we design our examination
system so that it tests if learning
outcomes have been achieved?
Important to ensure that there is alignment between teaching methods,
learning
g outcomes and assessment criteria.
Clear expectations on the part of students of what is required of them
are a vitally important part of students’ effective learning (Ramsden
(Ramsden,,
2003))
This correlation between teaching, learning outcomes and assessment
helps to make the overall learning experience more transparent and
meaningful for students.
Teaching for
understanding
Learning outcomes
There is a dynamic equilibrium between teaching strategies and Learning Outcomes.
16
Formative Assessment
‰ Assessment FOR learning – gives
‰
‰
‰
‰
feedback to students and teachers
to help modify teaching and
learning activities, i.e. helps inform
teachers and students on progress
being made.
Assessment is integrated into the
teaching and learning process.
Clear and rich feedback helps
improve performance of students
(Bl k and
(Black
d Willi
Williams, 1998)
1998).
Usually carried out at beginning or
during a programme, e.g.
coursework which g
gives feedback
to students.
Can be used as part of continuous
assessment, but some argue that it
should not be part of grading
process (Donnelly and Fitzmaurice,
2005)
17
Summative Assessment
Assessment that summarises student learning at end of
module
d l or programme – Assessment
A
t OF Learning.
L
i
Sums up achievement – no other use.
Generates a g
grade or mark.
Usually involves assessment using the traditional
examination.
Only a sample of the Learning Outcomes are assessed –
cannot assess all the Learning Outcomes.
18
Continuous Assessment
A combination of summative and formative
assessment.
Usually involves repeated summative
assessments.
Marks recorded.
Little or no feedback given.
19
It is important that the assessment tasks mirror the Learning Outcomes
since, as far as the students are concerned, the assessment is the
curriculum: “From
“
out students’ point off view, assessment always defined
f
the actual curriculum” (Ramsden, 1992).
Biggs (2003) represents this graphically as follows:
Teacher
Perspectives: Objectives
Student
Perspectives: Assessment
Learning
Outcomes
Teaching
Activities
Learning Activities
Assessment
Outcomes
“To the teacher, assessment is at the end of the teaching-learning sequence of
events, but to the student it is at the beginning. If the curriculum is reflected in the
assessment, as indicated by the downward arrow, the teaching activities of the
teacher and the learner activities of the learner are both directed towards the same
goal. In preparing for the assessment, students will be learning the curriculum”
(Biggs 2003)
20
Putting our assessment under
the microscope
It is important to focus on the Learning Outcomes of our modules
and
d programmes and
d the
h criteria
i i ffor achieving
hi i these.
h
Study our examination questions to see what is actually being tested
in the examination questions. e.g are we testing over the full range
of Bloom
Bloom’s
s Taxonomy?
Are we testing across the range in Bloom’s Taxonomy? [Knowledge,
comprehension, application, analysis, synthesis and evaluation].
“What
What are we looking for in this module?”
module?
“What are we looking for in this programme?”.
21
Do students have to achieve ALL the
L
i O
t
tto pass a module?
d l ?
Learning
Outcomes
“Yes” in theory but often “No” in practise.
Summative assessment - with a terminal examination by its very nature can
only assess a sample of the Learning Outcomes. One cannot assess
everything in a 2 or 3 hour examination.
Easier to assess all the Learning Outcomes with continuous assessment.
The 40% pass mark - what does it mean?
Try to assess all the Learning Outcomes at least once.
The role of the external examiner.
Caution should be exercised when specifying pre-requisite modules. To
allow
ll
greater
t flflexibility
ibilit (t
(transfer
f ffrom other
th iinstitutions
tit ti
att h
home or abroad),
b d)
list certain modules as “desirable”.
Allow flexibility when writing Learning Outcomes – if you make them too
specific, you restrict yourself when carrying out the assessment.
Remember Heisenberg’s Uncertainty Principle – the more tightly you
specify each Learning Outcomes, the less able you are in ensuring that the
student achieves a pass grade in each one.
Programme Accreditation
Module descriptors with
clearly written Learning
Outcomes – see handout
(1) CIT.
Framework for
Accreditation e.g.
Engineer’s Ireland.
Mapping of Programme
Areas vs Programme
Outcomes – see handout
(2) CIT.
Mapping of Module
Learning Outcomes vs
Programme Learning
Outcomes
Prog.
Outcome
1
√
√
Module
4
Module
6
Prog.
Outcome
4
√
Module
3
Module
5
Prog.
Outcome
3
√
Module
1
Module
2
Prog.
Outcome
2
√
√
√
etc
Steps involved in linking Learning Outcomes,
Teaching
g and Learning
g Activities and Assessment
Clearly define the learning
outcomes.
2. Select teaching and
learning methods that are
likelyy to ensure that the
learning outcomes are
achieved.
3 Choose a technique or
3.
techniques to assess the
achievement of the
learning outcomes
outcomes.
4. Assess the learning
outcomes and check to
see how well they match
with what was intended
1.
If the learning
outcomes
t
are
clearly written,
the assessment
is quite easy to
plan!
24
Linking Learning Outcomes, Teaching and
Learning
L
i A
Activities
ti iti and
dA
Assessmentt
Learning Outcomes
Teaching and Learning
A ti iti
Activities
Assessment
Cognitive
(Demonstrate:
Knowledge Comprehension
Knowledge,
Comprehension,
Application, Analysis,
Synthesis, Evaluation)
Lectures
•End of module exam.
•Multiple choice tests.
•Essays
•Essays.
•Reports on lab work
and research project.
•Interviews/viva.
•Practical assessment.
•Poster display.
•Fieldwork.
•Clinical examination
examination.
•Presentation.
•Portfolio.
•Performance.
•Project work.
•Production of artefact
etc. 25
Tutorials
Discussions
Laboratory work
Affective
(Integration of beliefs,
beliefs ideas and
attitudes)
Clinical work
Group work
Seminar
Psychomotor
(Acquisition of physical skills)
Peer group presentation
etc.
Learning outcomes
Module ED2100
Teaching and Learning
Activities
Assessment
10 credit module
Mark = 200
Cognitive
•Recognise and apply the basic
principles of classroom
management and discipline.
•Identify the key characteristics of
high quality science teaching.
•Develop a comprehensive
portfolio
tf li off lesson
l
plans
l
Lectures (12)
End of module exam.
Tutorials (6)
Portfolio of lesson plans
Affective
•Display a willingness to cooperate
p
with members of
teaching staff in their assigned
school.
•Participate successfully in Peer
Assisted Learning project
Participation in mentoring
feedback sessions in school (4)
Psychomotor
•Demonstrate good classroom
presentation skills
•Perform laboratory practical
work in a safe and efficient
manner.
Observation of classes (6) of
experienced science teacher
(mentor)
(100 marks)
k )
Report from school mentor
Participation in 3 sessions of
UCC Peer Assisted Learning
(PAL) Programme.
End of project report.
Peer group presentation
(50 marks)
Teaching practice
6 weeks at 2 hours per week.
Supervision of Teaching Practice
Laboratory work
Assessment of teaching skills
(50 marks)
26
The Experience of using Learning
Outcomes
27
1. Identify aims and objectives of
module
2. Write learning outcomes using
standard guidelines
3. Develop a teaching and learning
strategy to enable students to
achieve learning outcomes
4. Design assessment to check if
learning outcomes have been
achieved
5. If necessary modify module content and
assessment in light of feedback
28
“Writing Learning
Outcomes is a
Process not an
Event”
29
That’s all Folks.
Folks Hope you
learned something about
the Bologna Process and
how to link learning
outcomes to teaching
g and
assessment!
30
References
Biggs J, (2003) Teaching and Learning in Higher Education: New Trends and Innovations.
Innovations.
University of Aveiro, 13 – 17 April 2003
Black, P and William, D (1998) Inside the Black Box: Raising Standards through Classroom
Assessment London: Kings College
Assessment,
College.
Bologna Working Group on Qualifications Frameworks (2004). Report on “A Framework for
Qualifications of the European Higher Education Area”.
Bologna Process Stocktaking London 2007. Available at:
www.ond.vlaanderen.be/hogeronderwijs/bologna
www.ond.vlaanderen.be/hogeronderwijs/
bologna/documents/WGR
/documents/WGR2007
2007//Stocktaking
Stocktaking_report
_report2
2
007.pdf
007
.pdf
Donnelly, R and Fitzmaurice, M. (2005). Designing Modules for Learning . In: Emerging Issues
in the Practice of University Learning and Teaching,
Teaching, O’Neill, G et al. Dublin : AISHE.
ECTS Users’ Guide (2005) Brussels: DirectorateDirectorate-General for Education and Culture. Available
online
li at:
t http://ec.europa.eu/education/programmes/socrates/ects/doc/guide_en.pdf
htt //
/ d
ti /
/
t / t /d / id
df
ECTS Key Features: http://www.bologna.msmt.cz/files/ECTSKeyFeatures.pdf
31
National Qualifications Frameworks Development and Certification – Report
from Bologna Working Group on Qualifications Frameworks. May 2007
http://www.ond.vlaanderen.be/hogeronderwijs/bologna/documents/Working_gr
oup_reports_2007.htm
Ramsden, P (2003) Learning to Teach in Higher Education
Ramsden
Education, London:
Routledge.
Verification of Compatibility of Irish National Framework of Qualifications
with the Framework for Qualifications of the European Higher
Education Area - Summary of Final Report – November 2006
http://www.nqai.ie/en/International/VerificationofCompatibilityofIrishNationalFra
meworkofQualifications/File,1797,en.doc
www.bologna.ie
32