Supporting the Design of
Discipline-Specific Learning Outcomes:
Experiences of the Tuning Group for Physics
Gareth Jones
Imperial College London
OUTLINE
• The Tuning Project
–
–
–
–
What, Why, Who?
Competences and Learning Outcomes
Hierarchy of Learning Outcomes and link to Level and Standards
Surveys and Results
• Degree Programme (Re)Design
– Main Requirements
– A fresh start or improve what exists
– Incorporating competences and content requirements
• Specific Examples
– IOP Accreditation requirements for Physics degrees
– Example of Module and Thematic Learning Outcomes
What is the Tuning Project?
•
The universities response to the Bologna Process: Most work done by
separate but coordinated teams of discipline experts each with one member
from each EU country
•
To find ways to implement a three-cycle degree structure
– To develop learning outcomes and competences for each cycle (reference
points) on basis of consensus after much discussion
– To survey views of students, graduates, academics and employers on
importance of both generic and subject specific competences
– To survey and compare programme content and structure
•
Development of ECTS as a credit accumulation system
•
Best Practice in teaching & learning and quality enhancement
•
Tuning Coordinators/Leaders: Julia Gonzalez & Robert Wagenaar
•
Tuning Physics Group Leader: ‘Lupo’ Dona dalle Rose
From the Tuning Final Report
Two of the key driving ideas of the
Tuning Project
• One of the main objectives of the Bologna process is to
make study programmes and periods of learning more
comparable and compatible. This objective is strongly
promoted by making use of the concept of levels,
learning outcomes, competences and ECTS credits.
• The Tuning emphasis on competences and learning
outcomes is intrinsic to the paradigm shift from a
professor-centred to a student-centred approach which
is seen as a key way of improving the effectiveness of
European HE.
Competences
• Ability to do something.
• Competences range from:
– specific and small, e.g. competence to use an oscilloscope, to
– general and large, e.g. competence to solve problems
• Acquired by students and assessed either in a specific part of a
course or throughout programme in an integrative, holistic way
• Learning Outcomes often expressed in terms of competences (but
not all)
• Generic Competences, e.g. general cognitive abilities, interpersonal
skills
• Subject Specific Competences
– Competences required and/or valued by profession/discipline
– Different universities may emphasise particular competences and deemphasise others  Profile of degree
Examples of Generic Competences
from Tuning
•
•
•
•
•
•
•
•
Ability to apply knowledge in practical situations
Capacity for analysis and synthesis
Capacity to learn
Creativity
Adaptability
Critical and self-critical abilities
Concern for quality
To act in accordance with a basic knowledge of the profession
Tuning Survey 2008 – Employers’ Response:
Most important generic competences
Tuning Survey 2008 - General Competences of Graduates - Employers' Response
4
3.5
3
Score
2.5
Importance
2
Achievement
1.5
1
0.5
0
Apply Knowledge Identify, Pose and
in Practice
Resolve Problems
Determination &
Perserverence
Oral & Written
Communication
Competence
Teamwork
Make Reasoned
Decisions
Stay up-to-date
with Learning
Physics Specific Competences/Learning Outcomes
•
•
•
•
•
•
•
•
Able to enter new fields through independent study
Familiar with the ‘work of genius’, i.e. with the variety and delight of physical
discoveries and theories, thus developing awareness of the highest
standards
Have a good understanding of the most important physical theories
including a deep knowledge of the foundations of ‘modern’ physics
Able to evaluate orders of magnitude in situations which are physically
different but show analogies
Able to understand and master the most commonly used mathematical and
numerical methods
Able to perform calculations, including the use of numerical methods and
computing, to solve problems
Able to construct mathematical models of a process/situation by identifying
the essentials of a process/situation and making justified approximations
Have a good knowledge of at least one frontier physics specialty
Physics Specific Competences/Learning Outcomes
(Practical/Experimental/Research)
• Able to perform experiments independently, as well as to describe,
analyze and critically evaluate experimental data and to be familiar
with the most important experimental methods
• Understanding of the nature and methods of physics research and
how it can be applied in other fields e.g. engineering
• Familiar with the ‘culture’ of physics research, including the relation
between experiment and theory and ability to span many areas
• Able to find physical and technical information relevant to research
work and technical project development using literature search
methods
• Able to work with a high degree of autonomy, accepting
responsibility in planning and managing projects
• Able to carry out professional activities in the area of applied
technologies and industry
Physics Specific Competences
(Human Dimension)
• Able to present one’s own results (research or literature search) to
professional and lay audiences orally and in written form using
appropriate language
• Able to work in interdisciplinary teams
• Prepared to compete for school teaching positions in physics
• To show a personal sense of responsibility, e.g. meeting deadlines,
and to show professional flexibility
• To behave with professional integrity and an awareness of the
ethical aspects of physics research and its impact on society
Tuning Survey on Competences 2008
Opinions on the most important Physics Specific Competences
Employers
Graduates
Students
Academics
Ability to enter new fields
Problem Solving
Literature Search
Ability to enter new
fields
Deep Knowledge &
Understanding
Mathematical Skills
Mathematical Skills
Modelling Skills
Ability to enter new
fields
Experimental Skills
Estimation Skills
Estimation Skills
Problem Solving
Deep Knowledge &
Understanding
Ability to enter new fields
Foreign Language Skills
Deep Knowledge &
Understanding
Specific
Communication Skills
Modelling Skills
Experimental Skills
Experimental Skills
Problem Solving
Managing Skills
Estimation Skills
Competence
C
M
Ex
pe
rim
s
ills
on
al
S
lls
kil
ls
Ab
ilit
ie
s
Se
ar
ch
Sk
ills
Sk
ills
Sk
ills
Sk
ills
ng
ills
en
ta
lS
ki
Pr
of
es
si
cip
lin
ar
y
Li
te
ra
tu
re
at
ica
l
tio
n
an
ag
in
g
m
un
ica
M
at
he
m
om
In
te
rd
is
fic
Hu
m
an
Sp
ec
i
Sk
e
n
La
ng
ua
g
at
io
Sk
fie
ld
So
lvi
od
el
lin
g
Pr
ob
le
m
M
en
te
rn
ew
Es
t im
to
Fo
re
ig
n
Ab
ilit
y
Importance Score
Tuning Survey of Competences 2008 - Physics
3.8
3.6
3.4
3.2
3
Employers
Graduates
2.8
Students
Academics
2.6
2.4
2.2
2
Learning Outcomes: What and Why?
• Statements of what students should know, understand or
be able to do as a result of following a course
–
–
–
–
Knowledge and understanding
Problem solving
Skills: experimental, mathematical, design, …
Ability at communication, teamwork etc.
• Use in defining levels: 1st and 2nd cycle level descriptors
• Part of Bologna Process and Qualification Frameworks
• Use in Programme Design & QA methodology
– What education is all about
– Must be assessed
Hierarchy of Learning Outcomes
• Module Level Learning Outcomes
–
–
–
–
Specified by Module Teacher and Programme Director
Should/must be assessed  mark or grade
Desired and threshold Learning Outcomes  criteria
Need to be specific but not too detailed
• Thematic Learning Outcomes, e.g. Quantum Mechanics
– Refer mainly to overall or final abilities. Forest not the trees
• Year Learning Outcomes: useful for progression criteria
• Programme Learning Outcomes, e.g. BSc (Hons) in
Physics
– General and summative statements
– Holistic
– ‘Dublin Descriptor’ type statements but applied to discipline  Refer to
Academic Level
Academic Level and Learning Outcomes
• Intended Learning Outcomes give a good indication of
competence for performing particular tasks, but:
– Need to be fairly specific, e.g. able to use time dependent
perturbation theory to solve problems in atomic and nuclear
physics. But:
• What kind of problems?
• How difficult?
– Need to refer to how assessed, e.g. exam questions.
• Learning Outcome statements for programmes are not
enough to compare standards. How do you add up
Learning Outcomes? Need to specify content/volume.
Are Learning Outcomes Helpful?
• Can be very helpful for programme design
– Focus mind on “What are the students getting out of it?”
• Can improve teaching and the output competences of
graduates
• How to assess whether or not they are achieved?
– Exams OK for academic problem solving but not so good for
realistic problem solving
– Difficulty of questions is crucial for standards but is hard to
control and interpret
– Mark Scale: Raw data for testing hypothesis “Has this LO been
achieved?” but what is threshold mark?
– Practical competences easier to test
Traditional Programme Design
• (Professor) i  (Course) i
– “I will teach them what I know”
• Programme = Σ (Course) i
• Leads to content and professor dominated curriculum
• Danger of
– Content overload and excessive ‘derivations’
– Obscurity of purpose: “Why are we doing this?”
– Little increase in competence
• Advantages (if have good professors!):
– Produces deep understanding for best students
– Good for producing future professors!!!
The Programme Design Problem
• An existing module synopsis can be basis for a list of Learning
Outcomes for that module
• The general characteristics of a degree programme can be defined
by Qualifications Framework statements
• But what goes in the middle?
–
–
–
–
Subject specific qualification and level descriptors (Benchmark)
Thematic Learning Outcomes
Structuring of content to ensure linkage and progression
Development of teaching, learning and assessment methods to enable
learning outcomes to be achieved and assessed holistically
• Construction of a matrix of competences vs. modules is very helpful
– Helps to ensure competences appear explicitly in the design
Matrix of Competence vs. Content
Knowledge
&
Understand
Mechanics
& Relativity
50%
Maths 1
20%
1st Yr Lab
10%
Quantum
Physics
60%
Professional
Skills
Apprec.
work of
Genius
Problem Maths
Solving Skills
10%
30%
30%
10%
50%
10%
20%
15%
20%
Experimental Communication
Skills
Skills
50%
30%
5%
10%
30%
Steps in Physics BSc Programme Design
• INPUTS
–
–
–
–
–
–
(a) IOP Accreditation Requirements and QAA Benchmark statement
(b) National Framework of Qualifications (NQAI)
(c) Desired Qualification Profile (e.g. Applied, Pure,…)
(d) Desired/expected student intake and potential employers
(e) Resources and existing degree programme modules
(f) Tuning results on Competences, Learning Outcomes, Content, …
• PROCESS
–
–
–
–
–
–
–
–
Internal Discussion: where we are → where we want to be, SWOT
Construct Matrix of Competences vs. Modules, using (a), (b), (f)
Check (c), (d), (e)
Develop Learning Outcomes for whole programme, themes and modules
Check academic level
Develop Teaching and Learning Methods and Assessments
ITERATE!
Will it work? Does it meet requirements? Is it realistic?
Seek wide support and administrative approval
Use of Learning Outcomes in Practice
(‘Reverse Engineering’)
• Start from where we are now
–
–
–
–
–
–
–
–
–
LO’s for each module: Improve them, check how assessed
Examine content: remove redundancies, add missing items
Check accreditation, benchmark, Tuning competences are met
Construct matrix of competences vs. modules
Iterate! It is likely there are gaps or deficiencies
Construct more generalised LO’s for themes, years, programme
Ensure logical progression, e.g. C depends on A and B
Check requirements of NQAI. Check academic level.
Iterate, again! Pay particular attention to assessment and
recent student results (marks, drop-out rates, employment, …)
• Present new programme for approval
Example of Approaches to Teaching & Learning
Tuning Physics Group
Modelling (second cycle)
•
•
•
Modelling in a narrow sense means finding a simplified mathematical
description of a complex phenomenon. It often means also applying tools of
theoretical physics to non-physics situations.
… There is no course unit named Modelling. Students learn the modelling
description of nature throughout their whole degree-course. Possible
examples are: the “modelling” neglect of friction in the description of free
fall, the abundant use of harmonic oscillator for phenomena in the
neighbourhood of stable equilibria, the shell model average field for
nucleons in nuclei, the modelling of two-nucleon and three-nucleon forces,
and so on.
The whole teaching offer is then important: in lectures, exercise classes, in
lab classes, in student seminars and during research training students learn
about how theories were developed, how to select and then apply
theoretical tools (e.g. models) to a particular physical problem and how to
model the building blocks of a theory, by adapting these latter to the
experimental data description.
Example of Approaches to Teaching & Learning
Tuning Physics Group
Problem solving skills (first cycle)
Active Learning: in all classes (theory, lab or problem solving)
•
•
•
•
Several questions are posed to the theory class and a certain amount of time is
allowed for discussion in the same class.
Several question-problems are set to the class and assigned to groups of students.
They should find an answer (either exact or approximate) in a certain amount of time.
They are also requested to explain their reasoning to other students (Did they divide
the problem in simpler problems? did they use analogies with problems, for which
they already knew the answer? why are they confident about their own answer?…)
In the exercise classes the students are requested to correct and comment other
students ways of solving the exercises.
In the lab classes students are frequently asked to solve experimentally or propose
ways for solving other more complex problems that may be considered extensions of
the material proposed in the class. (ex: after studying an LC circuit they are
encouraged to solve the problem of coupled LC circuits and think about the problem
of impedance adaptation in a transmission line).
IOP Accreditation Requirements
• The degree programme should foster intellectual
curiosity in the minds of students
• Graduates should have acquired
– A secure knowledge of an agreed core of physics + a few extra
frontier topics
– Competences represented by ‘graduate skills base’
• The degree programme must incorporate project work
– BSc level project work may be a dissertation
– MSc/MSci level project work must involve research skills
• The degree programme must be consistent with QAA
Benchmark
IOP Graduate Skills Base
(Part of Programme Learning Outcomes)
• Physics Skills: Physics students should be able to
–
–
–
–
Tackle problems in physics
Use mathematics to describe the physical world
Plan, execute, analyse and report experiments
Compare results critically with predictions from theory
• Transferable Skills: A Physics degree should enhance
–
–
–
–
–
–
Problem solving skills (well defined and open-ended)
Investigative skills
Communication skills
Analytical skills
IT skills
Personal skills (group work, use of initiative, meet deadlines)
Graduates should have a secure knowledge of
the IOP Core of Physics
• Mathematics for Physicists
• Mechanics and Relativity
• Quantum Physics
– including atomic, nuclear and particle physics
•
•
•
•
•
Condensed Matter Physics
Oscillations and Waves
Electromagnetism
Optics
Thermodynamics and Statistical Physics
Example of Module and Thematic LO’s
• 1st Year Mechanics Module LO’s (selection)
– Understand the concept of conservative force and its relation to the
potential function (in 3 dimensions)
– Be able to solve single particle motion from a given potential function in
two dimensions
– Be able to use angular momentum and energy conservation in central
force problems
• Can be tested by answers to exam questions but how to
interpret exam marks
– Not just “Yes or No” but partial “Yes”
– Index of “cleverness” or speed of working
• Thematic Learning Outcome for Mechanics
– Able to use Newton’s Laws in a wide range of areas of physics
– Aware of the power of conservation laws
– Aware of more advanced methods of Lagrangians etc.
Conclusions
• The traditional approach to programme design stresses content too
much and does not pay sufficient attention to the change we are
trying to produce in students in terms of their competences.
• A Learning Outcomes approach requires a re-thinking of why, what
and how we teach and of how we assess students’ achievements.
• It will require more effort initially from teachers but will probably
enable reductions to be made in the amount of content taught.
• Students must be given more scope for activities like problem
solving, team-work and communications but also must accept more
responsibility for their own learning.
• The Learning Outcomes approach is firmly embedded in the
Bologna Process. Tuning has shown how it can be used in a PanEuropean way