Student Feedback on, and
Evaluation of, their Learning
Experience
Michael Prosser
Institute for Teaching and Learning
The University of Sydney
INTRODUCTION
Some distinctions in the way we talk about and describe
students learning experiences in higher education
A. Consumer and client centred views of student learning
experience
- ratings of student satisfaction with teaching and
services
B. Learner centred views of student learning experience
- indicators of student perceptions and experiences
relate to learning approaches and outcomes
In this presentation I wish to focus on the experience of
students as learners – learner centred views
2
A. Design of learning experiences:
-
focus on the way student experiences are designed
by the teacher
B. Student experiences of the design of student learning
experiences:
- focus on the way the student experiences the design
3
It is not (just) the way we design our courses and programs
in higher education that relates to the quality of student
learning but how our students experience and understand
those designs.
Finding out, understanding and responding to the way
students experience our courses and programs – the student
experience of our courses and programs
Working towards aligning the way our student experience
our courses and programs with the way we design those
courses and programs
4
OVERVIEW OF THE STUDENT LEARNING
PERSPECTIVE
Figure 1: Model of Student Learning
CHARACTERISTICS
OF THE STUDENT
(e.g. previous
experiences, current
understanding)
STUDENTS'
PERCEPTIONS
OF CONTEXT
(e.g. good teaching,
clear goals)
STUDENTS'
APPROACHES
TO LEARNING
(how they learn
e.g. surface/deep)
STUDENTS'
LEARNING
OUTCOMES
(what they learn
quantity/quality)
COURSE AND
DEPARTMENTAL
LEARNING
CONTEXT
(e.g. course design,
teaching methods,
assessment)
5
Student Approaches to Learning
A key variation represented by:
Surface Approach:
Intention to reproduce
• rote memorise information needed for assessment
• failure to distinguish principles from examples
• treat tasks as external impositions
• focus on discrete elements without integration
Deep Approach:
Intention to understand
• meaningfully memorise information for later use
• relate new ideas to previous knowledge
• relate concepts to everyday experiences
• relate evidence to conclusions
6
Students’ Experiences of The Learning Context
Student experiences of:
• Quality of teaching – including quality of feedback
• Clearness of course goals and standards
• Workload - too high to understand
• Assessment - reproduction and not understanding
are key aspects of student experiences found to relate to
how they approaches their studies and to their learning
outcomes
Entwistle and Ramsden, 1983
7
EXAMPLE FROM MEDICINE
Quantitative Survey
Study of over 8000 students in first year subjects around
Australia, including 618 first year medical students in three
different first year courses
Amongst the data collected were students responses to:
• Ramsden's Course Experience Questionnaire
• Biggs’ Study Process Questionnaire
1994-1996: Australian Research Council ; Academic Departments and the
Quality of Teaching and Learning; Paul Ramsden, Griffith University, Elaine
Martin, RMIT, Michael Prosser, La Trobe University, Keith Trigwell, UTS
8
Approaches to Study
Surface Approach
32. Although I generally remember facts and details, I find
it difficult to fit them together into an overall picture
Deep Approach
28. I try to relate ideas in this subject to those in other
subjects, wherever possible
Biggs’ Study Process Questionnaire
9
Student Experiences Of Learning Environment
Good Teaching
15. The staff made a real effort to understand difficulties
students might be having with their work.
Clear Goals and Standards
1. It was always easy to know the standard of work
expected
6. I usually had a clear idea of where I was going and
what was expected of me in this subject.
Appropriate Workload
25. The sheer volume of work in this subject meant that it
couldn't all be thoroughly comprehended (-).
Appropriate Assessment
8. To do well in this subject, all you really need is a good
memory (-).
10
Paul Ramsden’s Course Experience Questionnaire
Analyses showed that within each course:
1. A deep approach is associated with experiences that
the teaching is good and the goals and standards are
clear
2. A surface approach is associated with experiences that
the workload is too high and assessment tests
reproduction
That is, variation in students’ experiences of the learning
environment within subjects is associated with the
approaches to study within subjects
– within subject variation in experience is not measurement
11
Factor Analysis of Experiences of Learning Context and
Approach to Study
________________________________________________
Scale
Subject A
Subject B
Subject C
Factors
Factors
Factors
________
________
________
1
2
1
2
1
2
________________________________________________
Experiences
Good teaching
.84
.77
.79
Clear Goals & Stan .68
.70
.69
App. Workload
-.73
-.68
-.69
App. Assessment
-.70
-.60
-.60
Approaches
Deep Approach
.75
.84
.80
Surface Approach
.62
.64
.72
12
________________________________________________
EXAMPLE FROM MATHEMATICS
Qualitative Survey
First year university mathematics students, large research
intensive university, analyses of short open ended written
statements by students identified:
Key differences in the experiences of Mathematics:
Fragmented Experience
A. Maths as numbers, rules and formulae
B. Maths as numbers etc with applications to problems
Cohesive Experience
C. Maths as a way of thinking
D. Maths as a way of thinking for complex problem solving
E. Maths provides insights for understanding the world
13
Experience Quote
of Maths
A: Numbers Maths is the study of numbers and the
application of various methods of changing
numbers
C: Logical
system
Maths is the study of logic. Numbers and
symbols used to study life in a systematic
perspective and requires the mind to think in a
logical and precise way
E: Complex Techniques for thinking about observable,
/ World
physical phenomena in a quantitative way and
also for thinking more abstractly with little or no
relation to the directly observable universe
14
Approach to Study
of Maths
Quote
A: Learning by rote
memorisation, with
an intention to
reproduce
(surface)
E: Learning with an
intention of gaining a
coherent
understanding of the
theory and looking
for situations where
the theory will apply
(deep)
I liked calculus because I could
remember formulas which is how I
used to study. I would rote learn all the
formulas and summarise all my
theoretical notes
Read the relevant theory and try to get
on the same “wavelength” as the
person who actually discovered it.
Before I attempt any problems I try to
think where you can use the concept:
i.e. what the concept was invented for.
Then I attempt problems (on my own)
15
Relationship between Experience of Mathematics and
Approach to Studying Mathematics
______________________________________________
Experience
Approach
Total
_______________
Surface
Deep
(A & B)
(C, D & E)
______________________________________________
Fragmented (A & B)
179 (91%) 17 (9%)
196
Cohesive (C, D, E)
4 (10%)
36 (90%)
40
Total
183
53
236
______________________________________________
Chi-square=126, p<.001; Phi=.57; ES=2.1
Crawford, K., Gordon, S., Nicholas, J. and Prosser, M. (1998) Qualitatively different experiences of
learning mathematics at university. Learning and Instruction, 8, 455-468.
16
Defining Items from the Experiences of Mathematics
Questionnaire
Scale Item
Item
Fragmented Experience
9. Mathematics is figuring out
problems involving numbers
Cohesive Experience
17. Mathematics is a logical
system which helps explain
the things around us
17
Cluster Analysis of Experiences and Understandings
_____________________________________________________________
Variables
Cluster 1
Cluster 2
(n=147)
(n=127)
_____________________________________________________________
Prior Experiences and Understandings
Fragmented experience of Maths
0.31
-0.46
Cohesive experience of Maths
-0.39
0.45
Prior Surface Orientation
0.34
-0.44
Deep Orientation
-0.56
0.59
Academic Ranking (TER)
-0.29
0.35
Experiences and Approaches
Good Teaching
-0.37
0.37
Clear Goals and Standards
-0.23
0.24
Appropriate Workload
-0.30
0.32
Appropriate Assessment
-0.12
0.12
Surface Approach
0.44
-0.45
Deep Approach
-0.54
0.57
Post Experiences and Understanding
Fragmented experience of Maths
0.34
-0.41
Cohesive experience of Maths
-0.27
0.27 18
Achievement (Final Mark in Maths)
-0.34
0.40
Cluster Analysis of Variables (medium to large differences)
_____________________________________________________________
Variables
Cluster 1
Cluster 2
(n=147)
(n=127)
_____________________________________________________________
Prior Experiences and Understandings
Fragmented experience of Maths
High
Low
Cohesive experience of Maths
Low
High
Prior Surface Orientation
High
Low
Deep Orientation
Low
High
Experiences and Approaches
Good Teaching
Low
High
Clear Goals and Standards
Small difference
Appropriate Workload
Low
High
Appropriate Assessment
Small difference
Surface Approach
High
Low
Deep Approach
Low
High
Post Experiences and Understanding
Fragmented experience of Maths
High
Low
Cohesive experience of Maths
Low
High
Achievement (Final Mark in Maths)
Low
High
___________________________________________________
EXAMPLE FROM OXFORD UNIVERSITY
(Honours: n=428, Approach: n=2320)
_________________________________________________
Experience
Outcome
Approach
Honours
_________________
(1,2a,2b)
Surface
Deep
(+ve)
________________________________________________
Good teaching
Large
Low
High
Clear Goals and
Standards
Large
Low
High
Appropriate
Assessment
Medium
Low
High
Appropriate
Workload
Medium
Low
High
_________________________________________________
http://www.learning.ox.ac.uk/iaul/pdf/OLCPFinal.pdf
20
In summary:
Relationships between students experiences of their
course, their approaches to study and their learning
outcomes
View of student learning experiences:
• learner centred (learning) cf consumer / client
centred (satisfaction)
• student experience of learning designs cf design
of learning experiences
21
EXAMPLE FROM THE UNIVERSITY OF SYDNEY
Graduates:
Graduate Course Experience Questionnaire - Program
population survey of all coursework graduates
Postgraduate Research Experience Questionnaire - Program
population survey of all research graduates
22
Currently enrolled students:
Student Course Experience Questionnaire – whole year
stratified random sample of all enrolled coursework
students
Student Research Experience Questionnaire – whole year
population survey of all enrolled research students
Unit of Study Evaluation - course
compulsory triennial survey of all units of study
23
Supplemented and supported by:
Focus group discussions to further explore issues
identified in surveys
Academic Board Reviews of Faculties interview groups of
postgraduate and undergraduate students to better
understand their experiences
Performance based funding model for teaching to reward
Faculties enhancing the student experience
Faculty Teaching and Learning Plans with a focus on
Student Experience
24
IMPLICATIONS FOR EVALUATING STUDENT LEARNING
EXPERIENCES
1. Try to understand how and why student experience their
courses and programs they way they do – quantitative survey,
open-ended responses and follow up interview and focus
groups
2. In responding , try to respond to why, not how, they
experience their courses and programs the way they do – eg
workload, assessment
3. Implement institutional policies, procedures and strategies
that recognise and reward teachers, schools and departments
and faculties efforts to respond to and enhance students
learning experiences
25