The Problem with Reflection:
Lessons Learned
Alfredo Gaitan (alfredo.gaitan@beds.ac.uk_
University of Bedfordshire
Cathy Buyarski (cbuyarsk@iupui.edu)
Indiana University-Purdue University Indianapolis
Presentation delivered at the AAEEBL meeting 25-28 July 2011.
Session Overview
• Introduction of presenters and portfolios
• Three sections:
– Using reflection – institutional contexts and overview of portfolios
– Assessing reflection – how, when, how well
– What we learnt about reflection – in our projects and from each other
• We will aim to share two different projects/
perspectives, e.g.
– Campus versus department
– Freshman versus across enrolment
– Summative assessment versus formative assessment
– Insights on the students’ experiences of constructing eportfolios
Institutional contexts of PDP
IUPUI
Personal development planning is a
process which will enable first year
students at IUPUI to understand,
implement, and mark progress toward a
degree and career goal by creating and
following a personalized plan that is
open to revision and reevaluation every
semester in collaboration with an
academic advisor, faculty member, or
other mentor.
RISE Each undergraduate student is
challenged to include at least two of the
four RISE experiences - research,
international, service learning, and
experiential learning - into their degree
programs.
Principles of Undergraduate Learning
UoB
•
•
E-portfolios fit with the PDP model of
learning, support the personalisation
agenda and can also serve to
structure learning for students
studying remotely from the
University. “We will further develop
the use of e-portfolios as a framework
to enable all students to record and
review their progress and e-PDP as a
means for structured and supported
development.”
CRe8. The ‘realistic learning’
dimension emphasises a learning
experience which is meaningful,
active, reflective, collaborative and
challenging.
Overview of ePortfolios
IUPUI
• Implemented in first-year
seminar course across the
entire campus (342 students
in fall 2010).
• Reflection (with very specific
guided prompts) in seven
areas:
•
•
•
•
•
About Me
Educational Goals and Plan
Career Goals
Academic Showcase
Campus and Community
Connections
• My College Achievements
• Resume
UoB
• Implemented at levels 1, 2, 3
of the Psychology curriculum
with 225 students In Year 1
and 342 in Year 2.
• Reflection (with prompts) in a
optional structure:
–
–
–
–
Background,
My Learning Experiences
My Learner Development
My Employability (includes a
resume)
– Showcasing my performance
Portfolio Examples
• UoB
IUPUI
Example
Examples
Assessment of Reflection
IUPUI
•
UoB
Assessment to date has been
formative and designed to improve
prompts and rubrics.
Main assessment of the First Year
Seminar (1 credit): 100%.
•
•
Assessed at the level of each
prompt.
•
Assessed at the level of sections or
whole portfolio.
•
Rubrics based Bloom’s Taxonomy.
•
Rubric loosely based on
experiential learning (e.g. Kolb).
•
Feedback provided to students on
each section of portfolio.
•
•
Feedback: comments written on
each page/section and the
portfolio.
Conducted through content
analysis process.
•
Moderation in some units.
•
•
Assessment was summative of
students’ planning, experience,
reflection, prospective view.
Part of the assessments in each of
three units (30-credit modules):
5%, 20%, 10%.
Lessons Learned
IUPUI
Table 1: 2010 e-PDP Compared to Not E-PDP First-Year Seminar Sections:
Student Characteristics and Academic Success Indicators.
Fall DFW
Rate
Fall –
Spring
Retention
Rate
13%
12.10%
91%
2.78
18%
17.23%
89%
2.81
18%
16.45%
89%
Avg.
H.S.
GPA
Avg.
Fall
GPA
% Fall
GPA
below a
2.0
346
3.32
2.95
1936
3.30
2282
3.30
N
ePDP
Not e-PDP
Overall
Note 1: Missing cases were excluded from analyses.
Note 2: Students who Withdrew or who were Administratively Withdrawn from Seminars were excluded
(N=66 students).
Note 3: Bolded items are significantly different based on independent samples t-test or chi-square results
(p < 05).
Lessons Learning
University of Bedfordshire
Table 2. Submissions of e-portfolios over the two years of the project.
Level 1
Year 1 (2008-9)
49 (50%)
Year 2 (2009-10)
134 (75%)
Level 2
(n=96)
64 (72%)
(n=184)
66 (91.7%)
Level 3
(n=89)
8 (11%)
(n=72)
74 (86%)
Total
(n=70)
121 (53.8%)
(n= 86)
274 (80.1 %)
N=225
N=342
Lessons Learned
IUPUI
• Assessing reflection in developmentally
appropriate ways
• Scaffolding the use of reflection
• Faculty development
• Rubrics need to include cognitive development
AND critical thinking
• Grading versus assessment
Lessons Learning
University of Bedfordshire
An emergent ‘grounded theory’ 1 of the students’ experiences: interconnected
themes
(1) Glasser & Strauss, 1967; Pigeon, 1996; Pigeon & Henwood, 1997)
What We Learnt from Collaborating
Cathy
• Moving the PDP to embed
in academic programs is an
important next step
• Faculty buy-in and
development will be key
factors in our success
• Use of themes from UoB
will inform our thinking
about student participation
Alfredo
• The eporfolios at UoB could be
more directly linked to the
coursework.
• Our portfolios are snapshots
of a year, and only the Year 3
takes a long view (3 portfolios
rather than one).
• The strong ethos and
framework (e.g. RISE) may
provide an environment
favorable to e-portfolios.
• What can eportfolios do for
your institution?
Let’s talk!
• Any questions?
• What have you learnt from your own
experiences that you can offer us?
Thank you!