Exploring the Practice of
Assessment in First Year
Suzanne Guerin – School of Psychology
Jonathan McNulty – School of Medicine & Medical Science
Michael Staunton – School of History & Archives
UCD Fellows in Teaching and Academic Development
Introduction
• Assessment is central to teaching and learning practice
and policy (Goos, Gannaway & Hughes, 2011)
• Evidence of increased awareness and variation
• Continues to be an area for improvement (Craddock &
Mathias, 2009)
• Developments can create challenges for key
stakeholders (Brew, Riley & Walta, 2009)
• Little research into provision of assessment, choice of
methods and factors that influence practice (Craddock &
Mathias, 2009)
Aim and Justification for Study
• Aim of this study is to explore the practice of assessment in
first year teaching
• Underpinned by centrality of assessment, esp. in first year, in
UCD’s education strategy; One of the key aims of which is:
‘To foster early and lasting student engagement’,
which includes :
‘A review and reform of the structure, outcomes, assessment
and remediation strategies for first year, and in particular the
first semester, to support the transition from second- to
third-level and to adapt to the different needs of different
students; The further development of approaches to engage
and support students, especially in their first year, including
small group learning, peer-mentoring, academic advice and
mentoring, specific supports for the development of
transferable skills and information literacy, and general
welfare supports‘.
Justification
• We need to examine practice against UCD Teaching &
Learning assessment principles (O’Neill & Noonan, 2011):
– regular, low stakes assessment with opportunity for
feedback on their progress
– in-class self and/or peer review
– collaborative learning (peer & group work, project
work)
– redesign of learning sequence, blended learning
opportunities
– Active/task-based learning with more authentic
assessments (subject/discipline identity)
– Awareness of student work-load
Research Questions
• What factors currently contribute to assessment
practice in first year modules?
• What supports and obstacles are evident in
assessment practice?
• To what extent are assessment practices aligned with
the UCD assessment principles?
Overarching design
1. Module review (Autumn 2011-Spring 2012)
2. Survey of staff (Autumn 2012)
3. Case studies (Spring-Summer 2013)
Phase 1: Module Review
• Aim: to establish the types of assessment used in first year
modules and key patterns
• Examined institutional data provided in Module Descriptors,
which contain the core information provided to students
• Content analysis (Downe‐Wamboldt, 1992) of information
from 2011-2012
• 627 first year modules, drawn from 36 schools in the
Colleges of Arts and Celtic Studies, Business and Law,
Engineering and Architecture, Health Sciences, Human
Sciences, and Science
Module review: Key Findings
• Average module size = 149
(Range 10-647)
• Average number of
assessments = 2.5 (Range
0-8)
• Only 11.8% of modules
included reference to
assessment in the module
descriptor text
Module review: Assessment Type
Module review: Assessment Timing
Phase 2: Designing a Survey of Staff
• Aim of survey is to examine practice of assessment in
more detail
• Series of key stakeholder interviews conducted to
inform design of the survey
• Seven individuals (four male, three female) from
administration, academic and management positions
• Thematic analysis (Braun & Clarke, 2006) used to
identify main themes
Phase 2: Interview Findings
• Recognised importance of assessment – as both
a positive and negative influence:
“I think the assessment can be a hindrance [on
engagement]. On the other hand, of course, it
keeps the guys who are not genuinely excited
going because they know they are going to be
assessed.”
Phase 2: Elements of Assessment
Structure
• “[for] almost all of our modules, its 60/30/10, 60 being the end-of-term
assessment, 30 being the midterm assessment, and 10 being
attendance at [class]. Except for when its modules at level one, which
is 20/40/40, and the 20 includes presentation, attendance, and
contribution element, it is the one that is different on that particular
loading.”
Methods
• “has multiple forms of assessment. There are assessments based on
short grading exercises by the students. There are assessments based
on our involvement in class activities. There are assessments based on
presentation abilities, and finally there’s … a class project report that
has to be submitted. I think all of these assess different things, and for
that reason they’re pretty good”
Timing
• “What I’m finding is a lot of coursework seems to be coming in in week
twelve.” This participant went on to note that “because of UCD’s policy
of two weeks late submission … week twelve becomes week fourteen.”
Discussion: module review and interviews
• Average number of assessments lower than
previously identified and fewer modules had three or
more assessments (O’Neill & Noonan, 2011).
• Variation in methods reflects Craddock & Mathias
(2009) though clear emphasis on exams.
• Interviews identified a broad range of influencing
assessment choices reflects previous research (Harris
& James, 2006; Havnes & McDowell, 2008).
• Patterns suggest limited impact of the UCD
assessment principles (O’Neill & Noonan, 2011).
13
Staff survey: online questionnaire
• Survey of all staff involved in delivery of first
year modules
• Survey examines:
– Current assessment practice in first year
modules
– Related workload for staff and students
– Familiarity with and attitudes towards the
UCD Assessment Principles
– Factors influencing assessment practice
Staff Survey: Methodology
• Online survey of staff
• Targets staff listed as coordinating Level 1 UG modules in
2011-2012 and 2012-2013
• Initial email to Heads of School and Heads of Teaching &
Learning in all schools
• Invitation email sent by Prof. Bairbre Redmond to stress
institutional importance of the survey
• Follow up email after two weeks (due week of 26/11/12)
Staff Survey: Preliminary Analysis
• 52 responses to date, 61.5% completed
• 44% male, 44% female, 12% missing data
• Years teaching in UCD:
– Range 1-35 yrs, Average = 10.9 (SD = 7.35)
• Number of Stage 1 UG modules coordinated
– Range 1- 5, Mode = 1
• Number of Stage 1 UG modules taught
– Range 1- 5, Mode = 1
• Teaching & Learning Responsibilities: 65.4%
What College are you based in?
Assessment Types Examined
•
Annotated
bibliography
•
Exam: Essay-based •
Make a video
•
Exam: Multiple
choice
•
Peer assessment
•
Performance piece
•
Portfolio
•
Presentation: Poster
•
Book review
•
Case study
•
Exam: Mini-tests
•
Clinical: Objective
structured exam
•
Exam: Oral
•
•
Data-based project
•
Debates or
discussion
•
Exam: Short answer •
questions
•
Exam: Take home
•
exam
•
Group work
•
Demonstration
•
Lab report
•
Self evaluation
•
Design project
•
Learning contracts
•
Work-based problem
•
Essay
•
Learning journal
•
Library research
assignment
•
Clinical: Practical
assessment
•
Presentation: Oral
Problem scenario
Report
Role play
Types of Assessment Used
Module review: Assessment Type
Types of Assessment Used
• Evidence of group assignments, self- and peer- assessment,
active / task-based assessments, learning journals, etc.
• Assessment FOR learning
Additional Questions on Assessment
• What proportion of the module does the assessment
account for?
• How useful are the methods of assessment?
• What is the associated workload for staff and for
students?
• Definition and use of formative assessment.
• Awareness and views of UCD assessment principles
• Factors influencing assessment practice
• Views on strengths and limitations of current module
descriptor system
Phase 3: Case studies: sampling criteria
• Stratified sampling:
–
–
–
–
e.g.
Module size
Number of assessments above and below the mean
Traditional v non-traditional forms of assessments
Involvement in T & L
4 large modules, half with assessments above the mean,
half below
4 small modules, half with assessments above the mean
• one case study per block (n=8)
Case studies: data collection
• Documentary material e.g. module guides
• Student feedback (via module coordinators)
• Focus groups with students
• Interviews with module coordinators
Data analysis:
• anonymous cross case analysis (public),
• underpinned by individual case analysis (private)
Thank you
• Suzanne Guerin – suzanne.guerin@ucd.ie
• Jonathan McNulty – jonathan.mcnulty@ucd.ie
• Michael Staunton – michael.staunton@ucd.ie
UCD Fellowships funded under the Strategic Innovation Fund II