Webinar Series
This session is the first of e-Assessment Scotland 2013
20 Aug 2013:
Challenges and Opportunities Associated With
Implementing eMarking Workflows in a University Setting
Courtenay Harris, Connie Price and Helen Flavell (Curtin University, Australia)
Your hosts
Professor Geoff Crisp,
Dean Learning and Teaching, RMIT University
geoffrey.crisp[@]rmit.edu.au
Dr Mathew Hillier,
Teaching and Educational Development Institute,
University of Queensland
mathew.hillier[@]uq.edu.au
Challenges and Opportunities Associated
With Implementing eMarking Workflows
in a University Setting
Dr Courtenay Harris
Dr Connie Price
Dr Helen Flavell
Presenters
Dr Courtenay Harris
School of OTSW
Faculty of Health Sciences
Dr Connie Price
Manager of Assessment
Curtin Teaching and Learning
Dr Helen Flavell
Coordinator of SOTL
Faculty of Health Sciences
PERTH
WESTERN
AUSTRALIA
Assessment in the 21st Century Context
HE Context
Assessment Issue
 Threats to existence of
 Increased focus on role and value of
universities
assessment
 Accrediting student learning
remains key function of HE
 Infiltration of technology
 What aspects can be automated or
done by AI
 Decreasing government funding  Need to develop accurate costing
models for assessment
 Including different assessment
types
 Increasing accountability
 Greater proportion of workload
around/not in the marking process
 Increasing student numbers
 Every aspect of assessment is magnified
 Increasing student diversity
 Need for focused feedback
Foundations for Professional Health Practice 100
(FPHP 100)
 Common first year unit for all courses
in the Faculty of Health Sciences
 Super large unit
 2900 students across both semesters
 40 teaching staff
 Inter-professional education
 staff and students
 Internal (F2F) and external (online)
student enrolments
 Teaching model and use of class
based technology
 Varied assessment types
Drivers for eMarking within FPHP 100
• Electronic submission / return
•
•
•
•
Practicalities due to 2900 students /40 tutors
Feedback timeframes
Access to student submissions
Academic integrity
• Consistent and efficient assessment practice and feedback
• implementation of marking rubrics
• transparency for staff and students
• Change of balance of editing vs focused feedback
• Efficiency for staff, while maintaining rich feedback
• Students
• Student evaluation of the unit indicated improvement required
Electronic Management of Assignments
Assessment Management Workflow
Submission
Annotate
Grading
Apply
Rubric
Calculate
Grade
Record
Grade
Return
Feedback
Assessment Management Workflow
Submission
Annotate
Grading
Apply
Rubric
Calculate
Grade
Record
Grade
Return
Feedback
Assessment Management Workflow
Submission
Annotate
Grading
Apply
Rubric
Calculate
Grade
Record
Grade
Return
Feedback
Assessment Management Workflow
Submission
Annotate
Grading
Apply
Rubric
Calculate
Grade
Record
Grade
Return
Feedback
Workflow
Set-up your
iPad
• Open PDF Expert app
Set-up your
Computer
• Open up Blackboard FPHP
• Open Grade Centre with Smart view of tutorial group
• Open your commenting tool
Pre-marking
tasks
• Click on chevron to view student attempt
• Download student assignment file
• Open file in commenting tool (Word or Acrobat)
• Open master copy of marking rubric
Marking
tasks
• Check Originality Report
• Add comments to student’s file
• Complete and save marking rubric
Post-marking
tasks
Repeat for each student
• In Grade Centre enter mark
• Attach commented
assignment file
• Attach marking rubric
eMarking Working Party Priorities
1. Workflow
•
•
•
Minimal steps
Minimal tools
• Multiple screens
Minimise workload
2. Enhancing Integrity
•
•
•
Academic integrity (plagiarism check)
Moderation
Archiving of electronic artefacts
3. Assessment and Feedback
•
•
•
Evaluation -> Mark
Mark-up of artefact
Provide feedback in a variety of ways
Electronic Assignment Workflow for Blackboard Learn 9.1
Staff Workflow
Blackboard
Student Workflow
Desktop Application
Create Turnitin assignment
Create Blackboard eRubric
Submit assignment
View attempts via Grade Centre
Receive Digital Receipt
View Turnitin Originality Reports
View Originality Report
Download submissions from
Grade Centre
Launch MS Word or Adobe Acrobat
Annotate submission
Grade Assignment
Upload annotated submission
View mark
Download feedback
Store assignment artifacts
View eRubric
Electronic Assignment Workflow for ReMarksPDF
Staff Workflow
Turnitin
ReMarksPDF
Blackboard
Student Workflow
Create eSubmission Link
Submit assignment as .pdf
Download submissions from
Grade Centre .zip file
Upload .zip to Turnitin
View Turnitin Originality Reports
Launch ReMarksPDF App
Connect to Bb database
Create eRubric
Grade Assignment
Provide feedback
View mark
Upload annotated
submissions to Grade Centre
Download feedback
Store assignment artifacts
View eRubric
Electronic Assignment Workflow for GradeMark
Staff Workflow
Student Workflow
Create eSubmission Link
Create eRubric
Submit assignment
View Originality Report
Complete eRubric
Receive Digital Receipt
View Originality Report
Annotate document and
provide feedback
View mark
View feedback
Store assignment
artefacts
View eRubric
“The 2013 Horizon Project Australia Advisory Board agreed with the global
group that digital media literacy is not nearly pervasive enough in faculty
training.
There is a need for more training before being asked to teach, and for more
professional development opportunities once in the profession.
This key challenge is underscored by the widespread belief that most academics
are not leveraging emerging technologies for their own work, whether that be in
the classroom or in support of their own research.
While a lack of adequate training opportunities is a part of the challenge,
ultimately a change in the mind sets of disciplines and individual faculty will be
required, along with cultural shifts within institutions, before emerging tools and
technologies are routinely adopted and implemented as a matter of course.”
“Being agile as a teacher means staying
true to your purpose, fixed in your intent to
provide excellence in education but always
adapting and adopting new methods of
achieving that goal and where prompted to
by changes in the world around you: the
Internet demonstrates how agility is now a
key aspect of knowledge work—make it
part of your teaching.”
(Allen M, 2012)
Developing academic staff capacity for agile
teaching: a case study in implementing eMarking
workflows for student assessment in a large first
year Health Sciences unit
Office of Learning and Teaching
Seeding Grant Funded Project
Professional Development Approach
• Theories of change and change management within the
context of academic leadership and current higher
education change drivers
• Individual reflection on self as well as social and cultural
beliefs in relation to innovation, teaching and learning and
change
Professional Development Approach
• Exploration of intrapersonal skills such as self-talk to
deconstruct habitual thinking and its effect on performance
and acceptance of ICT innovations
• Develop an understanding of ICT system components as
opposed to “how to use” a program to support a shift in
thinking/approach from mastery t o adaptive responses.
Method
• Pre and post PD questionnaire combining measures relating
to attitudes to teaching and technology acceptance.
• Semi-structured interviews with 16 participants following the
first major assessment piece. Coded and analysed by a
qualitative researcher using NVivo.
• Two focus groups (12 participants in total) to check themes
that emerged from interviews.
Themes
• The PD influenced them in the following ways:
• Cognitive (majority of participants reported) - increased awareness by
bringing new knowledge, adding to existing knowledge, refreshing
knowledge. Knowledge on software applications, strategies, self
awareness to create creative thinking / planning for immediate and
future plans.
• Emotional (majority reported) – increased confidence, excitement,
enthusiasm, motivation, empowerment
• Practical (majority reported)– taking action to improve teaching
practice, manage both IT and teaching challenges, using IT and
teaching strategies, improving skills
Characteristics of the participant are key to
being agile in the adoption of ICT
• Characteristics demonstrated by participants included:
• Professional background / employment status /
professional identity
• Previous work and teaching experience
• Family / personal responsibilities and experiences
• Self efficacy, willingness to persevere
• Willingness to engage with others, mindful of others in
the unit / in regards to IT support
• Ability to employ intrapersonal strategies to cope with
changing environments
• ‘Can-do’ attitude
Discussion
•
•
•
•
•
Issues of self-selection
Sessional staff
Disciplinary background
Impact of workload/personal life on uptake
Change in behaviours requires ongoing
commitment/time to reflect
• Structured nature of the unit
Sources
•
Slide 3 images
•
•
•
•
•
Slide 12 image http://thatstotallytarot.com/wp-content/uploads/2011/01/work_stress.jpg
Slide 17
•
•
http://babynameobsessed.wordpress.com/2012/12/28/leilani-and-jax/the-cottesloe-beachperth-australia/
http://www.australia.com/explore/cities/perth/perth-surrounds.aspx
http://www.police.wa.gov.au/Aboutus/Policejurisdictions/tabid/1352/Default.aspx
Johnson, L., Adams Becker, S., Cummins, M., Freeman, A., Ifenthaler, D., and Vardaxis, N.
(2013). Technology Outlook for Australian Tertiary Education 2013-2018: An NMC Horizon
Project Regional Analysis. Austin, Texas: The New Media Consortium.
Slide 18
•
Allen, M.). Move quickly to adopt new approaches. Learning in Networks of Knowledge
Retrieved June 22, 2012, from
http://www.knowledgenetworklearning.net/concepts/pedagogy/teachingagility/