Student Placement e-Assessment
Student Placement e-Assessment
Pilot Plan: Clinical Placement e-Assessment
Version History
Date
Version
Author
Description
01-10-2014
V 0.1
Karina Judson
Initial draft
17-11-2014
V 0.2
Karina Judson
Updated after comments from
Tamara
24-11-2014
V 1.0
Karina Judson
Released as Final
Review History
Name
Title
Date
Version
Tamara Page
Course Coordinator
21-11-2014
V0.2
Yvette Salamon
Lecturer
21-11-2014
V0.2
Trisha Franceschilli
eLearning adviser
01-10-2014
V0.2
Mark Wittervan
Program Manager, MyUni
19-11-2014
V0.2
Pilot Plan Template, version 1.0
Page 1 of 31
Student Placement e-Assessment
Distribution List
Name
Title
Trisha Franceschilli
eLearning Adviser, Sciences
Mark Wittervan
Program Manager, MyUni
Pilot Plan, version 1.0
Page 2 of 31
Student Placement e-Assessment
Terms and Abbreviations
Term/Abbreviation
Meaning
CIO
Chief Information Services
UICTAC
The University Information and Communications
Technology Architecture Committee
UICT
The University Information and Communications
Technology committee
CAS
It stands for Central Authentication Service. It is a
single sign-on protocol for the web. Its purpose is to
permit a user to access multiple applications while
providing their credentials (such as user id and
password) only once. It also allows web applications
to authenticate users without gaining access to a
user's security credentials, such as a password.
LDAP
It stands for Lightweight Directory Access Protocol. It
is an open, vendor-neutral, industry standard
application protocol for accessing and maintaining
distributed directory information services over the
network.
iOS
It is a mobile operating system developed by Apple
Inc. and distributed exclusively for Apple tablets and
smartphones.
Customisation
It refers to making changes to software code to tailor
the product to meet certain requirements.
Configuration
It refers to the tailoring of a piece of software to meet
certain requirements using existing functionality.
Plug-in
It is a software component that adds a specific feature
to an existing software application.
LTT
Learning Technologies Team.
TS
Technology Services.
eLA
eLearning Adviser.
LTSG
Learning Technologies Steering Group.
LTOG
Learning Technologies Operations Group.
References
Document
URL/Notes
‘Key questions to
answer about the pilot’
guidelines
http://www.adelaide.edu.au/learningtechnologies/files/key-questions-to-answer-about-thepilot.pdf
Student Placement eAssessment: Prefeasibility report
V1.0 approved by LTOG on 11th September 2014
Pilot Plan Template, version 1.0
Page 3 of 31
Student Placement e-Assessment
Table of Contents
1. Document Overview .................................................................................... 5
1.1. Purpose of this document ............................................................................................ 5
2. Pilot Scope .................................................................................................... 6
3. Pilot Objectives ............................................................................................ 8
4. Requirements ............................................................................................... 9
5. Pilot Resources .......................................................................................... 13
5.1. Pilot Reference Group ................................................................................................ 13
5.2. Pilot Participants......................................................................................................... 13
5.3. Pilot Team ................................................................................................................... 14
6. Pilot Costs................................................................................................... 16
7. Pilot Schedule ............................................................................................ 17
8. Success Criteria and Metrics ................................................................... 19
9. Pilot Evaluation .......................................................................................... 22
10. Pilot Risks ................................................................................................. 23
Appendix A – Use Cases ............................................................................... 25
Appendix B – Supportability Matrix............................................................. 29
Appendix C – Key Questions to Answer About the Pilot ......................... 30
Pilot Plan, version 1.0
Page 4 of 31
Student Placement e-Assessment
1. Document Overview
1.1. Purpose of this document
This plan defines the pilot main characteristics and identifies the activities, schedule, and
deliverables of the Student Placement e-Assessment pilot. The plan also discusses
resource requirements; interfaces and dependencies with other groups; and risk.
Pilot Plan Template, version 1.0
Page 5 of 31
Student Placement e-Assessment
2. Pilot Scope
This pilot is to test an e-Assessment system for students doing clinical placements.
The pilot participants will be a subset of the clinical lecturers, course coordinators and
students of the Master of Clinical Nursing doing placements in November and or
December 2014.
The pilot will include the following services and functions:
 Mahara (ePortfolio software) will be used as the software tool to facilitate the
clinical placement e-Assessment. The following ePortfolio functions will be
trialled:
o ePortfolio built by the participant students. If applicable, a contingency
plan will be developed to manage the risk of Mahara failing when the
students try to build their ePortfolio,
o ePortfolio submission for feedback or assessment by the participant
students. If applicable, a contingency plan will be developed to manage
the risk of Mahara failing when the students try to submit their ePortfolio,
o Feedback on student ePortfolio from within Mahara by the clinical lecturer
and course coordinator if the students have allowed them to do so. If
applicable, a contingency plan will be developed to manage the risk of
Mahara failing when the lecturer/course coordinator provides feedback on
a student ePortfolio from within Mahara,
o Student ePortfolio assessment by the clinical lecturer and course
coordinator,
o Exporting ePortfolio to a html file by the students.
 The criteria to assess clinical placement electronically using Mahara as a
supporting tool will be documented and communicated to the student participants
for the purpose of this trial;
 Student and staff guides on the use of Mahara will be developed for the trial
participants;
 Staff and students involved in this trial will be trained on the use of Mahara;
 Documentation on how ePortfolios can be used for clinical placement
assessment will be developed as result of the pilot.
The pilot will evaluate if the following benefits are achieved by using Mahara for clinical
placement assessment:
• The e-Portfolio will enable a ‘one-stop shop’ for the housing of all the material
required for clinical placement.
• The information can be shared with clinical lecturers as well as the course
coordinator.
• Easier to access
– what the student has or hasn’t completed to date.
– Attendance record
– Clinical issues
• In addition it can house other documents required to be shown prior to each
placement
– Drug calculations
– Hand hygiene
– CPR
Pilot Plan Template, version 1.0
Page 6 of 31
Student Placement e-Assessment
–
–
Immunisation status
Working with children clearance
The following will not be include in this pilot:
 Mahara direct integration with MyUni will be excluded from this trial;
 Mahara integration with CAS and LDAP for single-sign-on will be excluded from
this trial;
 Customisation (adding new functionalities) of Mahara is excluded from this trial.
This trial only includes configuration of Mahara;
 Using Mahara other than for students placements is excluded from this trial;
 Deployment of Mahara after this trial is not part of the scope.
Mahara will be self-hosted and Technology Services will provide a production
environment for this trial. This tool will be used as an stand-alone system, no direct
integration with any other University’s software will be included as part of this trial.
The pilot for the Master
November/December 2014.
of
Clinical
Nursing
participants
will
occur
in
A trial evaluation report will be produced by the beginning of February 2015 and will be
submitted to LTOG and LTSG. This evaluation report will recommend: do more trials,
not to continue with further trials or mainstream/implement this ePortfolio tool
within the University; as per trial results.
Regardless of trial success, the Mahara instance used for this trial will be
decommissioned after the trial has finished. The participant students will need to export
their ePortfolio to a html file after they have finished trialling Mahara.
Pilot Plan, version 1.0
Page 7 of 31
Student Placement e-Assessment
3. Pilot Objectives
The objectives of this pilot are to:
1. ensure that Mahara enables students to create a centrally located ePortfolio by:
o collecting evidence of competencies, core skills and attributes that will be
used when applying for professional registration;
o uploading/updating clinical placement evidence as well as reflecting on
them as per course guidelines;
2. ensure that Mahara allows students to periodically seek feedback on their
ePortfolio for clinical placement by sharing it with the corresponding lectures and
coordinators from within Mahara;
3. ensure that Mahara allows students to submit their ePortfolio for assessment by
sharing it with the corresponding lecturers and coordinators from within Mahara;
4. ensure that the corresponding lecturers and coordinators can access the
students’ ePortfolios from within Mahara and include their feedback on the
students’ ePortfolio within these ePortfolios when the students have allowed
them to do so;
5. ensure that lecturers find the ePortfolio assessment process easier that when
they assess the portfolio documents through MyUni where we have to download
them, reformate them to pdf, complete the assessment, return them to the
student and then store a copy on the S:Drive;
6. ensure that the students can access the lecturers/coordinator comments from
within their ePortfolios in Mahara;
7. ensure that students are able to download their ePortfolio content from within
Mahara in html format;
8. ensure that Mahara is easy to use by students and academics;
9. ensure that the Mahara configuration for this trial (design) meets the business
requirements;
10. ensure that Mahara works properly in the University’s infrastructure;
11. gather information about the training requirements on Mahara for lecturers and
students
12. gather information
requirements;
for
estimating
actual
hardware
and
supportability
13. gather information about the deployment process.
Pilot Plan, version 1.0
Page 8 of 31
Student Placement e-Assessment
4. Requirements
Id
Requirement
Use case
#
Related to Pilot
objectives
Priority
(M - Must/
S - Should/
C - Could)
(Please
refer to
Appendix A
of this
document)
Business Requirements
Provide ePortfolio assessment rules
UC 1
1
The instructor defines the assessment criteria (including weighting) for ePortfolio.
Objective 1
M
2
The instructor defines the ePortfolio assessment dates.
Objective 1
M
3
The instructor publishes the ePortfolio assessment criteria and dates in the course outline as
well as MyUni and ensure the students are aware of this.
Objective 1
M
Build Student Portfolio
UC 2
4
Students record their student information such as Name, Faculty, Program, contact details in
the ePortfolio.
Objective 1
M
5
Students record evidence of meeting the employer competencies criteria as mandated in their
course. Students will upload the completed employer competencies form provided by their
School as evidence of meeting these criteria.
Objective 1
M
6
Students record evidence of placement to demonstrate they are meeting the placement
objectives. Students will upload the completed summative assessment placement form
(including clinical facilitator endorsement) provided by their School as evidence of meeting
these criteria.
Objective 1
M
Pilot Plan Template, version 1.0
Page 9 of 31
Student Placement e-Assessment
7
Students record evidence of placement attendance. Students will upload the completed
placement attendance form provided by their School as evidence of students’ placement
attendance.
Objective 1
M
8
Students record evidence of meeting any other requirement needed prior to each placement as
specified in the program information for work placements. I.e.: For Master of Clinical Nursing,
the documents required to be shown by the students prior to each placement are: Drug
calculations, hand hygiene, CPR, immunisation status and working with children clearance
among others.
Objective 1
M
9
Students must show placement/competencies evidence in the form of pdf, word and image
files.
Objective 1
M
Submit ePortfolio for feedback or assessment
UC 3
10
Students select the required subset of ePortfolio information they are seeking feedback on (or
to be assessed) from within the ePortfolio tool.
Objective 2, 3
M
11
Students share the selected ePortfolio information with their clinical lecturer and course
coordinator from within the ePortfolio tool.
Objective 2, 3
M
Provide feedback on ePortfolio & ePortfolio Assessment
UC 4 &
UC 5
12
Lecturer/Coordinator is notified that a student ePortfolio is ready to be viewed.
Objective 4, 5
M
13
Lecturer/Coordinator looks through the different student ePortfolio artefacts that the student
has shared with him/her.
Objective 4, 5
M
14
Lecturer/Coordinator has the option to comment on any specific artefact from within the student
ePortfolio following the ePortfolio criteria assessment if the student has enabled feedback.
Objective 4, 5
S
15
Students are able to view the Lecture/Coordinator feedback from within their ePortfolios. A
notification is automatically sent to a student advising on the new feedback received.
Objective 6
M
Objective 7
S
UC 6
Submit an ePortfolio for accreditation
16
The student selects all the ePortfolio artefacts required for registration to the Nursing and
Midwifery Board.
Pilot Plan, version 1.0
Page 10 of 31
Student Placement e-Assessment
17
The student exports all required ePortfolio artefacts to a file format (html) from within Mahara
that they will submit to the National Board when applying for registration prior to graduating.
Objective 7
M
Others
18
Lecture/Coordinator
create
groups
within
the
ePortfolio
forums/discussions/communications with students and manage them.
tool
for
S
Non-functional Requirements
Accessibility
19
Students are able to use the ePortfolio tool from within any device despite the operating system
of the device
Objective 12, 13
M
20
Students can access the ePortfolio tool from anywhere
Objective 12, 13
M
Objective 9
M
User Access/Security
21
Trial participants need to authenticate to access their ePortfolios or any ePortfolio that has
been shared with them
Performance
22
The ePortfolio Tool must be accessible 24x7
23
The ePortfolio platform is in line with the University’s ICT governance principles
M
Objective 10
M
Support
24
Administration and user guides (for students and academics) will be developed for this trial
Objective 11
S
25
Pedagogic and Technical support to participants are available during the trial
Objective 11
M
26
Technical support from MyUni Support Team for the ePortfolio tool will be provided from
Monday to Friday 8am – 6pm
Objective 11
M
Usability
Pilot Plan, version 1.0
Page 11 of 31
Student Placement e-Assessment
27
The ePortfolio tool is intuitive and easy to use
Objective 8, 9
M
Privacy & Ownership
28
The ePortfolio content must comply with the University’s Copyright Policy
M
29
Ownership of and responsibility for ePortfolio intellectual property and content resides with the
students
M
30
Students must comply with the University’s Code of Conduct for Students
M
31
Students must comply with the University’s IT Acceptable Use and Security Policy
M
Scalability Requirements
32
The ePortfolio tool has already been used by other institutions (at an institutional level) with
over 20,000 students
Pilot Plan, version 1.0
Page 12 of 31
Objective 12
M
Student Placement e-Assessment
5. Pilot Resources
5.1. Pilot Reference Group
Resource
Role
Responsibilities
Tamara Page
Business Leader
Provide leadership and subject matter
expertise to ensure the project meets
the clinical placement assessment
requirements and the employer
competency requirements.
Yvette Salamon
Subject Matter Expert
Provide subject matter expertise to
ensure the project meets the clinical
placement assessment requirements
and the employer competency
requirements.
Mark Wittervan
Senior Supplier - LTT
Provide the MyUni support staff
resources
&
business
analyst
resource required by the project.
Trisha Franceschilli
eLearning educational Provide expertise in regard to the use
design support
of the ePortfolio platform and
ePortfolio design.
Karina Judson
Business Analyst
Ensure the ePortfolio meets the
business requirements and evaluate
the success of the trial.
5.2. Pilot Participants
Resource
Role
Students of the Master of Trial participants
Clinical
Nursing
Semester
2
2014
(Placements in November
2014)
 Kate Cameron
 Leslie Fischer
 Candace
Bavaresco
 Rebecca Mitchell
 Senthil Kandasamy
 Shahzana
Rhasbudin
 Shiv Kanda
 Kynan Coley
 Nargiz Valiullin
 Raul Aguilera
 Belinda Byrne
Pilot Plan Template, version 1.0
Responsibilities
Trial the ePortfolio
feedback on it.
and
provide
Page 13 of 31
Student Placement e-Assessment
Lectures/Coordinators of Trial Participants Trial the ePortfolio
the Master of Clinical (Staff)
feedback on it.
Nursing - Semester 2
2014
(Placements
in
November 2014):
 Yvette Salamon
 Tamara Page
and
provide
5.3. Pilot Team
Resource
Role
Responsibilities
Tamara Page
Pilot leader



Yvette Salamon
Subject matter expert



Trisha Franceschilli
eLearning educational 
design support



Pilot Plan, version 1.0
Provide leadership and subject
matter expertise to ensure the
trial meets the clinical placement
assessment requirements and
the
employer
competency
requirements;
Train the trial participants and
communicate
the
required
messages to the participants in
preparation to the trial;
Provide support to students and
lectures in regard to the
learning/teaching side of the
ePortfolio.
Provide subject matter expertise
to ensure the project meets the
clinical placement assessment
requirements and the employer
competency requirements;
Train the trial participants and
communicate
the
required
messages to the participants in
preparation to the trial.
Provide support to students and
lectures in regard to the
learning/teaching side of the
ePortfolio.
Provide expertise in regard to the
use of the ePortfolio platform and
ePortfolio design.
Create user accounts for the trial
participants;
Configure Mahara as per the
agreed design;
Train Tamara and Yvette
Page 14 of 31
Student Placement e-Assessment
Karina Judson
Business Analyst



installation 
Hong Chang
ePortfolio
support
Ruby Jing
Technical support
Pilot Plan, version 1.0

Gather business requirements;
Ensure the ePortfolio design
meets
the
business
requirements;
Evaluate the success of the trial
through surveys facilitated by the
pilot leader.
Install Mahara 1.9
designated server.
in
the
Create a brief administration
guide of Mahara for the MyUni
Support Team who will be
providing
technical
support
during the trial.
Page 15 of 31
Student Placement e-Assessment
6. Pilot Costs
ITEMS
Year 2014 Cost
Source of funds
Internal resources
Academics
$3,690 Faculty of
(marking relief – approx. Sciences
Health
100 hours)
eLA’s
$0 LTT
MyUni support
resources
$0 LTT
Business Analyst
$0 LTT
Hosting (Internal hosting)
Hardware
$0 Technology Services
Training
$0 LTT
Mahara Licences
$0 Open source software
Other
N/A
2
Articulate
Storyline licences
Parallel Software
TOTAL TRIAL
Pilot Plan, version 1.0
$1,098 Faculty of
Sciences
Health
$212 Faculty of
Sciences
Health
$5,000
Page 16 of 31
Student Placement e-Assessment
7. Pilot Schedule
Key Pilot Activities
Task #
Tasks
T1
Expected
Completion
Date
Responsible
Actual Start
date
Actual
Completion
date
Identify business requirements & pilot 15/09/2014
objectives
06/10/2014
Karina
15/09/2014
13/10/2014
T2
Set up infrastructure for Mahara
26/09/2014
03/10/2014
Technology
26/09/2014
Services
–
Infrastructure
team
26/09/2014
T3
Deploy and install Mahara
07/10/2014
10/10/2014
Hong
29/09/2014
30/09/2014
T2
T4
Configure Mahara
requirements
business 13/10/2014
31/10/2014
Trisha/Karina
13/10/2014
21/10/2014
T3
T5
Communicate participant’s students about 13/10/2014
the pilot objectives, duration of pilot,
training dates, what is expected from them,
where to get pedagogic support from,
where to get technical support from and
where to find support material to build an
ePortfolio in Mahara
17/10/2014
Tamara
Yvette
& 13/10/2014
17/10/2014
T3
T6
Give permissions to participants to access 16/10/2014
Mahara
17/10/2014
Trisha
14/10/2014
14/10/2014
T3
T7
Test Mahara instance – Test cases
21/10/2014
Project team
14/10/2014
30/10/2014
T4
Pilot Plan Template, version 1.0
Expected
Start Date
as
per
16/10/2014
Page 17 of 31
Dependencies
Student Placement e-Assessment
T8
Develop user guides on the use of Mahara 20/10/2014
for students
24/10/2014
Tamara
Yvette
& 22/10/2014
23/10/2014
T4
T9
Send login details to participant students
30/10/2014
03/11/2014
Trisha/Karina
05/11/2014
05/11/2014
T7, T6
T10
ePortfolio training to tutor and clinical 08/10/2014
lecture/coordinator
10/10/2014
Trisha
08/10/2014
08/10/2014
T3
T11
ePortfolio training to students
05/11/2014
05/11/2014
Tammy/Yvette
05/11/2014
05/11/2014
T4, T5, T6, T7,
T8, T9 & T10
T12
Participant’s students on placements
10/11/2014
21/11/2014
Participant
students
10/11/2014
21/11/2014
T13
Pilot testing
10/11/2014
05/12/2014
Pilot
Participants
10/11/2014
T14
Participants to export their ePortfolios to a 05/12/2014
html file format
05/12/2014
Participant
students
T15
Send a list of issues/defects raised by the 08/12/2014
academics/students during the pilot
08/12/2014
Tamara
Yvette
T16
Prepare survey for trial participants to 18/11/2014
evaluate the ePortfolio system
28/11/2014
Karina/Tamara
& Yvette
T17
Complete survey for Pilot
(student & lecturer feedback)
Evaluation 01/12/2014
05/12/2014
Pilot
Participants
T13, T16
T18
Prepare Lessons Learned Session to 08/12/2014
review pilot if applicable
09/12/2014
Karina
T13
T19
Hold Lessons
applicable
if 11/12/2014
12/12/2014
Karina
T13, T17, T18
T20
Put together Pilot Evaluation Report
15/12/2014
18/12/2014
Karina
T13, T15, T17,
T19
T21
Distribute Pilot Evaluation Report
19/12/2014
19/12/2014
Karina
T20
Pilot Plan, version 1.0
Learned
Session
Page 18 of 31
T10, T11, T12
T12, T13
&
T13
01/12/2014
T1, T4
Student Placement e-Assessment
8. Success Criteria and Metrics
Category
Measurement description
System
performance
Scalability of ePortfolio
Minimal/No impact on standard
operating environment
Minimal/No impact on University
computer standards
Stability
Good software security by identity
control and authentication within
ePortfolio as well as
student/academic control
Software robustness when
exposed to a high load of
documents and user interactions
at the same time
Reliability of ePortfolio tool
Sustainability of ePortfolio tool by
having good technical support,
user and administrators materials
as well as allowing LDAP, CAS
and LMS integration
Ease of installation and
configuration
User performance
User satisfaction
Target metric
Acceptable range
of values
Agree/Disagree [70% - 100% of the
technical support
staff for the trial
agree]
Agree/Disagree [70% - 100% of the
technical support
staff for the trial
agree]
Agree/Disagree [70% - 100% of the
technical support
staff for the trial
agree]
Agree/Disagree [70% - 100% of the
technical support
staff for the trial
agree]
Agree/Disagree [70% - 100% of the
technical support
staff for the trial
agree]
% of participants Less than 40% of
reporting an
participants
issues with the
reported an issue
ePortfolio
during the trial
Agree/Disagree [70% - 100% of the
technical support
staff for the trial
agree]
Agree/Disagree
Reduction of time for students to
submit a portfolio for clinical
placement
Agree/Disagree
Successful completion of all tasks
carried out within the ePortfolio
during the trial
Agree/Disagree
Ease of use for students
Students’ satisfaction on how
easy is to create an ePortfolio in
Mahara
Pilot Plan Template, version 1.0
Agree/Disagree
[70% - 100% of the
technical support
staff for the trial
agree]
[70% - 100% of the
participants
(students) for the
trial agree]
[70% - 100% of the
participants students for the
trial agree]
[70% - 100% of
participants agree]
Page 19 of 31
Student Placement e-Assessment
Business goals
Pilot Plan, version 1.0
Students’ satisfaction on how
easy is to share their ePortfolio
with the clinical lecturer and
course coordinator within Mahara
Students’ satisfaction on having
access to periodical academic’s
feedback within their ePortfolio in
Mahara
Students’ satisfaction on feel &
look as well as content
completeness of the ePortfolio
html file produced when exporting
it from Mahara
Ease of use for academics
Academic’s satisfaction on how
easy is to find the student
ePortfolio content to review
Academic’s satisfaction on how
easy is to provide feedback within
the student ePortfolio
Academic satisfaction on ease of
student ePortfolio marking in
comparison to the current paperbased portfolio marking practice
Speed
Participants’ satisfaction on quick
system response time when
adding/updating/uploading
information in the ePortfolio tool
Support satisfaction
Participants’ satisfaction on
academic and technical support
during the ePortfolio trial
Participants’ satisfaction on
training received before the trial
Participants’ satisfaction on the
content of the training material
and user guides for this ePortfolio
trial
Facilitate student learning &
achievement of competencies
Agree/Disagree
[70% - 100% of
participants agree]
Agree/Disagree
[70% - 100% of
participants agree]
Agree/Disagree
[70% - 100% of
participants agree]
Agree/Disagree
[70% - 100% of
participants agree]
Agree/Disagree
[70% - 100% of
participants agree]
Agree/Disagree
[70% - 100% of
participants agree]
Agree/Disagree
[70% - 100% of
participants agree]
Agree/Disagree
[70% - 100% of
participants agree]
Agree/Disagree
[70% - 100% of
participants agree]
[70% - 100% of
participants agree]
Assist staff in developing best
practices in blended learning
Agree/Disagree
Enhance formative assessment
Agree/Disagree
Enhance summative assessment
Agree/Disagree
Enhance academic/ student
communication during placement
Agree/Disagree
Agree/Disagree
Agree/Disagree
[70% - 100% of
student participants
agree]
[70% - 100% of
academic
participants agree]
[70% - 100% of
academic
participants agree]
[70% - 100% of
academic
participants agree]
[70% - 100% of
academic
participants agree]
Page 20 of 31
Student Placement e-Assessment
In addition, the resource implications and cost implications to the university will be
evaluated:
 Total cost of ownership (including infrastructure cost)
 Benefits and risks (including privacy and ownership)
Technical appropriateness will also inform the evaluation:
 Storage implications (Amount of storage/space required/used)
 Support implications
 Technical implementation if tool is mainstreamed
o LDAP integration
o LMS integration
Pilot Plan, version 1.0
Page 21 of 31
Student Placement e-Assessment
9. Pilot Evaluation
The pilot results will be evaluated using the following methods:
 Survey of participants (Students & academics).
 List of issues raised by the participants during the trial as well as how they were
resolved.
 Interview with the admin and appointed support staff about how the backend of
our Mahara instance performed during the trial (i.e. performance, storage).
 Interview with the appointed support staff to find out the issues raised to them in
regard to the ePortfolio during the trial
 Project Team Workshop after the participants have finalised the ePortfolio trial to
discuss Lessons Learnt (success & improvements)
 Cost/benefit analysis
A Pilot Evaluation Report will be produced taking into account the Success Criteria and
Metrics defined in this document and using the methods mentioned above for assessing
the pilot results.
This report will also summarise the successful outcomes, challenges experienced,
recommendations (do more trials, mainstream or not to continue with further trials) and
conclusions.
Pilot Plan, version 1.0
Page 22 of 31
Student Placement e-Assessment
10. Pilot Risks
Ref
#
Risk
1
Technical resources & academic staff (who
submitted the trial request) are not available
when needed
Likelihood
Consequence
Risk Rating
M
H
Moderate
Control


Status
As part of the Pre-feasibility study Open
approval, technical resources must
be allocated to the project
Careful communication (including
next activities and timeframe) to all
parties so as to ensure resources
are made available when needed.
2
If solution option is hosted externally, there
might be legal risks associated with
jurisdiction, privacy and security, particularly
with offshore offerings.
L
H
Medium

It is recommended to self-host Closed
Mahara. If this recommendation is
not approved and Mahara must be
externally hosted:
o Technology Services should
provide clear direction of how to
deal with cloud services, as
there is not a current policy for
cloud services at the University.
o Request the Service Level
Agreement from the company
providing the hosting service
and agree on the terms.
3
Future upgrades to the ePortfolio solution
and interoperability of this solution with other
ePortfolios is difficult or impossible to achieve
as result of changes made to the solution by
University’s technical resources.
L
M
Medium

When customising the solution, it is
recommended to use Plug-ins
instead of making direct changes to
the solution. Plug-ins could be
developed by the University’s
technical resources to get the
required additional functionality
Pilot Plan Template, version 1.0
Page 23 of 31
Closed
(customiz
ation
is
not part of
the trial)
Student Placement e-Assessment
following
the
development
guidelines of the open source
software.
4
Installation and configuration of Mahara will
take longer than expected and might delay
the trial execution
M
H
Moderate

Technology Service will provide Closed
support for installation if needed.
This activity has started before
than expected will give us some
contingency time to complete the
tasks
5
Difficulty using ePortfolio tool during trial
M
H
Moderate

Ensure adequate training is
complete before trial.
Ensure participants have been
communicated of what is expected
from them and are aware of trial
dates, ePortfolio assessment
guidelines and support provided to
them.

Open
6
The ePortfolio tool fails (is down) during trial assessment
L
H
Medium

Tamara
(Master
of
Nursing Open
coordinator) will advise students
that in the case of the ePortfolio
tool being down during the trial,
communicate this to her. She will
raise this with Trisha/Karina and be
advised when the tool is back up.
Then, she will communicate this to
students.
7
All ePortfolio information is lost and there is
no way to get it back as there is not a backup
process
L
H
Medium

The participant students will have Open
to re-upload all hard-copy evidence
to Mahara
Despite these risks the project would appear to be viable at this stage, and it would appear that each of these risks would be able to be
effectively mitigated
Pilot Plan, version 1.0
Page 24 of 31
Student Placement e-Assessment
Appendix A – Use Cases
Use Case ID:
Use Case Name:
Actors:
Description:
UC 01
Provide ePortfolio assessment rules
Instructors
Instructors communicate the ePortfolio assessment rules to the students
before they start working on their ePortfolios.
Preconditions:

ePortfolio for student placement is part of the curriculum of the
course of study.
Normal Course:

The instructor defines the assessment criteria (including weighting)
for ePortfolio.
The instructor defines the ePortfolio assessment dates.
The instructor publishes the ePortfolio assessment criteria and
dates in the course outline as well as MyUni and ensure the
students are aware of this.


Alternative Courses:
Frequency of Use:
Business Rules
NA
Once every semester




ePortfolio assessment criteria must be published in MyUni.
ePortfolio assessment criteria must be published in the course
outlines.
Staff must comply with the University’s Assessment for Coursework
Programs Policy
Staff must comply with the Modified Arrangements for Coursework
Assessment Policy
Assumptions:
This is done at the beginning of the semester.
Use Case ID:
UC 02
Use Case Name:
Actors:
Description:
Preconditions:
Build ePortfolio
Students
Students build their ePortfolios during the completion of their program to
demonstrate evidence of their placements and meeting employer
competencies.




Normal Course:



Pilot Plan Template, version 1.0
Students are enrolled in the course that contains an ePortfolio as
part of the assessment method.
Students are aware of the ePortfolio assessment criteria, dates and
any other related information.
Students have the employer competencies form, placement
objectives form as well as the placement attendance record
available.
Students have a username and password to log in to the ePortfolio
tool.
Students record their student information such as Name, Faculty,
Program, contact details in the ePortfolio.
Students record evidence of meeting the employer competencies
criteria as mandated in their course.
Students record evidence of placement to demonstrate they are
meeting the placement objectives.
Page 25 of 31
Student Placement e-Assessment



Frequency of Use:
Throughout the students’ completion of their program

Business Rules



Assumptions:



Use Case Name:
Actors:
Description:
Preconditions:
Submit ePortfolio for feedback or assessment during
the program of study
Students, Clinical Lecturers, Course Coordinator
Students build their ePortfolio and share it (or a subset of this information)
with their clinical lectures and their course coordinator for feedback
periodically or when requested by the lectures or needed by the students or
for assessment.



As part of the students’ placement assessment, students have
assembled an ePortfolio.
Clinical lecturers and course coordinator have access permissions to
the ePortfolio tool.
Students select the required subset of ePortfolio information they are
seeking feedback on (or to be assessed) from within the ePortfolio tool.
Students share the selected ePortfolio information with their clinical
lecturer and course coordinator from within the ePortfolio tool.
Frequency of Use:
Periodically or when requested by the lectures or needed by the students or
at the assessment dates throughout the students’ completion of their
program.
Business Rules
The students must submit/share the ePortfolio on the dates specified by
their lecturers/course coordinator and as outlined in the course outlines.
Assumptions:
Pilot Plan, version 1.0
Students will upload the completed employer competencies form as
evidence of meeting the employer competencies criteria.
Students will upload the completed summative assessment
placement form (including clinical facilitator endorsement) as
evidence of meeting the placement objectives.
Students will upload the completed placement attendance form as
evidence of students’ placement attendance.
UC 03

Normal Course:
Ownership of and responsibility for ePortfolio intellectual property
and content resides with the students.
Students must comply with the University’s Copyright Policy
Students must comply with the University’s Code of Conduct for
Students
Students must comply with the University’s IT Acceptable Use and
Security Policy
The ePortfolio tool to use is the one supported for this trial by the university:
Mahara
Notes:
Use Case ID:
Students record evidence of placement attendance.
Students record evidence of meeting any other requirement needed
prior to each placement as specified in the program information for
work placements. I.e.: For Master of Clinical Nursing, the
documents required to be shown by the students prior to each
placement are: Drug calculations, hand hygiene, CPR,
immunisation status and working with children clearance among
others.
Students must show placement/competencies evidence in the form
of pdf, word and image files
Clinical Lecturer/Coordinator will provide periodical feedback to students.
Page 26 of 31
Student Placement e-Assessment
Use Case ID:
Use Case Name:
Actors:
Description:
UC 04
Provide feedback on ePortfolio during the program of
study
Clinical Lecturers, Course Coordinator
Lecturers/Course Coordinators provide feedback on or assess the students
ePortfolios from within the ePortfolio tool.
Preconditions: 



Normal Course: 


Alternative Courses: 


Frequency of Use:
Use Case Name:
Actors:
Description:
Lecturer/Coordinator can access the ePortfolio of any student in his/her
class directly from the ePortfolio tool.
Lecturer/Coordinator looks through the different student ePortfolio
artefacts that the student has shared with him/her.
Lecturer/Coordinator has the option to comment on any specific artefact
from within the student ePortfolio following the ePortfolio criteria
assessment if the student has allowed feedback.
Staff must comply with the University’s Assessment for Coursework
Programs Policy
UC 05
ePortfolio assessment
Clinical Lecturers, Course Coordinator
Lecturers/Course Coordinators review the students ePortfolios from within
the ePortfolio tool
Preconditions: 




Pilot Plan, version 1.0
Lecturer/Coordinator is notified that a student ePortfolio is ready to be
viewed.
Lecturer/Coordinator looks through the different student ePortfolio
artefacts that the student has shared with him/her.
Lecture/Coordinator has the option to comment on any specific artefact
from within the student ePortfolio following the ePortfolio criteria
assessment if the student has allowed feedback.
Periodically or as per feedback seeking /assessment submission dates
Business Rules 
Use Case ID:
The lecture teaches a course that contains an ePortfolio as part of the
assessment method.
The course has an ePortfolio criteria assessment included in the course
outline.
The lecture and course coordinator provide feedback on the progress of
the student ePortfolio or assess the ePortfolio as part of the course
assessment methods.
The lecture and course coordinator have access (user name and
password) to the ePortfolio platform that contains the students
ePortfolios.
The lecture teaches a course that contains an ePortfolio as part of the
assessment method.
The student has shared/submitted the ePortfolio with the Clinical
Lecturer/Course Coordinator.
The course has ePortfolio criteria assessment included in the course
outline.
The lecture and course coordinator assess an ePortfolio as part of the
course assessment.
The lecture and course coordinator have access (user name and
password) to the ePortfolio platform that contains the students
ePortfolios.
Page 27 of 31
Student Placement e-Assessment
Normal Course: 


Alternative Courses: 


Frequency of Use:
Business Rules
Use Case ID:
Use Case Name:
Actors:
Description:
Staff must comply with the University’s Assessment for Coursework
Programs Policy
UC 06
Submit an ePortfolio for accreditation
Students, Nursing and Midwifery Board
Each student submits a hard copy of an ePortfolio demonstrating their
completion of the preregistration requirements as outlined in the University’s
program they are enrolled in to the corresponding National Board for their
profession on completion of the program of study. In the ePortfolio, the
student includes placement evidence as well as addresses the employer
competencies requirements.


Normal Course: 

Business Rules
Assumptions:
Pilot Plan, version 1.0
Lecturer/Coordinator can access the ePortfolio of any student in his/her
class directly from the ePortfolio tool
Lecturer/Coordinator looks through the different student ePortfolio
artefacts that the student has shared with him/her.
Lecturer/Coordinator has the option to comment on any specific artefact
from within the student ePortfolio following the ePortfolio criteria
assessment if the student has enabled feedback.
As per assessment submission dates
Preconditions: 
Frequency of Use:
Lecturer/Coordinator is notified that the student ePortfolio is ready to be
viewed.
Lecturer/Coordinator looks through the different student ePortfolio
artefacts that the student has shared with him/her.
Lecturer/Coordinator has the option to comment on any specific artefact
from within the student ePortfolio following the ePortfolio criteria
assessment if the student has enabled feedback.
The student has collected evidence of the completion of their clinical
placements and evidence of meeting all the employer competencies as
per the NCAS in a personal ePortfolio within an ePortfolio tool.
The student has completed his/her program of study.
The student applies for registration to the National Board of Nursing and
Midwifery as per his/her profession requirements
The student selects all the ePortfolio artefacts required for registration to
the Nursing and Midwifery Board
The student exports all required ePortfolio artefacts to a file format
(html) from within the ePortfolio tool that they will submit to the National
Board of Accreditation when applying for registration prior to graduating.
On completion of the student program
The student ePortfolio is used to support the application to become a
registered nurse within the Application for general registration for students
completing an approved program of study for the profession of Nursing and
Midwifery
Student has built his/her ePortfolio using the University’s ePortfolio tool
Page 28 of 31
Student Placement e-Assessment
Appendix B – Supportability Matrix
Name
Contact Details
Role
Responsibility
(i.e.: first point of
contact
when
raising
issues,
environment
support,
software
defects resolution)
MyUni Support
Mark,
Ruby
8313 3000, option 3
Karina, mark.wittervan@adelaide.edu.au
karina.judson@adelaide.edu.au
ruby.jing@adelaide.edu.a
Technical
Support
1st point of contact
Technical
Support
Escalations
Pedagogic
Support
Pedagogic
Support
for
students – first
point of contact
Tamara/Yvette
tamara.page@adelaide.edu.au
yvette.salamon@adelaide.edu.au
Trisha
Franceschilli
trisha.franceschilli@adelaide.edu.au Pedagogic
Support
Pilot Plan, version 1.0
Pedagogic
Support
Page 29 of 31
Student Placement e-Assessment
Appendix C – Key Questions to Answer About the
Pilot
The Pilot Reference Group needs to answer certain “Key questions” regarding sustainability,
demand, strategic alignment, risks/benefits and total costs of the software that has been
trialled.
Overall the Pilot evaluation will contain an assessment of the piloted software in terms of
issues such as:
Perceived value / effect (to be answered by the requesting academic / Innovations Analyst)
What did the pilot reveal regarding the value of the software to the students? Or where the
primary value was to staff what was the value of the software to the staff?
Where the software had an impact on educational outcomes what was the impact on
student learning as shown in the pilot?
Please provide an analysis of feedback regarding both student and instructors’ experience of
the software. The more detail that can be provided here the better.
Resource management (to be answered by the Innovations Analyst / Server Administrator)
Overall what did the pilot demonstrate regarding total costs / time/costs /and other savings –
Note: Learning Technology Team will need to develop resources to help staff determine this.
From the pilot what was learnt about the benefits / risks of the software?
System architecture/enterprise (to be answered by the System Administrator)
What was the software impact on the standard operating environment and Uni computer
standards?
What did the pilot find out about the scalability of the software?
What did the pilot find out about the reliability of the software – for academics and students?
For the server administrator?
What did the pilot find out about the sustainability of the software?
What did the pilot find out about the security of the software?
What did the pilot find out about the load of the software?
Pilot Plan, version 1.0
Page 30 of 31
Student Placement e-Assessment
Change management (to be answered by the LTT Change Manager)
What Change Management activities would be required if the software is approved and
implemented? Which teams will be involved? How will they be involved?
The Pilot Reference Group will answer these questions in a Pilot Report and include a
recommendation to the Learning Technologies Operations Group to:
 approve the software, or
 request an extension of the pilot, or
 not implement the software.
Pilot Plan, version 1.0
Page 31 of 31