RUNNING ONLINE
EXAMS
Ailsa Dickie (ITS)
Mathew Hiller (TEDI)
www.elearning.uq.edu.au
Page 2 of 32
Table of Contents
Contributions ....................................................................................................................... 5
Advantages and Disadvantages .......................................................................................... 6
Tips & Strategies ................................................................................................................. 9
Cheating .............................................................................................................................. 9
Cheating.......................................................................................................................... 9
Blackboard course – exam setup ...................................................................................... 10
Hide links to Content Areas ........................................................................................... 10
Making messaging tools unavailable ............................................................................ 11
Create a new Content Area........................................................................................... 13
Creating an exam .............................................................................................................. 14
Creating questions and pools ............................................................................................ 16
Question randomisation .................................................................................................... 16
Options.......................................................................................................................... 16
Answer Randomisation ................................................................................................. 17
Question Randomisation ............................................................................................... 18
Randomised pool .......................................................................................................... 18
Randomised pool per question ..................................................................................... 19
Deploying a test (exam) .................................................................................................... 20
Recommended settings – Invigilated exam ....................................................................... 21
Recommended settings – Non Invigilated exam ............................................................... 23
Checking your exam.......................................................................................................... 25
Questions and answers................................................................................................. 25
Test Options.................................................................................................................. 25
Test exam deployment .................................................................................................. 26
Creating a copy of the exam ..................................................................................... 26
Adaptive release ....................................................................................................... 27
Exam checklist .......................................................................................................... 27
Student technical rehearsal ............................................................................................... 27
Invigilated exams .............................................................................................................. 28
Preparing the computer lab for the exam ...................................................................... 28
10-15 minutes prior to the exam ................................................................................... 28
During the exam ............................................................................................................ 28
Grade Centre – submitted exam ............................................................................... 28
Page 3 of 32
Exam link – submitted exam ..................................................................................... 29
Non-Invigilated exams ....................................................................................................... 29
Monitoring exams “Mission Control” .............................................................................. 29
Checking student progress. ...................................................................................... 29
Further Resources............................................................................................................. 31
Technical Support ......................................................................................................... 31
Faculty Educational Designers ...................................................................................... 31
Technical Resources..................................................................................................... 31
eLearning Blog .............................................................................................................. 31
UQ eLearning Twitter .................................................................................................... 32
Pedagogical Resources ................................................................................................ 32
ITS Workshops ............................................................................................................. 32
TEDI Workshops ........................................................................................................... 32
Page 4 of 32
Contributions
Thank you to Ms Jacqueline Bond and Ms Judith Burrows from the School of Pharmacy for
their advice and practical information on running online exams.
Thank you to Ms Lauren Hives, eLearning Applications Administrator, Bond University for
permission to use their Online Exam Supervisor‟s Manual.
Scope
This guide covers the running of online invigilated and online non-invigilated examinations
using the Blackboard Test (quiz) tool.
Due to the capabilities and scope of the Blackboard Test tool this guide covers the following
online exam types and assumed conditions:
Non-invigilated online exams which are to be treated as a „take home‟ exam.
Invigilated exams which are to be treated as an „open book‟ exam.
Note: In order to establish „closed book‟ exam conditions additional restrictions and
configuration of the computer itself would be required and this is beyond the scope of this
guide. Solutions are currently under investigation with some preliminary strategies for oncampus lab based exams outlined in „Custom Lab Computer Set-up (Images)‟ later in this
document.
Page 5 of 32
Quick Checklist

Computer lab booked/requested if the exam is to be run on campus.
o
o

Helpdesk ticket logged with your local desktop support.
o
o
o








Centrally controlled: as part of your normal exam request.
Faculty labs: your local desktop support.
Required exam image (software available, internet access?) for lab
computers.
Request the exam image on a lab machine for testing prior to the exam.
Request support on the day 15 minutes before and after the exam starts.
Quick access by phone for the rest of the exam.
Test options have been checked taking into account recommended settings.
Correct points have been assigned to questions.
Questions and Correct Answers have been checked for errors.
Tests have been review by a peer.
Release rules have been checked (time and date).
(Optional) Other site content areas have been hidden for the duration of the exam.
(Optional) All tools that allow messaging in Blackboard have been made
unavailable.
Exam has been checked on a lab machine, with the exam image, with content
areas hidden and with messaging options unavailable.
Advantages and Disadvantages
Online exams come with several advantages and disadvantages. These are outlined:
Advantages





The potential to increase the variety of question types that you can ask.
Reduce the administrative marking load (if using computer marked question types).
Provide a more familiar typed input method so that students can type answers
rather than handwrite them (Most students rarely hand write assessment
submissions anymore and are becoming less accustomed to writing for long
periods of time – hand cramps!)
Automatic and computer assisted processing enabling more timely feedback and
analysis of validity.
Reduce the consumption, sorting and movement of paper.
Disadvantages



Increased technical complexity of the examination environment requiring reliable IT
services and IT support.
Increased need for electronic security.
The need for technologically literate examination invigilators.
Page 6 of 32
Exam Modes
Particular Modes of online exams carry with them given characteristics. Examiners should
take this into consideration these characteristics when planning and determining exam
weighting and question types.
The two main modes of online exam covered in this guide are:
Non-invigilated
Conditions – Take home exams. Practically, you are not able to ensure that integrity of exam
conditions or the identity of the person undertaking the exam in non-invigilated scenarios.
However you can monitor student progress for time limited take home exams conducted
online and provide assistance in real time.
Invigilated Exams
In general invigilated online exams undertaken in campus computer labs should be treated
as an „open book‟ exam.
Using the default configuration found in most computer labs prevents the control of access to
network and internet based information and communication services. As such without major
changes to the computer lab environment a „closed book‟ style of exam is not reliably
achievable.
Guidelines for Effective Practice
Computer enhanced forms of assessment require sound principles for practice across
pedagogical, process and technology domains.
1. Have a clear purpose in mind for undertaking the assessment:
a) Define the learning outcome(s) you want students to demonstrate.
b) Consider suitable learning activities that will allow students to demonstrate
learning.
2. Choose appropriate technology to achieve your purpose (assistance is available from
various learning support services). The nature of the learning activities will inform the
required features/capabilities of the technology.

Be careful to design your overall workflow and choose technologies that will allow
you to advise requirements, receive submissions, assess, moderate, and provide
feedback efficiently and effectively.
3. Check the assessment is:
a) Valid (ensure it is measuring the desired learning outcomes and not is confounded
by other factors).
b) Reliable (test/pilot the assessment questions, the assessment process, response
options/methods and the technology)
c) Fair and Equitable (in terms of student workload, availability of equipment,
usability/ease of use, accessibility for deaf, colour blind, physical movement
disorders and provision of disability support)
4. Follow security principles for handling assessment questions, marking guides/model
answers and student responses that include ensuring (especially important for high stakes
events):


Confidentiality (keep material on a need to know basis restricted to authorised
staff/students using IDs, passwords, encryption)
Integrity (cannot be modified undetectably)
Page 7 of 32

Availability (correct functioning, available, reliable. Be aware of planned IT outages.
Have back-up and contingency plans in place, on-site IT help, secondary/spare
equipment, bookings for alternative venues/times)
 Authenticity (of materials, and the identity of the people involved)
 Non-repudiation (e.g. students cannot deny they sat/submitted the item)
 Ensure legal conditions/restrictions are abided (copyright, privacy, confidentiality,
sensitive materials)
5. Inform:
a) Students and instructors/tutors/invigilators of the assessment requirements, the
form it will take, processes, any conditions, the technology to be used/set-up
requirements, the location, time and duration of the assessment.
b) IT support personnel of the nature/technology, location, time, duration of the
assessment event (especially for time-limited summative e-exams).
6. Train:
a) Students in the skills needed for using the technology, processes and formats. Provide
opportunities for practice runs prior to a summative event.

For example - students should complete formative and/or small value summative
online tests (quizzes) regularly during the semester so they are familiar with the
technology and process for completing high steaks online exams.
 For non-invigilated exams run a short technical rehearsal (15 minutes) at least a
week before the exam.
b) Instructors/tutors/invigilators in the technology and in basic trouble shooting.
7. Review the assessment

Revisit the validity and reliability of the assessment (e.g. via item response
analysis, learning analytics, peer review).
 Evaluate human and technology factors (process, ease of use, stability, security
etc).
Address any concerns by making changes to the implementation for next time it is offered.
Page 8 of 32
Tips & Strategies


Students should complete formative and/or summative online tests (quizzes) regularly
during the semester so they are familiar with the technology and process for completing
online tests.
For non-invigilated exams run a short technical rehearsal (15 minutes) at least a week
before the exam.
Cheating
No system is fool proof and the opportunities for cheating must be considered.




Brainstorm with your teaching staff how students might cheat on your exam. i.e Can the
questions be googled?
Invigilated exams should be run the same way as paper based exams and treated as
open book (depending on computer setup).
Non-invigilated exams should treated as a take home exam when planning, determining
exam weighting and question types.
Question randomisation, question banks, one question per page and time pressure can
be used to make the ability to cheat more difficult.
Cheating
Know methods of cheating include:







A student leaves the examination early without submitting the exam and then
completes it on another computer.
A student saves notes in a draft discussion post or email and accesses it during
the exam.
Small electronic devices, doubling as articles of clothing such as watches or on key
chains with notes and stored data
Instant messaging between computers
The student in the room does not save or submit throughout the exam period
allowing a person outside the exam room to log on as the student and complete
the examination
A student makes a false claim of technical difficulties
Unauthorised software or websites are accessed.
Page 9 of 32
Blackboard course – exam setup
For an exam you may need to make the following changes to your course:



Hide links to Content Areas. (Invigilated exams).
Make tools that allow messaging between students unavailable. (Invigilated
exams).
Create a new Content Area for your exam. (Invigilated and optionally for noninvigilated).
Make these changes when you are testing your exam and just before the start of you exam.
Clearly communicate with your students well in advance when course materials and tools
will not be available.
Hide links to Content Areas
For invigilated exams you may want to remove student access to course resources.


Click on the Content Area (Learning Resources and Assessment).
Select Hide Link from the drop down list.

A
(hidden) icon will be displayed.
Page 10 of 32
Making messaging tools unavailable
For invigilated exams all Blackboard tools that allow students to communicate with each
other should be made unavailable.
Note: Exams should still be invigilated carefully, as no system is fool proof and students will
find new ways to cheat.

From the Control Panel select Customisation > Tool Availability

Uncheck the check boxes for Discussion boards, Blogs, Wikis, Messaging, Email,
Collaboration, Chat, and Groups.
Page 11 of 32

Click on the
(Submit) button.
Page 12 of 32
Create a new Content Area
If you hide the course Content Areas (Learning Resources and Assessment) you will need to
add a new Content Area for the exam.


Click on the
(plus) button.
Select Content Area from the drop down menu.


Enter the title in the Name textbox.
Check the Available to Users checkbox.

Click on the

Hover over the content area.

Click and drag the
position.
(Submit) button.
(double arrow) to the left of the Content Area to the required
Page 13 of 32
Creating an exam
There are three steps to creating a test (exam):
1. Create the test.
2. Add questions to the test. (Refer Tests, Quizzes and Pools tool page)
3. Deploy the test. (Refer p10)

From the Control Panel, select Course Tools > Tests, Surveys and Pools

Click on the Test link.

Click on the
(Build Test) button.
Page 14 of 32

Enter a descriptive name for the test in the Name textbox.
Note: The title should be the same as for the assessment item in the ECP.


Enter further information in the Description textbox.
Optionally instructions for the students on completing the test in the Instructions
textbox.

Click on the
(Submit) button.
Page 15 of 32
Creating questions and pools
It is recommended that you use only basic question types in an exam. i.e. Multiple choice,
Multiple answer and Calculated numeric. Students must be familiar with the question types
used in the exam.
Avoid:
Fill in the Blank: Students will be marked wrong if they add spaces, mis-spell words.
Hot Spot: This question type uses coordinators for the answer, these coordinators can be
different in different browsers.
Refer to the Quizzes, Tests and Pools tool page for instructions on creating individual
question types and pools.
Question randomisation
Options
Question randomisation helps make cheating more difficult.
Invigilated exams: By having question answers and question order randomised it makes it
difficult for a student to copy the answers of students on nearby computers.
Non - invigilated exams: By having question answers randomised, and question order
randomised and the use of question pools makes it more difficult for students to confer.
Page 16 of 32
Answer Randomisation
When creating questions, under Options:

Check the Show Answers in Random Order checkbox.
Page 17 of 32
Question Randomisation
This option is chosen when you deploy the test (exam).
Randomised pool
One random block: For example you may set your test so it displays 50 out of possible 150
questions in one pool.

Create a pool of questions (refer Quizzes, Tests and Pools).


Click on the
(Reuse Question) button.
Select Create Random Block from the drop down menu.


Select the All Pool Questions radio button.
Check the required pool checkbox.

Click on the
(Submit) button.
Page 18 of 32

Enter the Number of Questions to display.

Click on the
(Submit) button.
Randomised pool per question
Repeat the above questions creating a pool per question and then using a random block for
each question.
Page 19 of 32
Deploying a test (exam)

Open a Content Area.

Click on the

Select the required test from the drop down menu.

Click on the
(Assessment) button.
(Submit) button.
Page 20 of 32
Recommended settings – Invigilated exam
The below are recommended setting for summative invigilated tests. i.e. Usually exams
supervised in a Computer lab. Refer to the Running Online Exam guide for further
information.
Note: Your test settings may need to differ depending on your context and/or the learning
outcomes for the test.
Recommended Browser: It is recommended students use Chrome (a recent version).
Setting
Recommendation
Comments
Description
Optional
Instructions can be given verbally or written
on the whiteboard.
This description is visible while the students
take the test and should be kept short as
otherwise it will cause extra scrolling.
Open Test in New Window NO
Make the Link Available
This opens a new window and can make
navigation confusing for students.
YES, has the advantage of allowing students
to access other resources on the site.
YES
Use the Display After setting to set WHEN
students can access the test.
Add a New Announcement NO
for this Test
The automatic announcement is not very
informative; it is recommended you write
your own.
Multiple Attempts
OFF
Students will still be able to re-enter the test
(if Force completion is OFF) if they lose their
Internet connection.
Students should be directed to inform the
supervisor if they have a problem with the
test attempt (Refer Troubleshooting Tests
Guide)
Force completion
OFF
This ensures that students will be able to reenter the test if they lose their Internet
connection or browse away from the page.
Set timer
ON
Auto Submission, OFF
An elapsed time clock will be visible to
students during the exam.
Students should be given a verbal warning
can the exam time is almost up. Students
should be verbally told to submit and should
not be allowed to leave the exam room until
their submission has been checked.
Display After & Until
ON
Use these options to limit when the test link
is available.
Password
ON
The password should be entered by the
exam supervisor. This ensure the student
who completes the exam is in the exam
room (it is possible to log into Blackboard
twice and someone out the exam room to
complete the test)
Page 21 of 32
Setting
Recommendation
Test Availability Exceptions OPTIONAL
Comments
Can be used to give individual students or
groups of students:


Extra time (i.e. students with a disability)
Different test availability (i.e. students
who are not completing the test at the
same time as the rest of the students).
Due Date
ON
The date the test was completed / due is
shown to students in My Grades.
Self Assessment Options
Include this Test in
Grade Centre Score
Calculations, ON
The marks will be automatically recorded in
Grade Centre.
Test Feedback
(rule 1)
On Submission
Score, OFF
All Answers, OFF
Correct, OFF
Submitted, ON
Feedback, OFF
Show Incorrect
questions, OFF
Note, giving only information on what
answers were submitted when the test is
submitted gives you an opportunity to edit
the marking if there is a problem with a
question.
Test Feedback
(rule 2)
On Specific date
Score, ON
All Answers, ON
Correct, ON
Submitted, ON
Feedback, ON
Show Incorrect
questions, ON
Note, do not give students an opportunity to
copy the questions if you want to reuse
them. i.e. only turn the Score ON.
Presentation Mode
One at a Time, ON
Prohibit Backtracking,
OFF
This option and randomising the questions
makes it more difficult for students to cheat
by looking at a neighbour’s screen.
Prohibiting backtracking is not recommended
as it will cause the student to be thrown out
of the exam if they use the browser back
button.
Randomise Questions
YES
Changes the order of the questions, which
means that students will likely be working on
different questions as they progress. This is
useful to help prevent cheating by looking at
a neighbour’s screen.
Page 22 of 32
Recommended settings – Non Invigilated
exam
The below are recommended setting for summative non invigilated tests. i.e. Usually tests
that students at home on their own machines. This type of test is usually treated the same as
a Take home exam. Refer to the Running Online Exam guide for further information.
Note: Your test settings may need to differ depending on your context and/or the learning
outcomes for the test.
Recommended Browser: It is recommended students use Chrome (a recent version).
Setting
Recommendation
Comments
Description
Recommended
Allows you to give information to the
students before they open the test.
This description is visible while the students
take the test and should be kept short as
otherwise it will cause extra scrolling.
Open Test in New Window NO
Make the Link Available
This opens a new window and can make
navigation confusing for students.
YES, has the advantage of allowing students
to access other resources on the site.
YES
Use the Display After setting to set WHEN
students can access the test.
Add a New Announcement NO
for this Test
The automatic announcement is not very
informative; it is recommended you write
your own.
Multiple Attempts
OFF
Students will still be able to re-enter the test
(if Force completion is OFF) if they lose their
Internet connection.
Students must have a procedure to follow if
there is a problem with their test attempt,
such as accidental submission. i.e. A contact
number for the coordinator. (Refer
Troubleshooting Tests Guide)
Force completion
OFF
This ensures that students will be able to reenter the test if they lose their Internet
connection or browse away from the page.
Set timer
ON
Auto-Submit
Submission, ON
An elapsed time clock will be visible to
students during the exam.
The students’ exams will automatically be
submitted once the exam time is up.
Display After & Until
ON
Use these options to limit when the test is
available.
Password
OFF
Test Availablity Exceptions OPTIONAL
Can be used to give individual students or
groups of students”


Page 23 of 32
Extra time (i.e. students with a disability)
Different test availability (i.e. students
who are not completing the test at the
same time as the rest of the students).
Setting
Recommendation
Comments
Due Date
ON
The date the test was completed / due is
shown to students in My Grades.
Self Assessment Options
Include this Test in
Grade Centre Score
Calculations, ON
The marks will be automatically recorded in
Grade Centre.
Test Feedback
(rule 1)
On Specific date
Score, ON
All Answers, ON
Correct, ON
Submitted, ON
Feedback, ON
Show Incorrect
questions, ON
Note, giving no feedback when the test is
submitted gives you an opportunity to edit
the marking if there is a problem with a
question.
Presentation Mode
One at a Time, ON
Prohibit Backtracking,
OFF
This option and having a time limit makes it
more difficult for students to copy/print the
questions.
Prohibiting backtracking is not recommended
as it will cause the student to be thrown out
of the exam if they use the browser back
button.
Randomise Questions
YES
Changes the order of the questions, which
means that students will likely be working on
different questions as they progress. This is
useful to help prevent cheating by sharing
correct responses to numbered questions.
Note, do not give students an opportunity to
copy the questions if you want to reuse
them. i.e. only turn the Score ON.
Page 24 of 32
Checking your exam
Questions and answers
Have as many people as possible check the questions and answers



Hover over the exam title.
Click on the (arrow).
Select Edit the Test from the drop down list.
Test Options
Have someone else check that you have correctly followed the Recommended Settings
above.



Hover over the exam title.
Click on the (arrow).
Select Edit the Test Options from the drop down list.
Page 25 of 32
Test exam deployment
Have as many staff as possible check the exam by completing the exam as a student (refer
to the Become student tool page).
For invigilated exams also check completing the exam in the computer lab with the computer
image that will be used for the exam (organise this with desktop support).
Create a copy of the exam and deploy it with a password and the same settings as the
actual exam. For extra security use adaptive release to ensure only the dummy students
used by the Become Student tool can complete the test.
Creating a copy of the exam

From the Control Panel, select Course Tools > Tests, Surveys and Pools


Click on the Test link.
Hover over the exam title.


Click on the
(arrow).
Select Copy from the drop down list.
Also edit the exam name.
Page 26 of 32
Adaptive release
Deploy the exam in a content area, ensure that the test exam has a password and the same
settings as the actual exam.

Hover over the test exam title.


Click on the
(arrow).
Select Adaptive Release from the drop down list.

Enter the dummy student username (nc_student1 or nc_student2i) in the
Username textbox.


Click on the
(Submit) button.
Complete the exam as a student using the Become Student tool.
Exam checklist







Correct spelling and punctuation.
Understanding
Level of difficulty
Simplicity / difficulty of questions
Correct level of feedback given.
Points are correctly assigned.
The correct answer for each question was accurately assigned.
Student technical rehearsal

For non-invigilated exams run a short technical rehearsal (15 minutes) at least a
week before the exam.
o
o
o
o

The test used in the rehearsal should have the same test settings, question
types as the actual exam.
The rehearsal will allow you to check the exam settings.
The rehearsal will allow students to check their computer setup and ensure
they are familiar with the process.
Students should practice losing their internet connection (i.e. close the
browser) and returning to the test.
Also use this technical rehearsal to practice monitoring the exam.
Page 27 of 32
Invigilated exams
Preparing the computer lab for the exam



Ensure all computers are turned on and working.
Ensure the students only have access to required software.
Write the permitted software and hardware on the whiteboard.
10-15 minutes prior to the exam





Open the course Grade Centre on the staff computer.
Once students are seated have them login to their computers.
Have students open Blackboard and the exam.
The exam supervisor then should enter the exam password into all tests.
Inform student to raise their hand inform exam supervisor if they experience any
technical difficulties.
During the exam


The room should be patrolled constantly for prohibited software and website
access.
Exams must be all checked for submission before students can leave.
o
o
Check in Grade Centre on the staff computer. OR
At the student’s desk by clicking on the exam link.
Grade Centre – submitted exam
Page 28 of 32
Exam link – submitted exam
Non-Invigilated exams
Ensure there is a manned phone (or phones) so students can contact support if they have
any technical problems during the exam.
It is also possible to use Grade Centre to monitor student progress during the exam.
Monitoring exams “Mission Control”




Staff should monitor up to 30 students each and all work in the same room.
Students should submit a contact number before the exam.
Have a phone per staff member and give students all the phone numbers.
Keep a spreadsheet to record student progress.
Checking student progress.

From the Control Panel select Grade Centre > Full Grade Centre.

Submitted exams will be displayed with a

Exams in progress will be displayed with a
Page 29 of 32
(green explanation) icon.
(editor) icon.

Hover over the


Click on the
(arrow).
Select the Attempt.

Check which questions have been answered.
(editor) icon of the required student’s exam.
Page 30 of 32
Further Resources
Technical Support
Helpdesk staff - Susie Gardner Brown, Reece Hardy and Michael Luton
bbadmin@its.uq.edu.au or Ph 336 56000 Option 1 or itsupportdesk@its.uq.edu.au
Faculty Educational Designers
Faculty
Educational Designer
Email
Arts
Inge Matt
i.matt@uq.edu.au
BEL
TBA*
EAIT
Esther Fink
e.fink@uq.edu.au
Phone
336 57390
Health Sciences TBA*
SBS
Julie McCredden
j.mccredden@uq.edu.au 336 56641
Science
Marnie Holt
m.holt2@uq.edu.au
336 58869
* Contact TEDI for educational design support 3365 2788.
Technical Resources
http://www.elearning.uq.edu.au/content/tools-a-z-guide
A range of resources are available on our website, including self-help text based guides,
self-help videos, teaching and learning quick-guides and information on known technical
issues. Updates are also available on the progress of elearning projects such as the
implementation of new tools and upgrades to current systems. This site is currently being
redeveloped to improve
eLearning Blog
The focus of this blog is to provide information on what's new in eLearning. There will be
regular posts on topics such as new eLearning initiatives, progress on eLearning projects,
training news, technical issues, resources, tips and tricks and tutorials. The blog will include
contributions from all ITS Teaching and Learning Support team members as well as guest
posts from Faculty Educational Designers.
http://www.elearning.uq.edu.au/content/welcome
Page 31 of 32
UQ eLearning Twitter
https://twitter.com/elearningUQ
Pedagogical Resources
www.tedi.uq.edu.au
Pedagogical resources for eLearning are provided by the Teaching and Educational
Development Institute (TEDI).

Course Site Framework An overview of the elearning tools at UQ available.
ITS Workshops
A range of technical eLearning workshops are offered throughout the year. For descriptions
of the workshops, dates and to enrol, visit
https://staffdevelopment.hr.uq.edu.au/program/TALTEC
TEDI Workshops
A range of pedagogical elearning workshops are offered throughout the year.
For descriptions of the workshops, dates and to enrol, visit
https://staffdevelopment.hr.uq.edu.au/program/LRNSPA ..
Page 32 of 32