Is Online Assessment Safe ?
Is Online Assessment Safe ?
Our Journey
• SMU started to have controlled environment for Online Exams in 2005
• What we have tried ?
• 2005 ‐ OS, IP Address, User Profile
• High Manual Effort
• 2006 – 3rd Party Software like Software Secure
• High maintenance effort
• Frequent testing whenever computing environment changes
• Clashes with Anti‐Virus software
• Slow technical support from overseas
• 2007 – Self Made Exam Browser
Integrated Information Technology Services (IITS)
Jun 2011
1
Is Online Assessment Safe ?
Benefits of Online Assessment
• Easier to Setup Quiz & Changes
• No printing/re-printing
• Select questions based on categories and question sets
• Keep track of Questions Usage
• Easy to tell what questions have been used
• Easy to shuffle questions and MCQ options
• Harder for students to copy even if they sit close to each other
• Less Marking
• MCQ, T/F, Fill in the Blanks, Picture Selection auto-marking, only need to
mark essays, plus do not need to deal with horrible handwriting
• Quick Analysis and Structured Report
• % of right and wrong, easy to tell strong and weak areas
Integrated Information Technology Services (IITS)
Jun 2011
2
Is Online Assessment Safe ?
Our Users’ Requests
• Closed Book
• Block all access to Course Materials, Internet, Communications
• Semi‐Open Book
• Block access to Internet and Communications but not Course Materials
• Open Book
• Block communications but not others
Communications (Email, Chat, Social Media and … etc)
Integrated Information Technology Services (IITS)
Jun 2011
eLearning Forum Asia 2011, NTU
Singapore
3
1
Is Online Assessment Safe ?
Is Online Assessment Safe ?
What we have considered ?
• Wireless Signal Jammer
• License, public network (Wireless@SG), 3G
• No precise control
• Not portable
• Controlled Computer Lab
• Not scalable
• Respondus Lockdown Browser
• Only cater to Quiz and cannot overcome network connection break
Integrated Information Technology Services (IITS)
Jun 2011
4
Is Online Assessment Safe ?
Challenges we face
• New software can break the controls
• Korean Toolbar can allow pop-ups in our Exam Browser
• French OS can achieve Copy and Paste
• Exam Browser need maintenance along the way
• New Browser or Browser Version or Add-ons/Toolbars
• New OS
• Mac Laptop without Bootcamp
• Some laptops just cannot have this client software
Integrated Information Technology Services (IITS)
Jun 2011
5
Is Online Assessment Safe ?
Actions we take for Online Exams
• Seating arrangement to position the ‘high risk’ group in front
of instructor
• Randomized the questions and/or answers
• Invigilator is still needed, we provide easy way to check for
Exam Browser
• Change Password after exam started
• Prevent re-entry if student close the Exam Browser and switch to normal
browser
Integrated Information Technology Services (IITS)
Jun 2011
eLearning Forum Asia 2011, NTU
Singapore
6
2
Is Online Assessment Safe ?
Our Exam Browser
Is Online Assessment Safe ?
Special Features in our Exam Browser
• Auto-Update to latest version
• Easy to deploy latest changes
• Auto-Save
every 15 secs
• Ensure answers not lost when network connection breaks
• Ability to Backup and Restore student answers in Quiz
• Administrator overwrite control to retrieve answers when
necessary
• Detect Virtual Machines, Remote Computer & Desktop
Integrated Information Technology Services (IITS)
Jun 2011
8
Is Online Assessment Safe ?
How we support Exam Browser ?
• Intranet download link - Users Self Help
• Conduct a Mock Exam before Actual Exam
• Manage expectations
• Minimize support effort on actual day
• One technical engineer per room or 2 rooms (Ratio 1 : 50 or 100)
• Support starts 30 minutes before exam
• Last minute software update, troubleshooting
• Briefing to students and invigilator
• Support extends 30 mins after exam
• Check through submissions, troubleshooting
Integrated Information Technology Services (IITS)
Jun 2011
eLearning Forum Asia 2011, NTU
Singapore
9
3