Summary of Telephone and Email conversations about ExamSoft
Seven (7) respondents - 3 references from vendor, 4 from AACP Assessment SIG listserve, 6 current users of the system and 1 non-user whose school put the decision to purchase the product on hold due to the lack of power outlets in classrooms to support its students (140 students per class). The school will resume conversation about this product in the fall 2012 semester.

Most had used ParScore or other bubble sheet paper system before adopting . One had used another online testing application, QuestionMark, but had problems because it required test takers to be connected to the Internet throughout the exam. One is using PharmAcademic for mapping.

Most (4) had been using for less than one year. One had used for 3 years and was a beta-tester for
ExamSoft’s Pharmacy clients. Most had set up mapping categories and assigned them to questions but have not been reporting or utilizing this function yet.

Reasons for selecting ExamSoft

To ease the burden of use of Scantrons on faculty

Better security than other products used or tested

Longitudinal tracking of individual student outcomes

Mapping to outcomes by individual student, course, & program

Additionally, schools use it for

Course evaluations

Testing in self-study online courses

All either had or were adopting a mandatory laptop policy for students. One school also used in a campus computer lab as well as laptops in the classroom. One school gives students a laptop in year
1 and updates in year 3 for $500 per term technology fee [I’m collecting sample policies]

Implemented slowly with 2-5 courses in a pilot project one semester except for 1 school who had transferred from an online testing system

Best features of the system

Security/Preventing Cheating
Encrypted results, Saves every two minutes, Key-strokes tracking, off-line test taking, lock-down browser

Test creation
SofTeach-exam building-team taught courses, pool questions and approved by Coordinator
Easy to link to student learning outcomes
Can use to bring-in repeat questions to ensure retention
Very good for linking images (pdf or jpeg format)

Test implementation and grading
Ease of exam distribution,
Collective grading- students get raw score immediately
Multiple options for return of exam info from full exam to comments only on exam items missed.
1

Individual Feedback to students through Softscore– personally didn’t have to do it all - by creating comments sections for multi-choice items, short answers, and essays
Scoring- fill-in the blank- Very easy
Practice faculty can participate more fully in courses because they can create exams from home, monitor grades from home or off-site

Mapping and tracking outcomes:
Map the real time curriculum -Using the three levels of Bloom taxonomy, Appendices B,C,D
Ability to analyze the questions, by categories, some by performance level
Looking at individual level performance- using in Pharmacotherapy

Frees up resources – did cost analysis

Biggest challenges

Grade transfer function does not work with Sakai based system (like Laulima)

Start up is time consuming in the beginning – one day – two people- creating categories, adding faculty & courses

Two hours to train the faculty to create questions, exams – workshop

Resistance from some students- wanted scratch paper or copies of exam to mark; Did not want to put ExamSoft on their computers because it slows down other applications

Not enough electrical outlets in the test taking room

Students updating Software before exam

Satisfaction with ExamSoft tech support for exam takers and creators was very high but most were in the Eastern Time Zone. ExamSoft tech support can use Remote Desktop to help solve student problems

No academic dishonesty issues detected related to the use of ExamSoft

Other software considered
QuestionMark/Perception – did not have necessary security- Internet based examination caused problems during Internet outages
Respondus – did not integrate with Sakai, pricing
Criterion- did not integrate with Sakai, pricing

Administrators (persons who control access, setting up courses, mapping categories) are usually 1 or more faculty and 1-2 IT people
Assumptions
We have power outlets for all student laptops and they can all be used at the same time without blowing a fuse
Students are using standardized laptops in proctored exam environment
Internet can support uploading completed tests in the classroom
In-house tech support for hardware, software during exams
Recommendations
2

Slow roll-out – test in orientation, pilot in Spring courses and convert over Summer 2013. This means supporting ParScore simultaneously for awhile
Implement laptop policy in Fall 2012 or 2013
Assign 1 or more faculty and at least 1 IT person to Administrator status
Technology proctors are available at every exam
5-10 Back up computers with updated Softest and ready at exam time OR paper backups
Make a standing Technology Committee
Convert faculty computers to Windows 7
3