Test Writing: Moving Away
from Publisher Material
Bruce Butterfras, MS Ed., L.P.
Assistant Professor – Dept. of Emergency Health Sciences
University of Texas Health Science Center at San Antonio
Objectives
Explain the importance of an EMS
curriculum unique to your program
 Identify the steps involved in creating good
exams
 Create and evaluate exams for your courses

Why Customize My Course?

EMT/Paramedic National Standard Curriculum
From 1971 to 2000
 Contained specific knowledge and skills objectives


EMS Education Agenda for the Future (2000)
National EMS Education Standards
 “The EMS NSC, with their detailed declarative
material, limit instructor flexibility and the ability to
adapt to local needs and resources”

Publisher Resources



Most EMS textbooks come with instructor
resources
These often include chapter objectives, lesson
plans, PowerPoint presentations and test banks
to be used to construct exams
When selecting a textbook these additional
resources should be a strong consideration
Disclaimer: I have helped create Publisher material including PowerPoint presentations and test questions
Publisher Objectives




Publishers create objectives based on the
National EMS Education Standards
The National EMS Education Standards are
intentionally broad, so specific objectives vary
among the publishers
Publishers’ objectives are by necessity generic in
order to be applicable to all areas of the country
Often specific to their textbook as well
Publisher Lesson Plans


Publisher provided Lesson Plans are also
relatively generic in order to work for programs
around the country
These lesson plans usually focus on presentation
of material (that is found in the textbook) to
students in a standard (ie. lecture oriented)
educational model
Publisher Presentations



Publisher provided PowerPoint presentations
highlight the material presented in the textbook
This can allow students to follow along in the
textbook as the lecture progresses
Publisher provided PowerPoint presentations can
also be:
Reorganized to fit your way of teaching the subject
 Added to with additional materials from your own
research on the topic

Modifying Presentations

Publisher provided
PowerPoint presentations
can also be:
Deleted from if you feel there
is material included that
doesn’t need to be discussed
 Spiced up to make them
more appealing to students

Publisher Test Banks



Publisher provided test banks serve primarily to
determine if your students remember what they
read in the textbook
The questions almost always refer to a particular
page in the textbook
You need to test your students on what you
taught them and not just what is in the book
So What Do I Do?




To make your class more effective, you need to
modify publisher materials or create your own
Go through the objectives and modify them to
meet your needs. Add, delete, or edit objectives
to fit your particular course
Create your own presentations or edit those
provided to make them fit your course
Use other teaching techniques besides
lecture!
Can I write a good exam myself?


Finally create your own exams with questions
you create to test what your students have
learned
Start by asking what type of exam will this be
A short quiz to see if the students read their book
 A longer exam covering several chapters worth of
material
 A final exam testing mastery of the entire course

Where do I start?

Start by making a Blueprint for the exam
Identify what information will be tested
 Select an exam length appropriate for the
information and the purpose of the exam
 Determine how many questions will be included
from each section

Am I testing what I should?



Exams should test if the students can meet the
course objectives
So each test question should be tied to an
individual objective
Think about the objective as you write each
question and make sure the question you write
actually tests the objective
What question format is best?



There are many types of test questions
You need to choose the one (or ones) that best
fit your situation (and/or your students)
Essay Questions

What are the advantages and disadvantages of Essay
questions?
What about other types of
questions?

Short Answer / Fill in the Blanks
Name one advantage of a short answer question.
 Fill in the blank exams are good for testing ______
of presented materials.


True / False questions
Is this statement true or false:
 True / false questions are a favorite of students
because they are always clearly written and give
students a 50/50 chance of getting the answer right.

Can I use Matching questions?

Matching Questions
1. Matching questions make this
easy:
2. Matching questions are poor at:
3. Matching questions are good
for:
4. Matching questions are difficult
in this respect:
a. Creating
b. Evaluating
understanding
c. Grading
d. Testing recall
What about Multiple Choice?



Multiple Choice Questions
How many of you primarily or exclusively use
multiple choice exams? – Why?
Multiple choice questions are:
A. The best choice for testing recall of information
 B. Easy to write as a test of advanced understanding
 C. Capable of evaluating higher levels of thinking
 D. A poor choice for evaluating general knowledge

Writing Multiple Choice
Questions


Since this is probably the most popular type of
question used for EMS exams, let’s look at how
to construct a good MC question
Multiple choice questions have three parts
A ‘Stem’
 A ‘Correct answer’
 Several ‘Distractors’ (incorrect answers)

Multiple Choice Questions




Start with a Stem tied to one of the objectives
being tested
Try to create a stem that requires more than just
recall (memorization of facts)
Scenarios work well for these types of questions
Include enough information to choose the
correct answer and possibly additional irrelevant
information
Multiple Choice Questions

Write the Correct Answer
Tie it to the objective and make sure that there is
only one correct answer
 Avoid ‘All of the Above’ and ‘None of the Above’
 You can use ‘Both A and B’ type answers if you have
enough choices (usually more than 4)
 Consider writing a ‘feedback’ statement to explain
why this answer is correct

Multiple Choice Questions

Now write the Distractors
Make each distractor plausible
 Try to use answers that might be given by students
who do not have a good grasp of the material
 Make sure there is something in the distractor that
makes it incorrect
 Consider writing a ‘feedback statement’ telling why
this answer is not correct

Multiple Choice Questions

Distractors (continued)
Make sure both distractors and correct answer
sound correct (read each question out loud with each
answer option)
 Make all answer choices almost the same length


Shortest or longest answer choices often stand out
If necessary reword the stem to make the answers fit
 Avoid ‘NOT’, ‘EXCEPT’, etc. in stems if possible

Multiple Choice Questions

Now consider alphabetizing the answer choices



Prevents the tendency to write 2 distractors, the right
answer and then another distractor so the correct
answer is most likely “C”
Include enough questions to test the objectives
covered in the material
Somewhere within at least one exam there
should be a question for each course objective
Building the Exam

Use a variety of easy and difficult questions




Some recall, some higher level thinking
This will allow you to identify students at
different levels of understanding
It will also allow you to anticipate the grades
Aim for a mean grade of 85% and half should
be above and half below

If they score below a 70% they need remediation
Testing the Exam


When you finish your exam, test it out
Give the exam to your fellow program
instructors
See what grades they make & listen to their feedback
 Don’t expect their grades to be perfect – they didn’t
sit through your class


Consider also giving the exam to a few recent
graduates and listen to their feedback
Giving the Exam

Finally you should be ready to give the exam



Consider giving computerized exams if possible
Allow about one minute per question
Listen to the student’s feedback afterwards
If there is another way to interpret your
question, they will find it and point it out to you!
 Make adjustments to the questions based on student
feedback


Keep statistics on each question
Evaluating the Exam




Each time before you teach the class review
your exams and make adjustments
Keep your own ‘test bank’ of questions in case
you need to create a ‘make-up’ exam
It will also allow you to use different exams each
time you teach the class
Always be open to suggestions for improvement
Conclusion



Publisher provided materials are a great starting
point, but don’t rely on them exclusively for
teaching your class
Multiple choice questions can make great exams
so practice making them whenever you can
Constantly reevaluate your courses and strive to
make them better. The life your student may
eventually save may be yours!
Selected References

National EMS Education Agenda for the future:
http://www.nhtsa.gov/people/injury/ems/edagenda/final/agenda6-00.htm

Cason, D. McKenna K. Foundations of Education: An EMS Approach 2nd
edition Clifton Park, New York; Delmar Cengage Learning (2013) Print

My Contact Information:


Butterfras@uthscsa.edu
Office phone - 210-567-8719