STRATEGIES
TO EVALUATE COMPETENCE
Christina B. DeBiase, EdD
WVU School of Dentistry
STEPS IN EVALUATION
• Define the objectives of the
learning activity – a test should
evaluate the degree of student
learning
• Develop a test blueprint
TEST BLUEPRINTS
• How much emphasis should be given to
each objective?
• What type of question would best assess
the students’ attainment of objectives?
• How difficult should the questions be to
adequately discriminate the stronger
students from the weaker students?
• What considerations should be given to
test format and administration?
Basic Forms of Evaluation
1. Objective – data gathered
represents some predetermined
standard
2. Subjective – no prior standard
of performance has been
specified
METHODS FOR EVALUATING
STUDENTS
• TESTS
• CBL / PBL SCENARIOS
• WRITTEN & ORAL
PRESENTATIONS
• SIMULATIONS
• PRACTICUMS
• PORTFOLIOS
• OSCE
• ETC.
OPTIONS FOR TEST ITEMS
•
•
•
•
•
•
•
•
Stand-alone multiple-choice items
Case-based multiple-choice items
True/False
Fill-in the blank/ Short-answer/Completion
Essay
Listing
Oral
Rating sheets
COMMON TEST-WRITING
ERRORS
MULTIPLE-CHOICE
COMPOSITION
• STEM: SET UP THE PROBLEM OR SET OF
CIRCUMSTANCES; SHOULD HAVE AT
LEAST ONE VERB; CAN BE WRITTEN IN
QUESTION FORM OR AS A PARTIAL
STATEMENT
• DISTRACTORS: ONE CORRECT OR BEST
RESPONSE (tests higher level learning) AND 3-4
ALTERNATIVES; ALL OF THE ABOVE OR
NONE... ARE NOT SPECIFIC OPTIONS OR
SOLUTIONS AND SHOULD BE AVOIDED
KEEP IT SIMPLE!
• DO NOT PUT EXTRA, UNNECESSARY
INFORMATION IN THE STEM: Mary, a
16-year old female with cerebral palsy,
presents with a bruxing habit. Bruxism is
the rhythmic grinding of teeth that can
cause overdevelopment of a muscle of
mastication. Which of the following
muscles may be overdeveloped in a patient
with bruxism?
POSITIVE/NEGATIVE STEMS
• POSITIVE - Usually more effective because the
item tests what examinees know to be correct.
Which of the following antibiotics is
recommended for SBE coverage?
• NEGATIVE - Sometimes appropriate to
distinguish the incorrect response among several
correct alternatives.
All of the following congenital cardiac defects
require antibiotic premedication prior to dental
procedures except one. The EXCEPTION is ___?
RULES TO FOLLOW WHEN
CREATING ALTERNATIVES
• INCLUDE REPITITIOUS WORDS IN THE
STEM TO AVOID REPEATING WORDS IN
EVERY ALTERNATIVE.
• ALTERNATIVES SHOULD BE SIMILAR IN
CONSTRUCTION AND OF APPROXIMATELY
EQUAL LENGTH
• DISTRACTORS SHOULD BE PLAUSIBLE &
MAY REPRESENT COMMONLY HELD
MISCONCEPTIONS ABOUT A SUBJECT
• AVOID RESPONSE PATTERNS
The correct answer is longer or
more specific than the distractors:
All of the following determine the latent
period’s degree of biologic effect except one.
The EXCEPTION is ___?
a. The exposure time
b. The type of radiation
c. The type of tissue irradiated
d. The ability to cause substances to
fluoresce *
HINT- Put “The” in the stem!
ADDITIONAL PRACTICES
TO TO AVOID WHEN
DESIGNING
ALTERNATIVES FOR
MULTIPLE-CHOICE
QUESTIONS
The alternatives are not mutually
exclusive (overlapping alternatives);
if numbers, always list in ascending or
descending order & if ranges, intervals
should be equivalent:
Which of the following I.Q. ranges is
descriptive of a person with moderate
mental retardation?
a. 50-70
b. 35-50
c. 20-35
d. <20
Distractors are not plausible:
Which of the following primary molars is
a smaller replica of the permanent first
molar:
a. First
b. Second
c. Third
Grammatical clues in the stem
suggest the correct answer:
Which of the following drugs is an
antihypertensive agent?
a. Procardia
b. Prednisone
c. Cyclosporine and Imuran
d. Amoxicillin and Erythromycin
Information in the stem is also
found in the correct answer:
Which of the following forms of diabetes
requires insulin for effective
management?
a. Insulin - dependent
b. Non-insulin dependent
c. Hyperglycemia
d. Hypoglycemia
CAUSE AND EFFECT ITEMS
• (Statement)Prior to cardiac surgery, a patient
should receive a dental cleaning, (Reason)
BECAUSE in a plaque-infected oral cavity the
likelihood of bacteremia is greatly increased.
• a. Both the statement and reason are correct and related.
• b. Both the statement and the reason are correct, but
NOT related.
• c.The statement is correct, but the reason is NOT.
• d.The statement is NOT correct, but the reason is
correct.
• e. NEITHER the statement NOR the reason is correct.
PAIRED TRUE & FALSE ITEMS
• I. Medical record entries should be completed in
non-erasable ink. II. If a mistake is made when
making a notation, the incorrect information
should be covered with correction fluid.
• A. Both statements are true.
• B. Both statements are false.
• C. The first statement is true, the second is false.
• D. The first statement is false, the second is true.
ADVANTAGES OF MULTIPLECHOICE ITEMS
• Flexible
• Measure student’s knowledge in a subject area
without the influence of writing skills, spelling, or
grammar
• Scored quickly
• Content of course can be adequately sampled
• Comparable results can provide diagnostic
information about class
• The best answer does not have to be the only
correct response, but should be defensible
• Item- analysis is very feasible
DISADVANTAGES
• Questions are difficult to develop with
sufficient plausible alternatives
• Cannot measure higher level learning unless
case-based; stand-alone items usually
measure factual recall only
• Guessing the correct answer can make the
test ineffective
GUIDELINES FOR TRUE &
FALSE STATEMENTS
AVOID:
• Negatively worded statements
• Testing trivial detail
• Specific determiners that act as clues –
usually, never, always, all
GUIDELINES FOR MATCHING
ITEMS
• Specify whether alternatives can be used
once or more than once
• It is recommended to provide more
alternatives than premises, but the number
of alternatives should not greatly exceed the
premises
• List premises or phrases on the left and
alternatives on the right
GUIDELINES FOR COMPLETION
QUESTIONS
• Include sufficient information to tell
student what type of answer is
expected
• Omit as few words as possible
• Use a uniform blank space
• Avoid grammatical clues
• Place the blank toward the end of the
sentence
GUIDELINES FOR ESSAY
QUESTIONS
• Cover the objectives
• Prepare a structured response with elements students
should include for credit
• Give clear directions so that the student knows what is
expected
• Ensure the test is short enough to allow time to complete
the essay(s)
• Avoid testing trivial details
• Avoid giving students the option to answer 5 of 7 essays ,
for example
• Grade by reading all students’ responses to one essay
before going on to the next question