Designing Good Tests:
Item Analysis is Part of the
Equation!
Molly Herman Baker, Ph.D.
Black Hawk College
Planning Tests




Determine amt of time, location, level of
difficulty desired, resources available
during test, and % weight of test compared
to total # of assessments
Based on objectives, determine what is
important to know, not “nice to know”
Based on amt of time, estimate number of
questions possible (see Test Writing
Basics handout for estimates)
Prepare specifications chart
Specification Chart Basics

Specifications chart:
 Type
and number of items for
each content area or
objective.
 Type and number of items for
each type of HOT skill
(HOT=discipline-specific;
Bloom often selected, but
others may be more relevant
for your discipline)
Sample Specifications Chart #1
Obj 1
Obj 2
Explain
2
3
Apply
5
Predict
1
Synthesize 1
1
Total
10
4
2
2
1
Obj 4
1
5
Analyze
Evaluate
Obj 3
5
2
5
6
15
9
Sample Specifications Chart #2
MC
TF
Obj 1
(know)
5
5
Obj 2
(apply)
10
5
Obj 3
5
(compre)
Obj 4
(apply)
Matching
Total
5, 2-pt.
1 5-pt.
25 pts
10
5
25
Essay
15 pts
Obj 5
(predict)
Total
Short
Ans.
10
10
1 5-pt
20 pts
5, 2-pt
1, 10pt
25 pts
1, 5pt
1, 10pt
15 pts
20
30
100
Writing/Choosing Good Items


Write good objectives, using verbs that
indicate desired performance; share
with students
Write/select test items based on these
verbs;
 See
“Comparison of Bloom and Ebel’s
Taxonomies” handout for samples


See “Test Writing Basics” handout
Discuss strategies for selecting test
bank items
What about HOTS?



HOTS (Higher order thinking
skills)=application, analysis,
prediction, synthesis, evaluation or
alternative taxonomy relevant to your
discipline
Important: practice HOT in class
(critical thinking, creative thinking,
problem-based learning, case-based
learning, scenario-based lng)
Important: provide guidance, practice
& feedback on preparing for and
answering HOT exam questions
Item Analysis to Improve Tests
(Scantron QuickScore II: QC1-106)

# correct and # missed per item
 What

# who answered each MC distractor
 What

does this tell you?
Correlation between answers and overall test
performance (Discrimination Index/Pt. Biserial)
 What

does this tell you?
could this tell you?
Questions with poor item analysis score
 Discuss
with students to determine item problems
Testing Services at BHC




Administer individual tests to students in
Study Unlimited or online classes
Administer individual tests to students
taking make-up exams
Score and return Scantron tests if the key
is provided by the instructor (campus mail
or file drawer). More Scantron scoring machines in Bldg 2
and 3.
Email ilctesting@bhc.edu or submit ILC Testing Form to
make arrangements for online testing (course, instructor,
exam#, student(s), deadline for completing, time limit,
allowable notes or books, test password, what to do with
hard copies)
QuickScore II Tips



Item analysis software in QC1-106 (ILC)
Use score sheet #101864-ERI-L only
Options for output:
 Save
electronic analysis reports in PDF
format to your H: drive ( or floppy, Flash drive)
 Print reports (bring paper from your dept)

Laminated instructions in 1-106 binder (or
download from
http://facweb.bhc.edu/tlc/tech/stepbystep/quickscore.html )